The FEEL compiler should keep its current foundation: a semantic intermediate representation, a direct interpreter, and an optional BEAM backend. The strongest improvements are explicit contracts for observable effects, bounded preparation work, and targeted changes to generated control flow. A new native backend, general SSA framework, or adaptive JIT would impose substantial costs without an established need.
This assessment compares compiler designs to decide which changes are worthwhile for FEEL. It covers Java, Clang/LLVM, Rust, Go, OCaml, Elixir/BEAM, C# and F# on .NET, V8, Elm, CEL, and Dhall. The type-system assessment covers correctness and type design in detail; the existing architecture guide remains the module reference.
The implementation review was refreshed on 19 October 2026. The compiler comparisons were first researched on 17 October; their architectural lessons are not a version support matrix. Older performance figures are workload-specific historical measurements, not fresh results or general speed guarantees. Use the benchmark scripts for the current code.
Implemented foundations include shared BEAM containment for complete public preparation, compilation and evaluation; conservative transfer checks; compact FEEL/DMN payloads; per-call DMN plan reuse; and compilation-time evidence for recursive callees. Defaults are five seconds and 128 MiB. Trusted calls can opt out with sandbox: false; generated BEAM code requires that opt-out because module-name atoms and loaded code have VM-wide lifetime. See the security model.
The generation-limit ordering fix (C0) is complete: forms are emitted under a fixed placeholder and checked before a distinct module-name atom is created. The reproduced function-signature, projection, structural-reader, rule-certainty and unary-test-lowering failures are also repaired, including the function-reader and recursion completeness follow-ups. Preserve those regressions; the broader proof, effect and witness audits in the type-system assessment remain open. DMN caller typing, linked-closure caching and diagnostic origins are implemented behavior, not new roadmap tasks.
DMN also derives a business knowledge model's caller result from its function item's output (DD-33), including aliases and imports, while decision-service results follow their separate rule. Literal signatures are stable across tiers and optimization, runtime argument diagnostics are built from values, and DMN table proofs keep their uncertainty. The type-system assessment states those contracts; the broader fact/effect audit is open work.
Full analysis of syntax containing function literals now makes a bounded kind-only signature pass before refining the executable IR. The historical preparation measurements below predate that pass and should not be treated as current costs for function-heavy workloads.
Services derive named output contexts, honor their own result conversion and keep derived recursive runtime signatures. Guarded item graphs check complete finite data while analysis uses bounded upper projections, including post-conversion parameter specialization.
Unary-test lowering now preserves dynamic matching's null contract (D-40, D-209): a nullable scalar expression that reads ? does not turn a null result into implicit equality with null. -? and an equivalent conditional therefore agree. Explicit null equality and null literals keep ordinary equality. The remaining-reader audit adds these lowering checks alongside result-check regressions.
Resource-boundary follow-up: fresh module names have a VM-lifetime admission quota (D-223). DMN shares call, iteration and diagnostic counters, snapshots its clock once, and bounds model units (DD-45). These checks complement shared workers (D-221); they do not complete early emission limits, concurrency admission or the fact/effect audit proposed below.
Read in order for the design argument, or go directly to the decision you need:
- Current architecture — what exists and which semantics it preserves.
- Compiler comparisons — grouped by architectural model, with a lesson for FEEL.
- Recommended changes — priorities, tradeoffs, and acceptance criteria.
- Changes to defer — the evidence that would justify larger investments.
- Delivery roadmap and validation — effort, dependencies, the first implementation step, and evaluation of the work.
1 The current architecture
1.1 Preparation and execution
The public boundary is valuable: Feel.prepare/2 performs work independent of input data, while Feel.run/3 evaluates the prepared expression against bindings.
FEEL source and scope
→ lexer and parser
→ source AST
→ analysis, resolution, and lowering
→ semantic FEEL IR
→ FEEL optimization
├─ interpreter executable retaining the IR
└─ BEAM emitter
→ Erlang abstract forms
→ Erlang compiler
→ BEAM module
→ loader and, on a JIT-enabled runtime, BeamAsm
→ native codeThe parser uses scope information to disambiguate FEEL names. The analyzer resolves bindings and calls, decodes literals, computes facts, orders context entries by dependencies, and inserts declaration checks. The optimizer transforms the same IR. Both backends use shared runtime operations for language semantics. See Analyzer, IR, Optimizer, and Backend.
The DMN layer has a corresponding prepared-model boundary. A compiled model holds prepared FEEL units and their dependencies; evaluation runs the required units in dependency order. That is already the right level at which to reuse model preparation. See Dmn.Compiled.
1.2 The IR remains close to the language
The IR is a tree of FEEL operations with resolved references and metadata. It keeps concepts such as contexts, filters, unary tests, functions, and verification nodes. This makes it useful both for direct interpretation and for transformations that need FEEL semantics to remain visible.
It is not a CPU-oriented instruction language. Register allocation, stack layout, instruction selection, and native-code emission are delegated downstream. Adding those responsibilities to this project would duplicate capabilities already supplied by Erlang/OTP.
The interpreter itself runs on BEAM and may execute as native code. It still interprets the FEEL program by walking its IR. The BEAM backend removes that walk by generating code for the particular expression. These are different execution strategies for FEEL even though both ultimately use the same host runtime.
1.3 Execution preserves more than a value
A semantic error commonly produces null and a diagnostic in lenient mode; strict mode stops at its first error. Diagnostic order and source locations therefore matter. A diagnostic now carries a host-supplied origin so a function body's span can be mapped through its own unit instead of its caller's text (D-201). A function written in an expression keeps the origin of its run; hosts must label any function they bind that was made by a run without an origin. Calls can also depend on the clock or host functions, and evaluation is subject to resource limits.
An optimization must account for these observations. Computing the same final value is insufficient if a transformation suppresses an error, repeats an external call, or changes which strict failure is encountered first. Retained call, iteration and per-value limits have an explicit policy; there is no general step or cumulative-allocation quota to preserve. Wall-clock and heap-policy termination can vary with optimization and VM scheduling.
The optimizer already takes considerable care here. For example, it folds a constant computation only when its evaluation produces no diagnostics. However, the reasons that an operation may be removed or must remain are spread across specialized predicates and cases. See Optimizer and Identity.
1.4 Existing strengths to preserve
The project already has substantial compiler infrastructure:
- A shared runtime and two backends, with optimized versus unoptimized comparisons.
Feel.Paranoid, which checks intermediate results and compares execution paths.- Benchmarks separated into parse, analyze, optimize, and compile, with scaling checks and compilation break-even measurements.
- Generated-code chunking to avoid excessively large Erlang functions.
- A guarded scalar fast path with interpreter fallback.
- Generated-module deduplication, serialized loading, soft retirement, and an internal generated-code cache stamp.
- Per-run linked-closure reuse and host-supplied diagnostic origins across calls.
- Model-chain audits, rejected-declaration sweeps and generated mutation experiments.
These are implemented foundations, not proposed additions. See Paranoid, benchmarks, Chunks, Scalar, and Loader.
The loader also deliberately passes :no_ssa_opt to the Erlang compiler. Its recorded rationale is that the general emitted code largely calls runtime functions: disabling those SSA optimization passes substantially reduced compilation time with little execution difference on the measured workloads. Other compilation and code-generation work still occurs. Future changes to the emitted code should remeasure this choice rather than assume that either enabling or disabling every optimization is always best. See compiler options.
2 Comparison with other compilers
The comparisons are grouped by the architectural question they answer: ahead-of-time lowering, execution on managed runtimes, and embedded or restricted language boundaries. The diagrams show logical dataflow; real compilers interleave stages and process units at different times. Java means the usual OpenJDK/HotSpot configuration, Rust means rustc with its usual LLVM backend, Go means the standard compiler, and JavaScript uses V8 as a concrete example.
2.1 Ahead of time compilers
These implementations illustrate when a source-oriented representation should give way to one designed for optimization or execution.
Clang and LLVM
C or C++ source
→ preprocessing, parsing, semantic analysis
→ Clang AST
→ LLVM IR and optimization
→ target-specific machine representations
→ instruction selection and register allocation
→ object files and linkingClang owns the source-language frontend. LLVM supplies reusable optimization and code generation. LLVM IR expresses explicit control flow, arithmetic, calls, and memory operations after much of the source-language structure has been lowered. Link-time optimization can keep IR until linking to enable optimization across compilation units. Clang toolchain, LLVM code generator.
Current Clang development also includes an optional ClangIR route based on MLIR. It keeps higher-level structure for transformations before LLVM lowering. ClangIR.
Lesson for FEEL: keep language concepts until their useful transformations are complete, then reuse a backend. The Erlang backend already supplies that division. Clear pass contracts and analysis invalidation are useful ideas to borrow; LLVM's whole infrastructure is not required to implement them. LLVM pass management.
Rust
Rust source
→ parsing, macro expansion, name resolution
→ HIR (high-level IR) and type checking
→ THIR (typed high-level IR)
→ MIR (mid-level IR), borrow checking, and transformations
→ monomorphization and LLVM IR generation
→ LLVM optimization, native code, and linkingRust uses several language-specific representations before LLVM. MIR makes control flow and operations explicit for checking and transformation. Monomorphization generates concrete instantiations of generic code. Internally, queries request dependent computations and support caching and incremental compilation; rustc is not simply one global sequence of passes. Rust compiler overview.
Lesson for FEEL: introduce another representation when it enables specific reasoning that the existing one makes difficult. Reuse work at a meaningful boundary. Neither multiple IRs nor a query engine is an architectural goal by itself.
Go
Go source
→ parsing and type checking
→ compiler IR and Unified IR machinery
→ inlining, devirtualization, escape analysis
→ lowering
→ generic SSA and optimization
→ target-specific lowering and register allocation
→ object code and linkingGo's standard compiler has its own backend. SSA, or static single assignment, gives each computed value a distinct definition, making data dependencies easier to analyze. Earlier passes keep language structure; later passes simplify low-level computations and select machine operations. Some language operations become runtime calls. Go compiler architecture.
Lesson for FEEL: explicit, focused phases can serve a substantial language. Keep transformations at the level where their invariants are understandable, and measure preparation cost alongside execution cost. There is no need to adopt another project's entire optimization framework.
OCaml
OCaml source
→ parsing and type checking
→ lambda representation
├─ bytecode generation → ocamlrun interpreter
└─ native optimization and code generation → object files and linkingOCaml shares frontend work while providing bytecode and native compilation paths. Its lambda representation lowers source constructs and discards much of the source type structure; pattern-match compilation is a concrete use of this boundary. The two paths make different tradeoffs in compilation cost and execution strategy. OCaml compiler backend.
Lesson for FEEL: a shared semantic frontend and different execution paths are a coherent design. Add a lower representation when it makes a particular transformation easier to express and validate. The long-branch problem is a suitable experiment; the existence of OCaml's lambda form does not by itself justify another FEEL IR.
Type erasure also needs a semantic boundary. FEEL must keep descriptors required by runtime instance of and checks, even when temporary inference data can be discarded. This reinforces the separation between analysis facts and executable metadata discussed in the type-system assessment.
2.2 Managed runtimes and compiler services
These implementations distinguish source-language compilation from the runtime that executes its output. They also show that reusable compiler services and runtime optimization are separate architectural choices.
Java and HotSpot
Java source
→ parsing, resolution, semantic and flow checks
→ lowering
→ JVM bytecode in class files
At runtime
loading and verification → loaded bytecode
├─ interpretation and profiling
├─ C1 → quickly compiled native code
└─ C2 → more heavily optimized native codeJava divides compilation between javac, which handles source-language semantics, and runtime compilers. Tiered compilation balances quick startup against more expensive optimization of frequently executed code; individual methods need not visit every tier. Runtime profiles guide compilation and execution choices. C2 compiles from bytecode and associated information, rather than translating C1's machine code. javac phases, HotSpot tiered compilation.
Lesson for FEEL: compilation cost belongs in the execution strategy. A faster generated function is not necessarily a faster overall workload. Reuse and frequency should determine whether compilation is worthwhile.
Java also has an ahead-of-time native deployment route through GraalVM Native Image. That alternative illustrates how deployment requirements can justify a different architecture; it is not a prerequisite for a compiler that serves a VM well. Native Image.
Elixir and Erlang
Elixir source
→ AST
→ macro expansion and compile-time execution
→ Elixir analysis and lowering
→ Erlang abstract forms
→ Core Erlang and BEAM SSA
→ BEAM instructions and module
→ load-time native compilation on a JIT-enabled runtimeElixir performs its language-specific work and then shares the Erlang compiler. Macros execute during compilation and transform ASTs. Erlang contributes further representations and optimization passes. Elixir macros, Elixir's Erlang backend, Erlang compiler passes.
BeamAsm generates native code when BEAM modules load on supported JIT configurations. This is different from waiting for hot functions and repeatedly optimizing them using execution profiles. BeamAsm.
Lesson for FEEL: this is the closest architectural relative. Optimize the shape of the Erlang forms and the use of runtime operations. Maintain a clear distinction between FEEL preparation and the host VM's loading and execution responsibilities.
C# and F# on .NET
C# source → Roslyn frontend and lowering → CIL assembly
F# source → F# frontend and lowering → CIL assembly
CIL and metadata
├─ CLR JIT at runtime → native code
└─ Native AOT at publish time → native executableC# and F# have separate compilers and source-language type systems, while their usual .NET compilation paths share CIL and the runtime. F# is not a Roslyn frontend: its compiler implements its own parsing, type checking, optimization, and IL emission. Native AOT is a deployment alternative that compiles IL at publication rather than through a JIT during execution. Managed execution, F# compiler organization, Native AOT.
The distinctive architectural lesson is Roslyn's service model. Its APIs expose syntax, symbols, semantic analysis, diagnostics, and emission. Editor features use the compiler's information, and the compiler layer is independent of Visual Studio. Roslyn API model.
Lesson for FEEL: build focused inspection services around the same prepared analysis used for execution. A caller should be able to locate a binding, inspect a node's type and source origin, and understand why a check remains or was removed. This extends the existing Feel.Expression accessors; it does not require a workspace engine or a second analyzer for editor tooling.
The type systems also illustrate why metadata contracts matter. C# nullable-reference annotations guide warnings without changing runtime behavior; F# units of measure are checked statically and erased. FEEL's runtime type tests and optimizer have different obligations. An advisory fact must not silently become permission to remove a runtime check. C# nullable analysis, F# units of measure.
The shared-runtime lesson supports continued use of BEAM. Domain features such as units belong in a separate language-design proposal; adding another native backend would not provide them.
JavaScript and V8
JavaScript source → AST → Ignition bytecode
├─ Ignition interpretation and feedback
├─ Sparkplug → baseline native code
├─ Maglev → optimized native code
└─ TurboFan / Turboshaft → peak optimized native codeV8 balances startup with performance on hot code. Its current architecture includes Turboshaft's control-flow-based optimizing representation. The tiers are available execution strategies, not a mandatory sequence for every function. V8 architecture.
Runtime feedback supports guarded specialization. If an assumption fails, execution can deoptimize and later recompile. Multiple tiers reduce the gap between cheap initial execution and expensive peak optimization. Maglev and V8's tiers.
Lesson for FEEL: bounded specialization with a reliable fallback is useful. The scalar backend already applies a simpler form of this idea. Reconstructing arbitrary execution state, repeatedly recompiling, and maintaining many speculative versions would be a much larger undertaking.
2.3 Embedded and restricted language designs
These comparisons focus on host boundaries, observable outcomes, and resource guarantees. Elm targets a general JavaScript engine, CEL embeds an expression evaluator, and Dhall emphasizes configuration normalization; their useful similarities to FEEL are therefore different.
Elm
Elm compiles to JavaScript, which is then executed by a JavaScript engine. Its Maybe and Result types make absence and failure explicit in ordinary program data. These are separate choices: the target provides execution infrastructure, while the frontend and libraries define the source language's guarantees. Elm JavaScript integration, error handling.
Lesson for FEEL: generated code must preserve the source language's outcome distinctions even when the host runtime offers different error mechanisms. FEEL's runtime already carries values, diagnostics, and strict failures. Make those distinctions explicit in transformation contracts and explanations, so a null returned normally cannot be confused with a null accompanied by an error.
Replacing FEEL null semantics with an Elm-style result type would change the language. The worthwhile architectural work is clearer outcome contracts and source-oriented diagnostics within the existing semantics. Elm's application-level Model–View–Update architecture is a separate topic from this compiler comparison.
Common Expression Language
CEL source and declared environment
→ parse
→ check names, calls, and types
→ checked AST with resolution metadata
→ repeated evaluation with variable and function bindingsCEL is an embedded expression language. Its documented lifecycle separates parsing and checking from repeated evaluation; the host defines the available variables and functions, and runtime bindings must match the compilation environment. Its checked representation can be stored for reuse. CEL overview.
Lesson for FEEL: this is a close comparison for prepare/run and the embedding API. Treat a prepared artifact as a program with an environment contract. Different binding values can reuse it when they satisfy the applicable assumptions; changes to declarations, extension contracts, dialect, or preparation options can require a new artifact.
FEEL already fixes the dialect and extensions at preparation and rejects attempts to override them during run. Preserve that behavior and make the remaining host assumptions inspectable before adding cache machinery. See the run boundary and prepared expressions.
A stored AST is not automatically a portable or indefinitely compatible artifact. Persistence would need an explicit format and compatibility policy. CEL strengthens the case for disciplined preparation reuse, without establishing a need for a custom bytecode VM or for persistent FEEL artifacts now.
Dhall
Dhall is useful as a comparison for bounded configuration evaluation rather than machine-code generation. Its language design guarantees termination of well-typed evaluation, but its documentation explicitly notes that a finite computation can still take impractically long. Dhall safety guarantees.
Lesson for FEEL: type correctness, termination, and an acceptable resource cost are different guarantees. Even a restricted expression language needs operational limits on preparation and execution. Host callbacks and the downstream Erlang compiler also have costs that an expression-level counter does not completely control.
This supports keeping worker containment for preparation and execution. It does not justify imposing Dhall's restrictions on FEEL or presenting a logical work limit as a hard memory or wall-clock bound for the entire host process.
2.4 What the comparisons change
| Design question | Useful comparisons | Decision for this project |
|---|---|---|
| Where should native-code responsibilities live? | Clang/LLVM, .NET, Elixir/BEAM | Continue using the Erlang toolchain |
| When is another representation justified? | Rust, Go, OCaml | Require a concrete transformation or invariant that benefits |
| When should preparation become more expensive? | HotSpot, V8, OCaml's execution paths | Measure reuse, fallback rates, and total cost |
| How should tools inspect a program? | Roslyn and the F# compiler services | Extend accessors into focused queries over the prepared analysis |
| What makes reuse semantically valid? | CEL | Specify preparation assumptions and environment compatibility before caching |
| Which outcomes must transformations preserve? | Elm, C# and F# metadata distinctions | Separate values, diagnostics, runtime descriptors, and facts that justify transformations |
| Does a restricted language need resource limits? | Dhall | Keep whole-operation containment and local construction/analysis bounds |
These are lessons for particular boundaries, not a ranking of compilers. ASTs, bytecode, and SSA are not mutually exclusive categories: an AST is an IR, bytecode can be both an execution format and compiler input, and SSA describes definitions and dependencies. More representations are justified when their benefits repay lowering, validation, metadata preservation, and debugging costs.
The additional comparisons sharpen the existing priorities. Environment contracts belong in P1 correctness work, focused inspection APIs in P2 tooling work, and cache implementation remains P3 until reuse is demonstrated. They do not establish a new requirement for SSA, a native backend, or an adaptive JIT.
3 Recommended changes
P1 items establish reliable boundaries before broader optimization. P2 items address demonstrated costs or improve maintainability and explainability. P3 items depend on workload evidence. A deployment dominated by long generated expressions may reasonably promote the branch-lowering work to P1.
The C identifiers refer to the scoped implementation slices in the effort and dependency table. They separate work that can ship independently within a broader recommendation.
| Priority | Change | Expected benefit | Main tradeoff |
|---|---|---|---|
| Complete | Delay module-name creation until generation limits pass (C0) | Refused expressions do not consume distinct permanent module names | Full forms are still allocated before refusal |
| P2 | Early emission refusal and preparation-cost reductions (C1–C2) | Less work before refusal and faster bounded preparation | Counting overhead and conservative precision fallback |
| P1 | Effects, preparation contracts, and IR validation (C3–C5) | Auditable assumptions and transformation preconditions | Metadata maintenance and conservative answers |
| P2 | Flatter lowering of long branch chains (C6) | Lower BEAM compilation cost | More helpers and possible call overhead |
| P2 | Extract analyzer responsibilities behind stable interfaces (C7) | Easier review and local reasoning | Refactoring risk without immediate speedup |
| P2 | Semantic inspection APIs and bounded execution explanations (C8–C9) | Reusable tooling and useful decision traces | API scope, provenance, and trace-volume management |
| P3 | Prepared-artifact caching and backend admission policy (C10–C11) | Reuse preparation and bound operational cost | Ownership, identity, and invalidation complexity |
| P3 | Additional guarded fast paths (C12) | Faster common operations | Larger specialization surface and fallback testing |
3.1 Bound preparation and generation
The smallest resource-lifecycle fix, C0, is implemented. Beam.compile/2 emits candidate forms under a fixed placeholder, checks the exact statement limit, and only then obtains the distinct module name from the IR and runtime cache stamp. A refused expression creates no distinct module-name atom. This does not bound emission itself or the atoms of accepted compilations; C1 and C11 remain open. See compilation order and module naming.
Public interpreted preparation already has a whole-operation deadline and heap policy. C2 is a performance improvement within that boundary: profile repeated analysis and specialization, then reduce duplicated work through preparation-local memoization or targeted optional-precision caps. Local state needs an explicit lifetime and semantic key. Do not reintroduce a general work counter or cumulative allocation budget. Remaining preparation costs.
Separate optional precision from required checking. Exhausting an optional specialization budget should keep a conservative summary. Exhausting the ability to complete required semantic validation should produce an explicit limit error. A budget must not silently skip validation or allow partially checked IR to reach a backend.
The emitter should account for generated work while constructing it. Currently max_generated_statements is checked after forms have been emitted. Preserve a final exact check, but reject excessive generation before allocating the entire result. Include large generated constants in the resource model: statement count alone is not a complete memory bound. See BEAM compilation and documented generation limits.
The existing emitter counter numbers variables; it is not the final statement count. Nested functions, branch bodies, and chunking affect the relationship. C1 must establish a safe counting rule before using a partial count to reject an expression. An early rejection also cannot report an exact final total that was never computed: preserve exact diagnostics where available and specify any new lower-bound diagnostic explicitly. This is why C1 is a separate change from the smaller C0 ordering fix.
Tradeoff and acceptance criterion. Memoization and local precision caps add overhead and can reduce inferred precision. Measure ordinary and repeated-analysis shapes, preserve required validation, and check cleanup and accepted semantics. The worker stays the outer resource boundary for interpreted preparation; a local cap is not another memory guarantee. C1 applies to trusted BEAM emission, whose global atoms and modules outlive a worker. Preparation callbacks and constant evaluation stay subject to the host-capability limits described in the security model.
3.2 Make analysis and execution contracts explicit
For each fact used to change execution, define what it establishes, which preparation or host assumptions it depends on, and which transformations may use it. Distinguish advisory diagnostics from conservative value facts, runtime signature metadata, and accepted proofs. C# nullable analysis and F# erasure illustrate why these roles cannot be inferred from the presence of a type annotation. The type-system assessment states the current contracts for signatures, projection, rule certainty, tiers and DMN scope. Completing the wider fact and consumer audit remains a dependency described in the open obligations.
Introduce a small summary describing whether evaluating a node may report diagnostics, call external code, or depend on runtime state. Keep retained engine-limit behavior and worker ownership explicit rather than folding them into a single pure? flag. Unknown operations receive conservative summaries.
For example, if a / b = null then 0 else 1 can return a number while reporting division by zero, and strict execution can stop before reaching the result. An optimizer cannot infer silence from the final value. The existing effect-analysis note identifies this distinction.
Transformation APIs should answer specific questions: may this computation be removed, repeated, reordered, or replayed from the start? These questions can have different answers. A summary indicating no diagnostics does not prove termination, absence of host effects, or preservation of resource limits.
Make the preparation/execution environment contract explicit as part of the same work. Dialect and extensions are already fixed at preparation; record which declaration and callable assumptions must remain valid when bindings change. Where the host supplies a trusted contract, expose that obligation rather than imply it has been verified. This is a contract audit and consolidation, not a claim that the current API permits arbitrary option changes.
Add an IR validator at important development boundaries. It should check node shape, required metadata, binding references, and backend preconditions. Rewrites must update derived facts or invalidate them for recomputation. The validator establishes structural consistency; it does not prove semantic equivalence.
Tradeoff and acceptance criterion. A shared summary can become a shared source of mistakes. Start conservatively and replace existing special cases gradually. Differential tests must cover diagnostic order, source locations, strict failures, host-call counts, and the documented resource policy. Test that changing runtime values within the documented contract supports reuse, while an incompatible preparation setting is rejected or requires preparation again. Proof-dependent rewrites must keep checks when their premises are unavailable. These changes are prerequisites for broader code motion or common-subexpression elimination, not a reason to enable those transformations immediately.
3.3 Flatten long generated branch chains
Earlier development probes identified superlinear preparation cost for long branch chains, with much of the time in :compile.forms. Deeply nested generated case expressions are a candidate cause. Reproduce the scaling and profile the current emitter before choosing a transformation; the performance backlog states the measurement requirements.
Prototype a backend-specific transformation that divides long chains into bounded helper functions or otherwise reduces nesting. Preserve lazy branch selection, evaluation order, live values, and diagnostic spans. Existing straight-line chunking is a useful foundation, but it does not solve every nested-control-flow shape. See branch emission and Chunks.
Tradeoff and acceptance criterion. Helpers can add calls and argument passing; too much splitting can make ordinary code slower or larger. Compare preparation scaling, peak memory, generated size, and execution across several branch lengths and taken-branch positions. Adopt the change only if it addresses the measured bottleneck without unacceptable runtime regressions.
Keep this inside BEAM lowering initially. Introduce a small execution-plan IR only if several transformations need the same explicit sequencing, branches, and live-value information. That decision should follow the prototype, not precede it.
3.4 Extract responsibilities without multiplying traversals
The analyzer combines resolution, inference, lowering, constant evaluation, specialization, diagnostics, and memoization in one large module. Extract cohesive services while keeping an analyzer facade and the existing scope-aware traversal.
Possible boundaries are constant evaluation, bounded call specialization, binding management and declaration-check construction. Both analysis and optimization legitimately need constant information, so share its implementation rather than require all constant evaluation to occur in one late pass.
Memoization should have an explicit owner and semantic key. Process-local storage is not inherently wrong, but its lifetime, cleanup, and dependencies should be controlled by the preparation context. A small module interface may be sufficient; a global dependency-query framework is not necessary.
Condition facts currently use closures. Consider inspectable fact terms only when a concrete consumer needs them: joint facts over several paths, explanations, or measured folding and memoization improvements. A small shared graph of restrictions, conjunctions, and disjunctions is a possible representation. Canonical syntax alone does not establish semantic equality or eliminate the need for directed bounds. D-194 already implements outcome and provenance semantics on closures; use that as the baseline for any representation prototype, comparing it with fact_costs.exs, the preparation benchmarks, and the narrowing suites. Keep the existing representation until the benefit and migration invariants are established.
Tradeoff and acceptance criterion. Moving functions alone does not improve the architecture. Each extraction should reduce hidden dependencies or make an invariant locally testable. Preserve evaluation and analysis order initially, then compare diagnostics, prepared results, and performance. Do not add full-tree traversals merely to make the diagram more symmetrical.
3.5 Support semantic inspection and execution explanations
Build on the existing Feel.Expression accessors, which expose syntax, unoptimized IR, names, inferred result type, and diagnostics. Following Roslyn's service model, add focused queries over the same analysis used for compilation: binding resolution at a source location, a node's inferred type and assumptions, and the reason a transformation kept or removed a check. Begin with the information needed to diagnose the known failures.
Keep a small supported query API separate from version-specific debug dumps. Consumers should not need to reproduce name resolution or depend on every private IR tuple. A compiler workspace engine, incremental editor analysis, and a general analyzer-plugin system are separate investments that require their own use cases.
Compiler inspection and execution explanation serve different audiences. Decision authors need to understand which branch or rule was taken, where a value came from, and which constraint failed. Build on existing source spans and DMN source mapping. When transformations combine or split nodes, keep enough origin information to relate generated work to the source.
The unimplemented :explain profile is a concrete opportunity: begin with bounded interpreter traces and a defined event vocabulary. Trace the execution being explained; avoid running effectful input a second time merely to reconstruct an explanation. See the explain proposal.
Tradeoff and acceptance criterion. Public queries need stable meanings, and detailed traces consume memory. Verify that queries agree with the actual compiler decisions and report uncertainty honestly. Test shadowed bindings and source origins after rewriting. Define a maximum trace size and a truncation event, and verify that explanations preserve the normal result and diagnostics. An interpreter-first trace implementation is sufficient; optimized BEAM tracing can wait until its provenance requirements are understood.
3.6 Make caching and backend admission workload-aware
Keep preparation separate from execution. The benchmark harness reports cold BEAM preparation with its generated module absent, warm preparation while the same module is loaded, and release separately. Cold is a module-cache condition in an already running VM, not a fresh VM. Warm preparation still parses, analyzes and emits code; reusing the prepared expression avoids all that work. The table suite's first pass mixes cold and warm preparations because repeated cells share modules; it reports a separate warm pass and release of distinct modules. For a workload with positive per-run savings:
cold break-even runs = additional cold preparation time / per-run time savedRelease is excluded from preparation and this break-even calculation. It has its own cost, which depends on the VM's process count. If compiled execution is no faster, there is no execution-count break-even. Use representative inputs, including fallback cases, and account separately for memory and startup constraints. Historical benchmark prepare_beam_ns figures included release and must not be compared directly with the new prepare_beam_cold_ns figures.
The current loader deduplicates already-loaded code by a hash of the IR and cache stamp. A prepared-expression cache would additionally save parsing and analysis. Put that cache where reuse and lifecycle are known, often the embedding application or model service. Derive its identity rules from the P1 environment contract: account for source, scope names, declarations and declaration stance, dialect, extension and host-callable contracts, relevant options, and implementation compatibility. Runtime values may vary when the established contract permits them. Do not share a diagnostic-bearing artifact across different source locations without preserving the correct source mapping.
The CEL comparison moves specification of that compatibility contract ahead of caching; it does not move a cache itself out of P3. If a host-callable contract has no stable identity or version, keep reuse within an owner that controls its lifetime instead of inventing a universally safe cache key.
Bound concurrent compilation, live code, and admission of new generated modules. Unloading a module does not reclaim its atom name, so an LRU bounds live code but not the cumulative number of names created. For unlimited expression churn, keep the interpreter path available. Do not casually reuse a small pool of module names for different expressions: active code and versioning make that a separate correctness problem. See Loader.
C0 prevents name creation for expressions refused by the generation limit. D-223 separately caps new names over the VM lifetime, including later compiler failures. If a deployment accepts unbounded numbers of distinct expressions for BEAM compilation, promote C11 admission control to P1. A prepared-expression cache alone does not solve that lifetime limit.
Tradeoff and acceptance criterion. Caches and automatic promotion introduce ownership and invalidation problems. Start with explicit backend selection and documented reuse. Add a bounded cache only when measurements show repeated preparation in real callers; evaluate hit rate, saved preparation time, retained memory, and semantic-key correctness. Persistence across releases needs an artifact versioning design beyond the current generated-code cache stamp.
3.7 Extend guarded fast paths selectively
The scalar backend already checks assumptions, performs a bounded set of cheaper operations, and restarts in the interpreter when it cannot answer safely. It avoids much of the general environment and outcome handling on successful paths. Small scalar workloads motivated this path, but gains and break-even counts depend on the expression family and must be measured on the current code. See Scalar and performance notes.
Its restart strategy depends on speculative work not having committed observable effects. Extending it to host calls, diagnostics, or other externally visible operations requires proving replay safe, or designing a different recovery mechanism.
Tradeoff and acceptance criterion. Every added operation enlarges the specialization surface. Profile common models, select one operation family, and compare total preparation-plus-execution cost with realistic fallback rates. Preserve decimal behavior, conversions, errors, and limits. If most cost remains inside the same runtime operation, removing another small layer of dispatch may not justify additional generated code.
4 Changes to defer
These options are not inherently poor compiler designs. Their benefits do not currently justify their costs for this project.
| Change | Why defer it | Evidence that would justify reconsideration |
|---|---|---|
| LLVM or another native backend | Requires runtime/value integration and another execution implementation; native compilation alone does not remove decimal and collection costs | A concrete deployment requirement outside BEAM, or a measured workload with a credible prototype advantage |
| General SSA middle end | Adds control-flow construction, joins, analysis maintenance, and metadata obligations | Several valuable cross-branch or loop transformations cannot be expressed cleanly with targeted lowering |
| Adaptive JIT with multiple optimization tiers | Adds profiling, recompilation, version management, invalidation, and state recovery | Large stable workloads whose observed behavior yields substantial gains beyond existing guarded compilation |
| Custom bytecode VM | Adds an instruction set, compiler, debugger surface, and execution path | IR walking or IR memory is a measured dominant cost and a small prototype improves it materially |
| Rust-style incremental query engine | Dependency tracking and invalidation would be disproportionate to ordinary expression preparation | Interactive editing of large models with measured recompilation costs that unit-level reuse cannot solve |
| Generic pass/plugin or IDE workspace framework | Adds ordering, lifecycle, and invalidation machinery beyond focused compiler queries | Independent transformations or interactive large-model editing need capabilities the small API cannot supply |
| Broad common-subexpression elimination, code motion, or equality saturation | Value equivalence alone cannot justify changes to diagnostics, calls, or resource behavior | Effects and retained resource-policy contracts are established, and specific transformations have demonstrated value |
| A wholesale analyzer rewrite | Risks changing many semantic boundaries at once | Incremental extraction repeatedly fails to make required invariants expressible |
Similarly, a decision-table indexing or batching layer should be driven by model profiles. It can be more valuable than generic compiler optimization if repeated rule dispatch dominates, but must preserve the engine's hit-policy behavior and observable evaluation semantics. The existence of a dependency plan in Dmn.Compiled is not itself evidence that another planning layer is needed.
5 Delivery roadmap and validation
5.1 Effort estimates
These are rough engineer-day ranges for one maintainer familiar with this code, including implementation, focused tests, relevant measurements, and documentation. They exclude review delays, unrelated type-system repairs, and deployment-specific integration beyond the stated slice. They are planning judgments based on the inspected implementation, not measured delivery times. Re-estimate after the first change and after any prototype; shared refactoring and validation mean the rows should not be added into a calendar commitment.
| ID | Scoped deliverable | Effort | Prerequisites |
|---|---|---|---|
| C0 | Intern the generated-module name only after the existing generation check passes | Complete | Preserve the no-atom-on-refusal regression |
| C1 | Stop excessive emission early, retaining the final exact check | 3–5 days | A sound relation between partial emission work and the limit, plus a diagnostic contract; C0 is already implemented |
| C2 | Profile and reduce repeated preparation work with local memoization or targeted optional-precision caps | Re-estimate after profiling | Preserve existing worker containment and required checking; coordinate semantic keys/fallbacks with C3 |
| C3 | Specify preparation assumptions, environment compatibility, and transformation preconditions | 3–5 days | Audit the current public and runtime contracts; no new representation required |
| C4 | Initial structural IR validator at analysis and optimization boundaries | 2–4 days | Agreed node, binding, and metadata invariants; use C3 where assumptions cross a boundary |
| C5 | Conservative effect summaries and migration of one optimizer transformation family | 5–10 days | C3; repaired type facts for any restored proof-dependent optimization; C4 is useful for validation |
| C6 | Prototype flatter branch lowering, then harden it if worthwhile | 2–4 days prototype; 3–6 more to ship | Baseline scaling measurements and preserved evaluation order; C4 is recommended, full C5 is not required for a local order-preserving rewrite |
| C7 | Extract one cohesive analyzer service without changing traversal order | 2–4 days per service | An identified boundary and its invariants; count extractions required by C2 or C5 there, not twice |
| C8 | Initial source-location queries for bindings, inferred types, and available assumptions | 3–5 days | Defined source and binding identity; C3 for the meaning of reported assumptions; richer transformation reasons may depend on C5 |
| C9 | Bounded interpreter explanation events with source provenance | 5–10 days | Outcome/event contract and provenance rules from C3; reuse C8 where useful, but a public query API is not a hard dependency |
| C10 | Bounded prepared-expression cache for one identified embedding owner | 4–8 days | Measured repeated preparation, C3 compatibility rules, explicit ownership and invalidation policy |
| C11 | Compilation admission and concurrency limits for one identified owner | 3–6 days | Defined quota lifetime and handling of VM-lifetime module names; coordinate with C0 and C3; no cache required |
| C12 | One additional guarded operation family on the scalar fast path | 3–7 days | Representative profiles, sound operand assumptions, and a local replay/effect argument; reuse C5 where applicable |
The broader features cost more than their first slices. C4 initially checks structural consistency, not soundness. C5 does not migrate every optimizer rule. C8 does not provide a complete IDE or all optimization remarks. C9 excludes tracing optimized BEAM execution. C10–C11 exclude a distributed cache or fleet-wide quota service. C12 excludes arbitrary effectful host calls and general deoptimization.
5.2 Dependencies and sequencing
Independent groundwork: C0 is complete. C1 needs a counting argument rather than a new analyzer architecture. C2 and C3 can progress independently where their interfaces are agreed. The smallest C4 validator can use existing IR definitions; it need not wait for every effect summary.
Semantic dependencies: C3 supplies the assumptions used by C5 and by reuse policies. Type-system repairs gate any optimization that relies on the affected facts; they do not block delaying module-name creation, structural validation, or inspecting existing analysis. A structural validator passing is not permission to re-enable an unsound optimization.
Targeted improvements: C6 can follow its own measurement and equivalence checks without waiting for a complete optimizer refactor. C7 should accompany work that needs a cleaner boundary. C8 and C9 share provenance requirements, but neither needs a new backend or query engine.
Workload decisions: C10 requires evidence of repeated preparation. C11 is independently valuable when distinct-expression churn is a deployment concern and should then move ahead of caching. C12 requires both measured benefit and safe fallback. None of these follows automatically from completing the earlier work.
5.3 Completed first step and next implementation slice
C0 is complete. Its regression derives the prospective module name as a binary from the IR and cache stamp, uses String.to_existing_atom/1 to check absence without creating it, and verifies that a refused expression creates neither an atom nor a module. Accepted expressions keep exact limit checks, deduplication and serialized loading. The placeholder replaces only the module attribute, preserving literal atoms in generated expressions. Feel.Runtime.abi/0 is a cache-generation stamp in module names; there are no ABI markers, compatibility modules or version negotiation. Prepared expressions must be rebuilt on upgrade; persisted generated modules are unsupported.
Next: C1, early emission refusal. Estimated effort: 3–5 engineer-days. Count generated work during emission using a sound lower bound on the final statement count, and stop before allocating excessive forms. Keep the final exact check for accepted candidates and define diagnostics for early refusals without claiming an exact final total that was never constructed. Test nested functions, branches, chunking and large constants, along with zero, exact and infinite limits. Compare general and scalar output against the interpreter and preserve diagnostics and evaluation order.
This remains separate from the worker policy: BEAM compilation requires trusted input, and accepted module-name atoms have VM lifetime. The library caps fresh module names (D-223); C11 concurrency and per-owner admission stay the host's concern.
5.4 Delivery order
- Preserve completed C0. Keep the no-module-name-atom regression and the existing generation-limit contract while changing emission.
- Clarify contracts and reduce preparation costs. Develop C3 alongside measured C1–C2 slices, extracting only the services needed for that work. Add C4 at useful development boundaries and migrate one transformation family through C5.
- Run the C6 branch-lowering experiment. Ship the hardening work only if the measured preparation gain justifies its runtime and maintenance cost.
- Add C8 inspection and C9 explanations. Reuse existing analysis and start execution traces in the interpreter.
- Choose C10–C12 from deployment evidence. Promote C11 when expression churn makes admission a correctness requirement; choose cache and fast-path work using actual reuse and profiles.
This sequence keeps the current public prepare/run boundary and both backends. The type-system soundness work stays a separate prerequisite for affected optimizations, rather than a reason to postpone every architectural improvement.
5.5 Validate semantics and cost separately
Use the existing rejected-declaration and extension-condition sweeps, rule-table sweep, DMN chain audit and mutation generator for the corresponding changes (D-195, D-199, D-206). A surviving mutation needs semantic review: safe widening is different from a missed under-approximation. These tools still share membership and runtime machinery, and do not replace independent expectations.
For each transformation, compare optimized and unoptimized execution on both backends, including returned values, ordered diagnostics, source locations, strict failures, and host-call behavior. Include missing bindings, null inputs, conversion boundaries, overflow, and resource-limit edges. The interpreter and Feel.Paranoid provide a strong comparison mechanism, but shared analysis and runtime code can share bugs; targeted independent expectations remain necessary.
For performance changes, use the existing prepare, run, scale, and table suites. Record phase timings, growth with source and input size, compilation memory, generated code size, and steady-state execution. Include small ordinary expressions as well as stress cases: an improvement to worst-case scaling can coexist with an unacceptable cost on the common path.
Measure fallback-heavy inputs separately from successful specialized runs. Measure cold compilation separately from reuse of an already-loaded module. Cache experiments should report hit rate and retained resources, not just hit latency.
For inspection APIs, compare reported facts with compiler decisions and preserve stable query meanings as the internal representation changes. For reuse, vary declarations, extension contracts, source locations, and relevant options to test compatibility decisions independently from cache hit rate.
A change should earn its place through a clearer invariant, a bounded failure mode, a useful explanation capability, or a reproducible workload improvement. The priority is to make the existing compiler easier to reason about and operate while preserving its ability to reuse the Erlang toolchain.