Effects Without Monads: Non-determinism -- Back to the Meta Language

We reflect on programming with complicated effects, recalling an undeservingly forgotten alternative to monadic programming and checking to see how well it can actually work in modern functional languages. We adopt and argue the position of factoring an effectful program into a first-order effectful DSL with a rich, higher-order ‘macro’ system. Not all programs can be thus factored. Although the approach is not general-purpose, it does admit interesting programs. The effectful DSL is likewise rather problem-specific and lacks general-purpose monadic composition, or even functions. On the upside, it expresses the problem elegantly, is simple to implement and reason about, and lends itself to non-standard interpretations such as code generation (compilation) and abstract interpretation. A specialized DSL is liable to be frequently extended; the experience with the tagless-final style of DSL embedding shown that the DSL evolution can be made painless, with the maximum code reuse. We illustrate the argument on a simple but representative example of a rather complicated effect – non-determinism, including committed choice. Unexpectedly, it turns out we can write interesting non-deterministic programs in an ML-like language just as naturally and elegantly as in the functional-logic language Curry – and not only run them but also statically analyze, optimize and compile. The richness of the Meta Language does, in reality, compensate for the simplicity of the effectful DSL. The key idea goes back to the origins of ML as the Meta Language for the Edinburgh LCF theorem prover. Instead of using ML to build theorems, we now build (DSL) programs.

💡 Research Summary

The paper challenges the prevailing view that monads are the default or even the only viable way to structure effectful programs in functional languages. Instead, the authors propose a two‑layer approach: first, design a small, first‑order domain‑specific language (DSL) that contains only the data types and primitive operations required for the problem at hand; second, embed this DSL into a powerful meta‑language (OCaml) using the tagless‑final style.

The DSL is deliberately minimal. In the case study it defines two abstract types, int t and ilist t, together with constructors for integer literals, list literals (nil, cons, list), a recursor recur (a variant of Gödel’s System T iterator equipped with a thunk to control evaluation order), and two non‑deterministic primitives: fail and binary choice |||. The semantics of these primitives are constrained by a set of equational identities (e.g., cons x fail ≡ fail, associativity of |||, distributivity of ||| over cons and recur). These identities are not mechanically enforced by OCaml’s type system, but they guide the implementation and serve as a contract that any concrete interpreter must satisfy.

Embedding the DSL in OCaml follows the tagless‑final methodology: a signature (module type NDet) declares the abstract types and operations, and concrete modules provide interpretations. Because the DSL lacks function types, it is not a lambda calculus; this restriction is intentional, keeping the language first‑order and simplifying reasoning about effects. The meta‑language supplies all higher‑order facilities (functions, modules, objects), allowing the programmer to write DSL programs in a style that is as concise and readable as the original Curry code.

The authors demonstrate the approach with the classic list‑permutation problem, which in Curry is expressed as `perm = foldr insert