Optional Record Fields: What Changed and Why
Filtrera now has a first-class field?: T syntax for optional record fields, replacing the old field: T | nothing convention that quietly conflated two different ideas.
For a long time, the way to say “this record field is optional” in Filtrera was to add | nothing to its type:
{ requiredField: text, optionalField: number | nothing }It looked fine. It compiled. People read it as “the field may be missing”.
But it was quietly wrong, in a way that produced some genuinely weird type errors. This post is about what was wrong, what it cost users in practice, and how the new field?: T syntax fixes it.
The thing the old syntax couldn’t say
number | nothing is a union of two types: a number, or the value nothing. That’s a description of a value. Whether the field is present in the record at all is a question about the shape of the record, which is a different thing entirely.
The old type system papered over this distinction. A missing field was treated as field = nothing. So field: number | nothing and “field may be absent” looked indistinguishable from the inside.
This is fine right up until it isn’t.
When it isn’t fine
Consider a script processing API responses, where each response is either an error or a successful payload:
param response: value
from response match (r: { error: value }) |> { error = r.error, next = 'aborted' } |> let rows = response match { data: [value] } |> response.data |> []
let nextHref = response match (c: { links: { next: { href: text } } }) |> c.links.next.hrefThe two inner match cases — checking for data and for the nested links.next.href — would emit the match input type will never match this condition warnings.
The compiler wasn’t lying. Under the old rule:
- The outer
matchnarrowed the catch-all branch to “not a record witherror: value”. - But under the implicit “missing field = nothing” rule, every record was assignable to
{ error: value }— because if it didn’t have anerrorfield, the missing field counted asnothing, andvalueacceptsnothing. - So
{ error: value }matched everything. Its negation matched nothing. The inner patterns were unreachable.
The author was clearly thinking “records that don’t declare an error field”. The compiler was thinking “records whose error field, present or implied, has type value”. Same syntax, different mental model. The compiler won, and the script got warnings on patterns that obviously could match.
The fix
Required is now the default. Optional is opt-in, with a marker right where you’d expect it: ? after the field name.
{ requiredField: text, optionalField?: number }Three changes follow from that:
- Strict assignability. A record without a declared field is no longer assignable to a type that requires it. The
not { error: value }case from the example above now actually means “records that don’t declareerror”, and the inner patterns match. ?:desugars cleanly.{ a: A, b?: B }is equivalent to{ a: A } & ({} | { b: B })— the union of the two record shapes (with and withoutb). Pattern matching, intersection, narrowing — they all already know how to walk that shape, so no new machinery was needed.T | nothingkeeps its plain meaning. It’s a value union. Nothing in the type system pretends it has anything to do with the field being absent.
The distinction matters more than it looks
Once you can spell the two ideas separately, you discover they are both useful, and they actually mean different things in real APIs.
param updates: { name: text | nothing // required, "explicitly cleared" if nothing email?: text // optional, "no change" if absent}This is a typical PATCH-style update payload:
- Setting
name = nothingmeans clear the user’s name. - Omitting
emailmeans don’t touch the email.
These are different intents. With the old syntax there was no way to express them in the same record — you could only pick one and write a comment for the other. Now you can spell exactly what you mean.
JSON serialization, finally aligned
The JSON contract is now formal:
| Filtrera | JSON |
|---|---|
Field declared, value is a T | "field": <value> |
Field declared, value nothing | "field": null |
| Field not declared | omitted from output |
Absent on the Filtrera side is absent on the JSON side. Explicit nothing becomes null. This is what most people thought was happening already; now it’s the formal contract.
Migrating
Existing scripts that wrote field: T | nothing to mean “optional” will fail to type-check until they’re updated.
The migration is mechanical:
- Field intended to be absent when not provided? Replace
field: T | nothingwithfield?: T. - Field that is always present, with
nothingas a meaningful value? Keep it asfield: T | nothing.
If you read a record’s field that might be optional, replace the unconditional record.field access (which used to silently return nothing when missing) with a match:
record match { field: T } |> /* present */ |> /* missing */That’s it. Two patterns, no surprises.
In short
Filtrera’s job is to be predictable. Two distinct ideas — the field may be absent and the field may be nothing — deserve two distinct ways to spell them. They have those now, and the type system doesn’t have to lie to anyone to make it work.