Skip to content

Optional Record Fields: What Changed and Why

4 min read

Filtrera now has a first-class field?: T syntax for optional record fields, replacing the old field: T | nothing convention that quietly conflated two different ideas.

For a long time, the way to say “this record field is optional” in Filtrera was to add | nothing to its type:

{ requiredField: text, optionalField: number | nothing }

It looked fine. It compiled. People read it as “the field may be missing”.

But it was quietly wrong, in a way that produced some genuinely weird type errors. This post is about what was wrong, what it cost users in practice, and how the new field?: T syntax fixes it.

The thing the old syntax couldn’t say

number | nothing is a union of two types: a number, or the value nothing. That’s a description of a value. Whether the field is present in the record at all is a question about the shape of the record, which is a different thing entirely.

The old type system papered over this distinction. A missing field was treated as field = nothing. So field: number | nothing and “field may be absent” looked indistinguishable from the inside.

This is fine right up until it isn’t.

When it isn’t fine

Consider a script processing API responses, where each response is either an error or a successful payload:

param response: value
from response match
(r: { error: value }) |> { error = r.error, next = 'aborted' }
|>
let rows =
response match
{ data: [value] } |> response.data
|> []
let nextHref =
response match
(c: { links: { next: { href: text } } }) |> c.links.next.href

The two inner match cases — checking for data and for the nested links.next.href — would emit the match input type will never match this condition warnings.

The compiler wasn’t lying. Under the old rule:

  • The outer match narrowed the catch-all branch to “not a record with error: value”.
  • But under the implicit “missing field = nothing” rule, every record was assignable to { error: value } — because if it didn’t have an error field, the missing field counted as nothing, and value accepts nothing.
  • So { error: value } matched everything. Its negation matched nothing. The inner patterns were unreachable.

The author was clearly thinking “records that don’t declare an error field”. The compiler was thinking “records whose error field, present or implied, has type value”. Same syntax, different mental model. The compiler won, and the script got warnings on patterns that obviously could match.

The fix

Required is now the default. Optional is opt-in, with a marker right where you’d expect it: ? after the field name.

{ requiredField: text, optionalField?: number }

Three changes follow from that:

  1. Strict assignability. A record without a declared field is no longer assignable to a type that requires it. The not { error: value } case from the example above now actually means “records that don’t declare error”, and the inner patterns match.
  2. ?: desugars cleanly. { a: A, b?: B } is equivalent to { a: A } & ({} | { b: B }) — the union of the two record shapes (with and without b). Pattern matching, intersection, narrowing — they all already know how to walk that shape, so no new machinery was needed.
  3. T | nothing keeps its plain meaning. It’s a value union. Nothing in the type system pretends it has anything to do with the field being absent.

The distinction matters more than it looks

Once you can spell the two ideas separately, you discover they are both useful, and they actually mean different things in real APIs.

param updates: {
name: text | nothing // required, "explicitly cleared" if nothing
email?: text // optional, "no change" if absent
}

This is a typical PATCH-style update payload:

  • Setting name = nothing means clear the user’s name.
  • Omitting email means don’t touch the email.

These are different intents. With the old syntax there was no way to express them in the same record — you could only pick one and write a comment for the other. Now you can spell exactly what you mean.

JSON serialization, finally aligned

The JSON contract is now formal:

FiltreraJSON
Field declared, value is a T"field": <value>
Field declared, value nothing"field": null
Field not declaredomitted from output

Absent on the Filtrera side is absent on the JSON side. Explicit nothing becomes null. This is what most people thought was happening already; now it’s the formal contract.

Migrating

Existing scripts that wrote field: T | nothing to mean “optional” will fail to type-check until they’re updated.

The migration is mechanical:

  • Field intended to be absent when not provided? Replace field: T | nothing with field?: T.
  • Field that is always present, with nothing as a meaningful value? Keep it as field: T | nothing.

If you read a record’s field that might be optional, replace the unconditional record.field access (which used to silently return nothing when missing) with a match:

record match
{ field: T } |> /* present */
|> /* missing */

That’s it. Two patterns, no surprises.

In short

Filtrera’s job is to be predictable. Two distinct ideas — the field may be absent and the field may be nothing — deserve two distinct ways to spell them. They have those now, and the type system doesn’t have to lie to anyone to make it work.