Register and share your invite link to earn from video plays and referrals.

Mitchell Hashimoto
@mitchellh
Co-founder of @Superlogical. Creator of Ghostty. 👻 Prev founded @HashiCorp, created Vagrant, Terraform, Vault, and others.
Joined January 2008
156 Following    233.8K Followers
The latest Rex preview release includes a very important subsystem I've been working on for weeks to enable full forwards+backwards API compatibility with clients and servers. This was gating the Linux test program and massively opening up the Mac test program. We just invited another 1,000 (to Mac) and if it goes well this will step up considerably this week. The Linux test program will likely start this week (separate invite waves amongst existing testers). The API compatibility system is super cool. At a high level: Every API change is a rule on a JSON pointer (RFC 6901). For each message, we compile all the rules between your version and ours into a trie keyed by JSON path tokens. This happens lazily since we don't want a full quadratic explosion (a message with N revisions has N possible chains of length 1 to N). We don't want to build N^2/2 trie nodes when most WON'T be visited. Next, we wrote a custom recursive descent streaming JSON editor (forward-pass only). If a key doesn't match any subtree in the trie, we fast-path copy the full value onto the wire (nothing in it can possibly matter for compatibility). Equal chains of changes are interned to the same pointer, so when we broadcast an event to many old clients, we translate it once per distinct chain, not once per client. The result of all this plus some other work is that we simulated various shapes of 10,000 long revision chains (note: we're on revision 4 at the moment...) with pathological changes of various shapes and an upgrade costs about 35 _microseconds_ on a 20KB JSON body.
Show more