Conundrum: Orthogonal Persistence

Proposed Definition: What to do with state when a vat is put to sleep - so that when it resumes, it will be in a reasonable and predictable state.

Electric Communities experimented with a “binary restore” sort of OP, but this lead to all sorts of problems [missing links/discussion here] - eventually we implemented the ability for objects (Una) to be in charge declaring which state should be persisted, and how it is restored.

[See also Conundrum: [Object] Upgrade]

In other words, do you snapshot a running process via the language runtime or by how the objects describe themselves? And given that, how do you make sure that objects can’t claim to have more authority than they actually have? This has been mostly figured out, and the key ideas here are outlined in Safe Serialization
Under Mutual Suspicion
(aka “The JHU Paper”, or maybe I’m the only one that calls it that).

Spritely is currently implementing this with a currently-separate layer from Goblins called Aurie, however after some conversations with @frandallfarmer and actual implementation experience by Jessica Tallon, Aurie probably won’t be a separate thing anymore and will become part of Goblins’ core repository (same thing for the debugging layer, but that’s its own thread).

2 Likes

Marc Stiegler and I built a couple of prototypes with waterken, which was orthogonally persistent. Early upgrades weren’t a problem; we just restarted from scratch. Things changed when we got to the testing stage and users had substantial investments in preserved state.

Some changes were easy, such as adding a new field to an object. All we had to do was define a default value. Other changes were harder, so Tyler wrote an upgrade tool that processed the stored object state. As I recall, that tool worked most of the time but not always. When it failed, we had no choice but to restart from scratch.

If we can’t guarantee upgrades that preserve state, should we rely on orthogonal persistence? An alternative is versioning, which is a pain, but it can work. Client Utility included a version negotiation step. Each party presented the versions it supported, which meant you had to be careful not to include vulnerable versions in your supported set. If there was no overlap, the only way to interact was to find a 3rd party service to translate, something we deemed acceptable. Is that acceptable for Spritely?

2 Likes

EC’s attempt at orthogonal persistence was an expensive dead end. We did much better with explicit persistence with the deliberate recording of state bundles. One benefit was that buggy extraneous state was not retained, so that bugs can be shed by persisting.

I liked what KeyKOS did, using the virtual memory hardware to keep two versions of memory: one that was actively mutating, and a stable one that can be recorded.

1 Like