Please Review: The Heart of Spritely (HoS)

Friends, we need your help!

We’re working on our key documents to describe the “how” of Spritely infrastructure.

The rough draft of the main portion of our capabilities primer is available for review here:

https://spritely.institute/static/papers/spritely-core.html

(NOTE: This is a different link than before!)

[There are .odt, .org, and .pdf versions available there as well, if you are so inclined…]

There is a lot in this paper so far, and I have reached the limits of my ability to be an effective single reviewer, and so we need your help!

The goal is to make sure what we have makes sense, is technically correct, and clearly communicates what we’re trying to accomplish.

Notes:

  • The people on this server are the only ones who are being invited to review this early draft at this time. For now, please do not distribute more widely. If there is someone who should be a part of this, recommend them so we can invite them to the community.
  • Please initially focus on sections 1, 2, and 3 and their footnotes. The other sections aren’t ready yet - though if you have resources to help with those, please share!
  • We are planning wire diagrams and bullet summaries for each section - we’ll be adding them as we go.
  • There will be periodic pushes of changes as we make them - initially a few days apart and will announce them on this thread. The doc will include it’s own change history.
  • The initial audience for this paper is other technical folks working with potential technical partners or funding sponsors.
  • Any help with clarifying terminology would be most helpful.
  • Simple thread feedback on this thread is fine. Feel free to cut-and-paste from the doc for context.
  • For feedback that will likely spawn discussion - please create a new thread in the Whitepapers category. It’s OK if you don’t, we’ll happily use our moderation tools to split it after the fact, if needed.

Table of Contents

  1. Introduction
  2. Capability security as ordinary programming
  3. Spritely Goblins: Distributed, transactional object programming
    3.1. A taste of Goblins
    3.1.1. A simple greeter
    3.1.2. State as updating behavior
    3.1.3. Objects which contain objects
    3.1.4. Asynchronous message passing
    3.1.5. Transactions make errors survivable
    3.1.6. Promise pipelining and coroutines
    3.1.7. Errors propagate through pipelines
    3.2. Security as relationships between objects
    3.2.1. Making and editing a blogpost
    3.2.2. A blog to collect posts
    3.2.3. Group-style editing
    3.2.4. Revocation and accountability
    3.2.5. Guest post with review
    3.3. The vat model of computation
    3.4. Turns are cheap transactions
    3.5. Time-travel distributed debugging
    3.6. Safe serialization and upgrade
    3.7. Distributed behavior and why we need it
2 Likes

Just getting started. The paper recommends reading Spritely: New Foundations for Networked Communities, first, but the link is 404.

2 Likes

Those other documents don’t exist yet.
Probably should comment it the links.

1 Like

Thanks for the catch. I’ve made it so that the other documents export also but have a giant WARNING at the top saying they aren’t anywhere near being really worked on yet, and to focus on the core one. The command I use to build the exports should do so for all three documents now.

1 Like

I haven’t yet tried to run most of the code (need to get some wisp tooling up and running to make it easier) but this was a cool read and seeing a practical application example (the blog) explained in so much depth was very illuminating.

Section 3.6 on serialization could use a similar technical walkthrough, I think - though I know it’s near the end of the doc so you may not have gotten to that yet.

2 Likes

Thanks @jfred! I’m glad it was illuminating.

I agree the serialization part needs more explanation. Partly it also needs a better pass: we’ve recognized that Aurie, as it exists, works but involves too much boilerplate. Part of the goal is that in guile-goblins that we clean it up so that it “feels” natural to do serialization… and in fact, Aurie is being folded into the Goblins library proper as part of the port to Guile. That has full consensus from @frandallfarmer, @tsyesika, and myself. One can see good motivation for it in that there’s plenty of things in the Racket version’s actor-lib that would be useful if they had a serialization mechanism, but don’t.

One of the changes I think we’d like to make, but I haven’t made yet, is that it seems that the whole section with the blog tutorial (which has gotten fairly large!) should be its own toplevel part of this document. In other words, probably the overview of Goblins in terms of its ideas and features should come first, and then the tutorial should come after. Having read it, what do you think? (That question is open to you @jfred, but also everyone!)

EDIT: Oops I just noticed that Randy raised this very question as its own thread!

Also, thanks for posting this Randy, and for all the back and forth with me in crafting this.

To everyone else: this is why I haven’t been weighing in as much as I’d like to the interesting threads on this forum the last few weeks… I’ve been in crunch trying to get the document to the state it is because we wanted to share it here, high priority. So your feedback is really, really appreciated.

2 Likes

How do you want us to provide comments? I find Google Docs the easiest to make comments, but I can add comments directly to the .odt version. I can also make a copy and put the comments there.

When making text changes, I like to use the notation <old text|new text>, but I’m open to any style you guys prefer.

Please post them here.

It’s far easier to correct small things in the document, e.g., fixing typos. The same is true for substantive comments where context is important. It’s also easier for you, because you don’t have to look for where the changes should be made.

May I create a Google Doc of the .odt file? I’ve found that format the easiest way to handle this task.

I just noticed that creating a Google Doc from the .odt version won’t work because the .odt doesn’t have the footnotes. I can created one from the HTML version, though.

This is also all in a git repository; it isn’t public yet, but it’s pretty much all the same content as here. If you would like access to read the repo and make PRs, that’s an option.

I’d like to capture any discussion of specific language/concepts on this board, but if folks want to use other tools for annotating simple (non-controversial) editorial suggestions, have at it - just don’t share widely, please.

NOTE: The only means we are providing for getting updates to the document are via the web directory we shared. We are not going to be keeping a live copy of this document anywhere else.

Fair enough. I can use the Google Doc only for my own edits and for you to see what they are, or I can post the link here if you’d like to collect all the editorial-type comments in one place.

1 Like

Thanks! When we integrate the changes into the document, we’ll request that you delete the old copy.

No problem. I’ve set up the share for reviewing. Let me know if it works properly.

1 Like

I’m through Section 3.1.4, and I’m struggling with Guile. That wouldn’t be a problem if the purpose of this document is to onboard developers, but I thought the idea was to introduce the main concepts of ocaps for distributed applications. If so, I think the document would be much more effective if the concepts were illustrated with a more commonly used language. (Fortran would be my choice :grinning:) Even better might be a pseudo-language where you could focus on the concepts without dealing with the cruft that comes with real languages. Of course, that won’t work if you want to provide a REPL. Still, you could leave it as an exercise for the reader to implement the pseudo-code examples in a favorite languge.

One big thing that is missing (already noted) is the diagrams, which we think will also help a lot.

We’ll see, but I’m skeptical. My big concern is that a developer will take one look at a couple of the Guile examples and say, “Not for me.”

We will be implementing in multiple languages. This is the one we have working now.

Readability is important, no matter what syntax we use, so all feedback about how to improve understanding is welcome.

1 Like