Questions about libp2p integration and Goblins' spawn implementation

Architecture
1

Based on my reading of the goblins source code, I have a question about Goblins’ spawn implementation:

I notice that Goblins implements its own spawn system (spawn, spawn-vat, spawn-fibrous-vat, etc.) rather than using Guile’s native spawn module. I’m particularly interested in:

  1. The architectural decision behind this - what specific requirements of the actor model led to needing a custom spawn implementation?
  2. The relationship between Goblins’ spawn and Guile’s native spawn - particularly in the context of the fiber scheduling system, since I see spawn-fiber being used in various places like syscaller-free-fiber.
  3. How the different spawn variants (spawn, spawn-vat, spawn-fibrous-vat, spawn-persistent-vat) interact with Goblins’ actor model and persistence system.

This would help me better understand both the implementation details and the design philosophy behind Goblins’ actor system.

This question should help clarify both the technical implementation details and the architectural decisions behind Goblins’ spawn system, which seems to be a core part of its actor model implementation.

Thanks in advance :slight_smile:

2

Hey!

It can be a bit confusing given Guile has a procedure named spawn, there’s in fact not related, but to answer your questions one by one:

  1. The spawn function in Guile spawns a separate process with a program running inside it. For historical context, Goblins was initially written in Racket Scheme, so Guile having a procedure with the same name is a coincidence. The spawn function in Goblins essentially invokes the constructor of an actor within a vat and creates a reference to that actor.

  2. There is no relation between Guile’s native spawn and Goblins’. Goblins by default offers fibers which provide an event loop. This event loop can actually be swapped out for other event loops - in some projects, it’s more beneficial to use the event loop for GTK, a game loop, or something else. syscaller-free-fiber shouldn’t be needed very often outside of core Goblins development, so probably as you’re starting out you can ignore this.

  3. Vats are event loops which have two main things:

    • A transactional map of actors (called an actormap)
    • A message queue (FIFO). All messages sent to actors (using <-, <-np, or from OCapN) are queued up on the vat and dispatched in turn.

Vats in Goblins are a fairly critical part, while you can technically use actormaps on their own, really in almost all instances you want to use vats. When using vats, each actor would be spawned within a vat and then all messages an actor recieves are handled by that vat.

Here’s what the different spawn functions do:

  • spawn: Takes an actor’s constructor and spawns it within a vat, returning a reference to the actor
  • spawn-vat: Creates a brand new vat (event loop + transactional actor map) on the default event loop (currently fibers)
  • spawn-fibrous-vat: Creates a vat specifically with fibers as the event loop
  • spawn-persistent-vat: Creates a vat that will persist to one of Goblins’ persistence stores
1 Like
3

Thanks so much for the detailed reply! Your explanation really helps clarify the distinctions between Guile’s spawn and Goblins’ spawn system, as well as the purpose of the various spawn-* variants in the actor model.

I’d like to share some of what I’m working on and ask for further guidance:

I’m exploring the integration of Tailscale and libp2p into Goblins to enable secure and flexible peer-to-peer communication between vats. My goals are to leverage Tailscale’s simplicity and libp2p’s advanced protocol features. Here’s where I’m currently focused:

  1. Persistent Vats and Distributed State:
  • With Tailscale, I’m using spawn-persistent-vat to manage state across nodes and ensure smooth reconnections.
  • With libp2p, I’m working on integrating peer IDs and multiaddresses into Goblins’ OCapN system. Do you have recommendations for handling state consistency or recovery in distributed settings with either system?
  1. Event Loops:
  • I’m currently using the default fiber-based event loop for Tailscale and exploring socket-based I/O for libp2p.
  • Are there specific scenarios where swapping out the fiber-based loop for a custom loop would significantly improve performance, especially in high-throughput environments?
  1. Actor Interactions:
  • For both integrations, I envision actors representing nodes, connections, or sessions. I’ve implemented basic peer discovery and messaging, but I’m curious about best practices for scaling these interactions securely—especially when bridging Tailscale’s simplicity with libp2p’s extensibility.

Your insights into how Goblins’ spawn system supports flexibility (e.g., swapping event loops) and persistence are invaluable for these integrations. If there are additional patterns or resources you’d recommend, I’d greatly appreciate it.

Thanks again for your time and guidance! :blush:

4

Hey,

Sorry for the late reply. I’m not familiar with Tailscale so my answers might be a bit general :slight_smile:

Persistent Vats and Distributed State

  1. The persistence system can save and restore any kind of local reference to actors as well as data, it cannot as of yet save and restore any remote references (those across CapTP). I’m not sure if that’s what your meaning but if you do have remote references that you want to save you can make sturdyrefs and give those out, the nodes can then persist the sturdyref and a reference to a enliven capability (from mycapn) and restore those…
  2. Generally with the libp2p node, Goblins will speak to the libp2p daemon and make an IP and peerID, these will be “encoded” into the OCapN locator and you can just save and restore that. The libp2p netlayer itself can be persisted with goblins and will keep hold of the keypair it needs to restore the peerID into the libp2p daemon. So long as you’re keeping the mycapn / libp2p netlayer in the persistence graph, it should just work.

Event Loops:

I think fibers is probably the best event loop for almost all applications. Generally the reasons to use a different event loop are if you need to work with another library like a game engine with its own event loop or a graphical interface library with its own event loop.

Actor Interactions

I think without knowing more about Tailscale I’m not too sure. Generally using goblins you can write it in a netlayer agnostic way, so you typically don’t need to concern yourself with specific features within a given netlayer (though you might want to pick them based on the qualities they have - speed, privacy, platform availability, etc.). Maybe you could share some code so I can help understand your project a bit better :slight_smile:

Hope that helps, if I’ve misunderstood something let me know.