ERIS goes Hoot (hooot hoooot)

Hello Spritely beings,

I’m happy to announce some progress towards ther ERIS integration with Hoot (see also ERIS over CapTP): A Hoot compatible implementation of ERIS.

You can find an example here: Supported keywords are: gitdir The data command syntax). To import an existing note, a blank.

It currently only consists of Hoot bindings to an ERIS implementation compiled to WebAssembly. Complete bindings to goblins (i.e. as provided by [goblin eris]( The patch. If a boolean value, or. )) will follow.

Initial idea was to compile the guile-eris implementation to WASM using Hoot and just provide suitable implementations of the cryptographic primitives in WASM land. However, some more issues surfaced: guile-eris uses SRFI-60 and some weird stuff from (rnrs io ports) which didin’t work with Hoot. We could have changed guile-eris to not use those things, but we took the rocky path. We implemented ERIS in Zig so that it can be compiled to WebAssembly by itself( zig-eris).

This may seem crazy and unnecessary, and probably it is. My motivation was to have an implementation of ERIS that can be used as a C library - mostly for finally getting ERIS over/in GNUnet working. It also makes ERIS usable from plain JavaScript. Another neat thing about the Zig implementation is that it does not do any memory allocation. This should make it go fast and make it usable from constrained contexts. See the announcment on the ERIS mailing list for more information.

From Hoot/Goblins it is straightforward to add necessary cond-expands so that the (goblins eris) module provides the same interface when compiling with Hoot or natively.

Another note on ergonomics: Working from Scheme and Hoot is a pleasure, working with Zig is nice. Working with the Javascript glue between the two is bloody hell. I’d really like to be able to link the two WASM modules (the Hoot application and the ERIS implementation) to a single WASM module. This would get rid of the JS interop/glue and would feel much nicer. If somebody knows how this might work, please let me know!

Cheers,
pukka

2 Likes

Wow what an update! This is awesome!

Regarding the issues with compiling guile-eris to Hoot as-is, I think it would still be worthwhile to work on this as it’s a good use-case for Hoot<->Guile compatibility. If you still have details about some of the incompatibilities it would be great to get some issues filed. SRFI-60 doesn’t seem to be too challenging as Hoot already has a lot of these procedures. (rnrs io ports), being an R6RS module, will probably take a bit more work. I know that (rnrs base) has trouble compiling right now, so that’s probably the blocker (I told Andy Wingo about this and he says this should be a change made to Guile as the implementation of (rnrs base) uses module introspection and it shouldn’t.)

Regarding combining two Wasm modules into one, I’m not aware of a tool that could do it today (a quick glance at the wabt and binaryen READMEs didn’t say so, anyway.) It might be possible with the following rough process:

  1. Parse both binaries
  2. Compute the max of all ids in the first module (types, funcs, etc.)
  3. Renumber all the ids in the second module so they don’t conflict with the first
  4. Do some magic to map exports to imports that are now part of the same module and rewrite all call sites to use the internal functions

There could very well be dragons here, though. This would all depend on Wasm multi-memory support, too, which apparently doesn’t work in WebKit yet according to this feature matrix.

Anyway, great update! Thanks for sharing!

That’s pretty much it.

Alternatively to making Hoot work with those modules, I think it would be pretty straight-forward to not use SRFI-60 and the R6RS modules in guile-eris (patches welcome! :slight_smile: )

I was also a bit surprised that this is not something completely standard people do all the time.

Still there seems to be some support:

Hoot also seems to have some WASM linking stuff: Linker (Guile Hoot) . But I don’t know (or think) it follows the conventions above.

I must admit that I am personally not so interested in WebAssembly that I want to figure all this out and experiment. But I think it would be a neat little thing to do: Link Hoot WASM modules with C/Zig code compiled to WASM with LLVM. Anyone up for such a little quest?

1 Like

More hooting!

We now have two more important pieces:

  1. A somewhat uniform interface for using ERIS that works from Hoot and Goblins. This is done by liberal use of cond-expand. The ERIS implementation in hoot is backed by the Zig implementation.
  2. A file-sharing demo using IndexedDB as a storage for encoded blocks. This just shows how to use web-native storage for encoded blocks.

The examples use the JavaScript Promise-Integration extension for WebAssembly. This is a bit of magic that allows WebAssembly callbacks to use asynchronous Web APIs (like IndexedDB).

The demo is still quite resticted, as it does not do any block transfer over the network. One would need to use the WebSocket and possibly Prelay netlayers to do so.

There also a couple of caveats when using all this from Hoot. The whole machinery moves across a couple of boundaries: Hoot WebAssembly to JavaScript to Zig WebAssembly and back and between in all ways. This is not really fun and quite finicky. There’s quite lot of room for improvement (as also noted in the comments above).

In any case, it’s here and is ready for interested beings to dive in. Good spelunking and happy hacking!

2 Likes