The last feature for version 0.4 is about adding mv (move) and cp (copy) functions to the SDK plugin. I’m taking advantage of this development to explain how the plumbing works under the hood.
Right now, the SDK plugin is undocumented because there are a lot of breaking changes coming. Every time I develop a new plugin, I discover better ways to do things, and the SDK needs to adapt.
For example, when I added apex_code (a plugin that takes code as a string and returns syntax-highlighted HTML), I wondered: “Is this a standalone plugin?”. Not really, because there’s no file to manipulate, neither in input nor output. Instead, it manages blocks inside an existing file. The SDK wasn’t originally built with that use case in mind.
Anyway, I think explaining how the SDK works through a concrete example will be helpful, if not for you, at least for me.
The SDK exists to pass data from GitRoot (the host) to the plugin (the guest). As you know, GitRoot uses WebAssembly (WASM) to manage plugins. Unfortunately, WASM is still a bit low-level to implement: between the host and the guest, only integers can travel.
Wait, only integers?
Yes. So, how do we send a string? The host can store any kind of data in the guest’s memory. GitRoot stores a string, retrieves the pointer (the memory address) and the size, and sends those two integers to the guest. The guest then retrieves the string from its own memory. It’s a complex dance just for a simple string. The SDK is here to hide all that mess.
When you build a plugin, you just implement an interface. The SDK handles the rest: you receive objects and strings, and you send back data without ever worrying about the underlying WASM pointers.
When I add a new function, I always start with the host side. All functions are located in runtime.go.
We need to register the function in wazero, our WASM runtime. I add two versions because I haven’t found a clean way to bridge the difference between AssemblyScript and Go/Rust yet. Hopefully, I’ll figure that out one day.
1r.wazRun.
2 NewFunctionBuilder().WithFunc(r.MoveFileBuilder(false)).Export("moveFile").
3 NewFunctionBuilder().WithFunc(r.MoveFileBuilder(true)).Export("moveFileAS")
Then I implement the builder:
1func (r *runtime) MoveFileBuilder(forAS bool) interface{} {
2 if !forAS {
3 return func(_ context.Context, m api.Module, ptr, size, ptr2, size2 uint32) {
4 // from ptr/size to string or object
5 r.moveFile(fromPath, toPath)
6 }
7 } else {
8 return func(_ context.Context, m api.Module, ptr, ptr2 uint32) {
9 // from ptr to string or object
10 r.moveFile(fromPath, toPath)
11 }
12 }
13}
Yeah, I know. As a Gopher, seeing
interface{}as a return type and all these uint32 feels wrong. But hey, we’re not here for a code review, we’re here to add an API. If you have a cleaner way, feel free to submit a graft!
We want to allow any plugin to move a file. Let’s define the internal moveFile function:
1func (r *runtime) moveFile(fromPath, toPath string) error {
2 //need to do something...
3}
Actually, there’s a catch, GitRoot uses three internal filesystems: one for the git worktree, one for web content, and one for the cache. Even if they are abstracted under one root, the code often expects a specific FS. I’ll probably clean this up later, but for now:
1func (r *runtime) moveFile(fromFs, fromPath, toFs, toPath string) error {
2 //need to do something...
3}
Now your plugin can call moveFile("cache", "artifacts/myBin", "web", "releases/myBin-v1.0") to publish an artifact, or moveFile("web", "index.html", "web", "old/index.html") to back up a file.
I won’t bore you with the implementation logic here. If you’re curious, check the code.
Here is how the final builder looks, converting pointers to strings:
1func (r *runtime) MoveFileBuilder(forAS bool) interface{} {
2 if !forAS {
3 return func(_ context.Context, m api.Module, fromFsPtr, fromFsSize, fromPathPtr, fromGlobPathSize, toFsPtr, toFsSize, toPathPtr, toPathSize uint32) uint64 {
4 fromFs, err := r.readString(m, "MoveFile fromFs", fromFsPtr, fromFsSize)
5 if err != nil {
6 return r.sendEmptyOrError(m, err)
7 }
8 fromPath, err := r.readString(m, "MoveFile fromGlobPath", fromPathPtr, fromGlobPathSize)
9 if err != nil {
10 return r.sendEmptyOrError(m, err)
11 }
12 toFs, err := r.readString(m, "MoveFile toFs", toFsPtr, toFsSize)
13 if err != nil {
14 return r.sendEmptyOrError(m, err)
15 }
16 toPath, err := r.readString(m, "MoveFile toPath", toPathPtr, toPathSize)
17 if err != nil {
18 return r.sendEmptyOrError(m, err)
19 }
20 return r.sendEmptyOrError(m, r.moveFile(fromFs, fromPath, toFs, toPath))
21 }
22 } else {
23 return func(_ context.Context, m api.Module, fromFsPtr, fromPathPtr, toFsPtr, toPathPtr uint32) uint64 {
24 fromFs, err := r.readASString(m, "MoveFileAS fromFs", fromFsPtr)
25 if err != nil {
26 return r.sendEmptyOrError(m, err)
27 }
28 fromPath, err := r.readASString(m, "MoveFileAS fromGlobPath", fromPathPtr)
29 if err != nil {
30 return r.sendEmptyOrError(m, err)
31 }
32 toFs, err := r.readASString(m, "MoveFileAS toFs", toFsPtr)
33 if err != nil {
34 return r.sendEmptyOrError(m, err)
35 }
36 toPath, err := r.readASString(m, "MoveFileAS toPath", toPathPtr)
37 if err != nil {
38 return r.sendEmptyOrError(m, err)
39 }
40 return r.sendEmptyOrError(m, r.moveFile(fromFs, fromPath, toFs, toPath))
41 }
42 }
43}
When I need to pass complex objects, I use json.Marshal in GitRoot and json.Unmarshal in the SDK. It works, but don’t get too attached to it, I might change the serialization method in the future.
Now that the host is ready, we need to implement the other side: the SDK itself. I usually start with Go, then move to Rust and AssemblyScript. Sometimes I forget the others… shame on me! I’m planning to automate this with some boilerplate generation later.
In the server interface:
1type Server interface {
2 //...
3 MoveFile(fromFs, fromPath, toFs, toPath string) error
4 //...
5}
To make it more “user-friendly”, I use a struct to represent the filesystem in model/fs.go:
1func (g *GrFs) MoveFile(fromPath string, toPath string) error {
2 return g.server.MoveFile(g.base, fromPath, g.base, toPath)
3}
4
5func (g *GrFs) MoveFileToFs(fromPath string, toFs *GrFs, toPath string) error {
6 return g.server.MoveFile(g.base, fromPath, toFs.base, toPath)
7}
The actual WASM call happens in plugin/server.go, where we transform strings back into pointers:
1// MoveFile implements [model.ServerNeeded].
2func (s *pServer) MoveFile(fromFs model.FsBase, fromPath string, toFs model.FsBase, toPath string) error {
3 ptr, size := stringToPtr(string(fromFs))
4 ptr2, size2 := stringToPtr(fromPath)
5 ptr3, size3 := stringToPtr(string(toFs))
6 ptr4, size4 := stringToPtr(toPath)
7 resPtrSize := _moveFile(ptr, size, ptr2, size2, ptr3, size3, ptr4, size4)
8 runtime.KeepAlive(fromFs)
9 runtime.KeepAlive(fromPath)
10 runtime.KeepAlive(toFs)
11 runtime.KeepAlive(toPath)
12 return ptrSizeToError(resPtrSize)
13}
And finally, the link to the host via go:wasmimport in plugin/imports.go:
1//go:wasmimport gitroot moveFile
2func _moveFile(fromFsPtr, fromFsSize, fromPathPtr, fromPathSize, toFsPtr, toFsSize, toPathPtr, toPathSize uint32) (ptrSize uint64)
Now, any plugin can use this. Here’s a simple Go plugin that triggers a build whenever a .go file is modified and moves the binary to the web folder.
1package main
2
3import (
4 "fmt"
5 "path/filepath"
6 "strings"
7
8 gitroot "gitroot.dev/libs/golang/plugin"
9 "gitroot.dev/libs/golang/plugin/model"
10)
11
12var defaultRun = []model.PluginRun{{
13 Path: "**/*.go",
14 When: model.PluginRunWhenAll,
15 Branch: []string{"main"},
16 Write: model.PluginWrite{
17 Web: []model.PluginWriteRight{{
18 Path: "releases/main",
19 Can: model.PluginWriteRightCanAll,
20 }},
21 },
22}}
23
24type Plugin struct {
25 server model.Server
26 needToBuild bool
27}
28
29func (p *Plugin) Init(repoName string, confHasChanged bool, serializedConf string) error {
30 p.needToBuild = false
31 return nil
32}
33
34func (p *Plugin) StartCommit(commit model.Commit) error { return nil }
35
36func (p *Plugin) AddFile(file model.File) error {
37 p.needToBuild = true
38 return nil
39}
40
41func (p *Plugin) ModFile(file model.File) error {
42 p.needToBuild = true
43 return nil
44}
45
46func (p *Plugin) DelFile(file model.File) error {
47 p.needToBuild = true
48 return nil
49}
50
51func (p *Plugin) EndCommit(commit model.Commit) error { return nil }
52
53func (p *Plugin) Finish() error {
54 if p.needToBuild {
55 res, _ := p.server.Exec(model.Exec{
56 Cmds: []model.Cmd{
57 {Cmd: "go", Args: []string{"build"}},
58 },
59 Artifacts: []string{"main"},
60 })
61 return p.server.Cache().MoveFileToFs(res.Artifacts[0], p.server.Webcontent(), filepath.Join("releases", "main"))
62 }
63 return nil
64}
65
66func Build(server model.Server) model.Plugin {
67 return &Plugin{
68 server: server,
69 }
70}
71
72//go:wasmexport install
73func main() {
74 gitroot.Register(defaultRun, Build)
75}
Nope. I still need to update the Rust and AssemblyScript SDKs and, of course, add tests. But this should give you a good idea of how the plumbing works.
If anything is unclear, let me know!