Craft your forge, Build your project, Grow your community freely
1# The teenager starts building its own future
2 3## If you only have 5 minutes
4 5A lot has been done in this new release of GitRoot. When I saw other FOSS projects claiming "this is the biggest release", I thought it was a joke. But in fact, it's true: every new release gets bigger and bigger. It's just astonishing to start a sprint with almost zero ideas and, 4 months later, see all of this accomplished.
6 7> If you skim this article, don't forget to check out the conclusion about ["what's next?"](#what-next).
8 9### The thing i'm most proud of
10 11Executors can finally build GitRoot! As I love to say: GitRoot can build GitRoot on GitRoot. And it was a pain to do. [Read more](#executor).
12 13### The thing with the most visibility
14 15The plugin registry is here! After a lot of reading and reflecting, I started this "epic story". Today is only the beginning, but you can now find plugins directly at [gitroot.dev/plugins](../plugins/index.md). What's next? [Read more](#plugin-registry).
16 17### The thing that changes everything but was so simple
18 19Previously, a branch (and graft) couldn't be rendered on the web. The problem was: if I have an `index.md` in my git that renders at `myinstance.com/index.html`, anyone can create a branch. If a "bad user" comes along, changes everything in this file, creates a branch, and pushes for review, where should this file render? Certainly not at `myinstance.com/index.html`. [Read more](#apex-render-branch).
20 21### The thing that was more complex than I thought
22 23Apex was a big plugin. Basically, it manages the rendering of your git files to the web. But over time, it started to grow out of control because of markdown, code, worktrees, emojis... A simple (in theory) solution was to split it into multiple plugins. [Read more](#plugin-deps).
24 25### The things I'm just happy are done
26 27No more mandatory signed commits! Well, it's still better if you sign them, but this change means the `.gitroot/init.sh` has been ~~deleted~~ moved to a plugin. [Read more](#sigma-plugin).
28 29Moving a file from git to web was hard, deleting a file was "not implemented", and a lot of APIs were missing in the plugin-sdk. I got my hands dirty with all that, and now everything is running smoothly. [Read more](#fs-plugin-sdk).
30 31## In-depth Details
32 33### Executor
34 35When I started this feature, I just wanted a lightweight container to run `make build` and output the result as a `gitroot-next` canary binary. If I had known how much time and energy it would actually cost, I’m not sure I would have taken this road.
36 37At first, I chose [bubblewrap](../doc/technicals/executors.md#bwrap-executor) as a lightweight container, and immediately started building a [container](../doc/technicals/executors.md#container-executor) version. Then I realized I needed a [bareMetal](../doc/technicals/executors.md#baremetal-executor) one. And then, I needed a way to run it on other machines, so [ssh](../doc/technicals/executors.md#ssh-executor) came along.
38 39But when I launched it for the first time, I discovered that having `make` isn't enough for GitRoot, I need Golang, Rust, TinyGo, and so on. "No problem," I thought, "I'll use [mise](https://mise.jdx.dev/) to manage that." But running `mise install` takes a lot of time (like 10-15 minutes on my computer). So, I built a cache system to ensure that hit only happens the first time.
40 41Still not the end! After that, I needed to configure port numbers, so I built a complete environment variable system, and used it to configure some `mise` paths.
42 43Is it finished? Not yet! To be able to move a binary generated from a bwrap environment to the web, it has to travel a lot. And by "a lot," I mean moving it to a place where the plugin that launched the executor can access it: its own cache. Then the plugin needs to move it from there to the web. No problem, I just added a bunch of [SDK API](../blog/modifying-API-SDK-plugin.md).
44 45After all that, some commands finally started working... well, kind of. When I ran a bwrap or a container command, I couldn't get the stats of what was happening inside. Having process stats is something I’m proud of (even if it’s still evolving in future versions), but seeing that bwrap reported 20 MB of RAM while the sub-process was actually consuming 80 MB was very frustrating. Furthermore, I needed a way to extract stats from the remote SSH executor.
46 47I tried a lot of things, but ultimately, I chose to build an [executor CLI](../app/executor/main.go) to handle it. I now inject that binary inside your bwrap, container, or SSH environment. This little CLI launches commands, gathers system stats, command status, and logs. It packs everything into a well-formatted JSON and returns it to the executor in GitRoot. GitRoot then manages artifacts/caches and hands the result back to the calling plugin (probably [hop](../plugins/name/hop.md)), which tells you what’s happening. In this case, it reports back so the [grafter plugin](../plugins/name/grafter.md) can include the info in the current graft.
48 49```mermaid
50flowchart TD
51 A[Hop] -->|Exec cmds| B[GitRoot]
52 B --> F[exec]
53 F --> |bwrap/container/ssh|C[executor cli]
54 C --> D[cmd]
55 D --> C
56 C --> |status/code/logs/stats|F
57 F --> |status/code/logs/stats|B
58 B --> |status/code/logs/stats|A
59``` 60 61Phew, is that enough now? It should be, but we'll see how it performs in real-time on [gitroot.dev](https://gitroot.dev). Not everything is finished: for example, the SSH executor doesn’t manage cache yet [#94af](../issues/94af-executor-cache.md) and can't use the container strategy [#f050](../issues/f050-ssh-executor-subCmd.md). You can’t run the executor on other architectures [#4d49](../issues/4d49-executor-compile-other-archs.md) yet, and I’m sure you’ll soon want a way to clean up temp directories [#647f](../issues/647f-clean-executor-temp-dir.md).
62 63But hey, let’s celebrate what we’ve built before jumping into the next sprint!
64 65### Plugin registry
66 67Until now, explaining GitRoot was always about "data in git and the plugin system", but what exactly are the plugins? Where are they? What can they do? All of that was a bit mysterious. With this milestone, I’m answering some of those questions with a first shot at a registry. Be careful, though: this first version is just a "hand-written" preview of what it could be in the future. If you look at the [source](https://gitroot.dev/worktree/plugins/index.md), you’ll see that everything is ready to be automated, but nothing is... yet.
68 69I'm not entirely sure what you, as a consumer, want to see here. Probably an explanation of what the plugin does, some docs, and the available versions. But as a plugin creator, you likely want to manage all of that in your own repository instead of the [gitroot.dev registry](../plugins/index.md). I chose to keep the registry very light and put the details directly in the [plugin's directory](../app/plugins/apex/index.md) to give you an idea of what your own repo could look like.
70 71I'll need to iterate on this. Even if I choose to embed package details in the registry, I still need to decide how: maybe via a JSON in your repo, maybe in the registry itself, or perhaps inside your package (WASM)? I don't know yet what’s simplest for you and for the registry.
72 73Anyway, I hope this registry helps you understand what a plugin is and how it shapes your forge. Some are small ([ladybug](../plugins/name/ladybug.md)), others are massive ([apex](../plugins/name/apex.md)), and some aren't usable solo as they are [dependencies](#plugin-deps) for others (like [apx_mermaid](../plugins/name/apex_mermaid.md)).
74 75Since plugins are now decoupled from GitRoot, I’ve moved all changelogs from the [general](../CHANGELOG.md) one to the individual plugin directories. This way, I’ll be able to release plugin versions independently of GitRoot (and vice versa). I hope to break the plugin API less often, even if I know I probably will in the near future. All of this paves the way for you to build your own plugins and I hope to see them in the gitroot.dev registry soon. Or, of course, you can just build your own registry!
76 77### Apex render branch
78 79For as long as I can remember, this has been a problem with GitRoot: how to render arbitrary changes, especially deletions, on the web? I had imagined a lot of strategies. The last one was to render inside a directory named after the git tree ID (the abstract representation of a working directory). Then, I’d create a symlink between that tree and a main directory.
80 81The idea was: if you push a file in a branch, git gives you a tree ID like `a654fs`. It would be represented as `mydomain.com/myBranch/file -> mydomain.com/tree/a654fs/file`. When you merged that branch, GitRoot would simply symlink the main directory to it: `mydomain.com/file -> mydomain.com/tree/a654fs/file`.
82 83Elegant, but honestly, a total over-engineered mess. When I actually sat down to code this, I ended up adding just two lines to the [apex plugin](../plugins/name/apex.md):
84 85```go
86if p.currentCommit.Branch !=""&& p.currentCommit.Branch != p.config.defaultBranch {
87 path = filepath.Join(p.config.branchesDir, p.currentCommit.Branch, path)
88}
89``` 90 91And... yeah, that was enough. I've been overthinking this since the very beginning of GitRoot! Rendering files from branches into a dedicated branches directory... It's simple. Just like GitRoot should be.
92 93So, what does it mean in practice? The file is now rendered at `myinstance.com/mybranch/index.html` and the graft at `myinstance.com/mybranch/grafts/mybranch.html`. How did I not think of something this simple before?
94 95### Plugin deps
96 97The catalyst for this development was the [apex plugin](../plugins/name/apex.md). At over 3 MB, it was by far the largest plugin. It was packed with configuration, specifically a bunch of booleans to [toggle features](../issues/8b29-apex-working-tree.md). Furthermore, I needed to render Markdown in other plugins, like [pollen](../plugins/name/pollen.md), to create a proper [RSS feed](../issues/83d8-pollen-content-flux.md). It became clear to me: this plugin needed to be split into smaller, more specialized ones.
98 99So, I started building the [apex_markdown plugin](../plugins/name/apex_markdown.md), then [apex_code](../plugins/name/apex_code.md), and next week [apex_mermaid](../plugins/name/apex_mermaid.md). But what I hadn't fully anticipated was the sheer amount of work required to make it all work together.
100101What if a dependency is missing? What if a method is deprecated or removed in a newer version? To handle this, I’ve chosen to make all dependencies entirely optional for now. It’s an easy choice to make today, but I might change my mind tomorrow, after all, there’s a reason most package managers handle version ranges and resolution.
102103This new architecture allows for adding tons of features without rebuilding an entire plugin. Imagine adding Org-mode or RST support: you just swap out [apex_mardown](../plugins/name/apex_markdown.md) instead of touching the whole stack. It also allows other plugins to reuse existing behaviors effortlessly.
104105With this in place, I’m not even sure if I’ll keep the `report` system, which was an other way of passing information between plugins.
106107### Sigma plugin
108109Signed commits should be the default in git today. With SSH signing, it’s easy to do, and everyone should use it because impersonating a name and email is way too easy in git. When I started GitRoot, I made signed commits mandatory before allowing a push. But in fact, making signatures mandatory should be a choice for the instance owner. GitRoot is all about the freedom to choose how your forge works for you.
110111Furthermore, a friend told me he preferred signing with PGP and already had his setup ready. So why force him to use a `.gitroot/init.sh`? It’s frustrating when you want to test a new forge and you’re forced to run random sh commands, even if it's just to configure your git client.
112113With this release, I have completely removed `init.sh` from GitRoot itself. But of course, as always, I’ve created a plugin to "almost" recreate this feature. The only difference is: instead of refusing your commit when you push, the [stigma plugin](../plugins/name/stigma.md) will simply report unsigned or badly signed commits in your graft.
114115Now, you have the choice to accept unsigned commits in your repo. You can even ignore the whole signing topic entirely if you don't install the stigma plugin.
116117That said, I’m still convinced that signing commits should remain the default.
118119### FS plugin-sdk
120121After finishing all the [executors features](#executor), one thing was still missing: the ability for a plugin to move a file from one path to another. As explained in this [blog post](./modifying-API-SDK-plugin.md), even though a plugin sees one single filesystem, GitRoot actually uses three different ones: one for cache (a directory), one for the web (a subdirectory), and a virtual one for the git working directory.
122123I was missing several methods in both the Rust and TS SDKs, and the existing API was poorly designed with inconsistent naming. I decided to redesign everything and added a lot of new methods accessible via `FS` in the SDK. Now, you can copy, move, delete, replace, and write files in every filesystem. Moving and copying can now be done between different filesystems, making it easy to make a file accessible on the web even if it originated from git or the cache.
124125I have marked all old methods as deprecated, and they will be removed in the next version of GitRoot (or more specifically, the plugin-sdk, though they share the same release number for now).
126127## What next?
128129I have a massive roadmap for GitRoot (in no particular order):
130131- Discover GitRoot without installing anything.
132- Create your own plugins with ease.
133- Transform GitRoot into a registry for (at least) Go, Rust, and NPM.
134- Plugins should be usable via PURL.
135- Plugins can send/receive HTTP requests.
136- Plugins can be scheduled.
137- Rebase go-git to take advantage of their performance improvements.
138- Generate the plugin SDK automatically.
139- Add plugin SDKs for (at least) Zig, Kotlin, and Python.
140141Yeah, I could go on all night. And those are just core GitRoot features. I haven't even mentioned making instances to build projects in hosted environments. I haven't talked about specific plugins (sending a "toot" on commit, managing secrets with age, translating your app...). I haven't even touched on federating GitRoot with other GitRoot instances, or with Forgejo, SourceHut, or Radicle.
142143All of that is possible, and I’m confident it will be accessible in the future. But "when"? That’s the real question. If you think you can help in any way (coding, of course, but also management, communication, or just to brainstorm), please [let me know](../contact.md).
144145This is only the beginning. GitRoot will get better and better with every release.