4.7 KiB
| title | date |
|---|---|
| Vieter 0.5.0 | 2022-12-29 |
As 2022 comes to a close, and in the middle of exams, I'm ready to release another version of Vieter!
What's Vieter?
As usual, a small refresher on what Vieter actually is for the new readers. Vieter provides two main services: a Pacman repository server and a build system for publishing Arch packages to said server. The goal is to fully replace any need for an AUR helper, with the AUR packages being built "in the cloud" instead. Of course, one can also publish their own packages to this server, allowing you to create your very own customized Arch repositories!
What's changed?
Server-agent architecture
The biggest change this version introduces is the migration to a polling-based server-agent architecture.
Previous versions relied on a "cron daemon". This daemon needed to be deployed on every architecture the user wanted to build packages for, with the daemon periodically polling the server for the list of targets to build. All scheduling was done on the node performing the builds.
While this system served me well for a while, it did limit possibilities for improvement. Building on multiple servers wasn't possible, as the cron daemons had no way of synchronizing with each other, meaning they'd all run all builds. There was also no way for the server (or the user for that matter) to control these daemons; they had a fixed build schedule that could only be changed by changing a target's configuration.
Due to these limitations, I've decided to revamp the build system & convert it to a server-agent architecture! With this new system, the main server handles all scheduling. On each server running builds, a "build agent" is deployed which periodically polls the main server for new jobs. This allows a Vieter instance to run builds on an arbitrary number of build nodes! Thanks to this, I'm able to run 137 builds in under 40 minutes, whereas before, I needed this time to process less than half of that :) As a bit of a stress test for my instance, I've started replicating the EndeavourOS repository for fun.
With the main server now being in control of scheduling, I've also been able to implement manual scheduling of builds on the server. If a package needs to be rebuilt, you can simply send an HTTP request (or use the accompanying CLI command) to schedule a build job.
Quality-of-life improvements
This release was mostly the build system redesign, but I've also added some QoL improvements. Noteable additions are the option to periodically remove logs, as an active Vieter instance can collect thousands over time. The CLI tool should now also be a lot more stable, and will correctly display HTTP errors if needed. There's also the option to specify what subdirectory of a Git repository to use for a build. This is for example useful when building packages using a Git repository containing multiple PKGBUILDs.
What's to come?
My brain never stops, so I still have a lot of cool ideas I wish to implement in the coming months.
For starters: better build awareness. The build system right now does not track the progress of jobs. Once a job is dispatched to an agent, the main server forgets about it and moves on. If a build fails due to unknown means causing the logs to never be uploaded, it'll be as if the build never happened. I'd like to change this. I want the main server to be aware of what jobs are running, have failed, or have perhaps timed out. This information could then be made available through the API, providing the user with valuable insight into the workings of their Vieter instance.
Building on this idea, I wish to know what specific command caused the build to fail. Was it makepkg, an HTTP error? And if it was makepkg, what error code? can the build system respond to it by itself? The main goal is to provide a deeper understanding into the workings of builds and the build system as a whole.
Another big one to tackle is API access to the repositories. These are currently only accessible through Pacman, but having this information available as a convenient REST API, useable from the CLI tool, sounds like a valuable asset. This would pave the way towards repository-level configuration.
Of course there's a lot more ideas, but the list would be too long to put here ;p
Conclusion
As you can see, I still have a lot of ideas for Vieter. As usual however, I can't predict when any of these features will get implemented. It all depends on whether the uni life leaves me some time :)
If you're interested in Vieter, considering joining #vieter:rustybever.be on Matrix! The source code can be found on my personal Gitea.