100 lines
4.7 KiB
Markdown
100 lines
4.7 KiB
Markdown
|
---
|
||
|
|
title: "Vieter 0.5.0"
|
||
|
|
date: 2022-12-29
|
||
|
|
---
|
||
|
|
|
||
|
|
As 2022 comes to a close, and in the middle of exams, I'm ready to release
|
||
|
|
another version of Vieter!
|
||
|
|
|
||
|
|
## What's Vieter?
|
||
|
|
|
||
|
|
As usual, a small refresher on what Vieter actually is for the new readers.
|
||
|
|
Vieter provides two main services: a Pacman repository server and a build
|
||
|
|
system for publishing Arch packages to said server. The goal is to fully
|
||
|
|
replace any need for an AUR helper, with the AUR packages being built "in the
|
||
|
|
cloud" instead. Of course, one can also publish their own packages to this
|
||
|
|
server, allowing you to create your very own customized Arch repositories!
|
||
|
|
|
||
|
|
## What's changed?
|
||
|
|
|
||
|
|
### Server-agent architecture
|
||
|
|
|
||
|
|
The biggest change this version introduces is the migration to a polling-based
|
||
|
|
server-agent architecture.
|
||
|
|
|
||
|
|
Previous versions relied on a "cron daemon". This daemon needed to be deployed
|
||
|
|
on every architecture the user wanted to build packages for, with the daemon
|
||
|
|
periodically polling the server for the list of targets to build. All
|
||
|
|
scheduling was done on the node performing the builds.
|
||
|
|
|
||
|
|
While this system served me well for a while, it did limit possibilities for
|
||
|
|
improvement. Building on multiple servers wasn't possible, as the cron daemons
|
||
|
|
had no way of synchronizing with each other, meaning they'd all run all builds.
|
||
|
|
There was also no way for the server (or the user for that matter) to control
|
||
|
|
these daemons; they had a fixed build schedule that could only be changed by
|
||
|
|
changing a target's configuration.
|
||
|
|
|
||
|
|
Due to these limitations, I've decided to revamp the build system & convert it
|
||
|
|
to a server-agent architecture! With this new system, the main server handles
|
||
|
|
all scheduling. On each server running builds, a "build agent" is deployed
|
||
|
|
which periodically polls the main server for new jobs. This allows a Vieter
|
||
|
|
instance to run builds on an arbitrary number of build nodes! Thanks to this,
|
||
|
|
I'm able to run 137 builds in under 40 minutes, whereas before, I needed this
|
||
|
|
time to process less than half of that :) As a bit of a stress test for my
|
||
|
|
instance, I've started replicating the EndeavourOS repository for fun.
|
||
|
|
|
||
|
|
With the main server now being in control of scheduling, I've also been able to
|
||
|
|
implement manual scheduling of builds on the server. If a package needs to be
|
||
|
|
rebuilt, you can simply send an HTTP request (or use the accompanying CLI
|
||
|
|
command) to schedule a build job.
|
||
|
|
|
||
|
|
### Quality-of-life improvements
|
||
|
|
|
||
|
|
This release was mostly the build system redesign, but I've also added some QoL
|
||
|
|
improvements. Noteable additions are the option to periodically remove logs, as
|
||
|
|
an active Vieter instance can collect thousands over time. The CLI tool should
|
||
|
|
now also be a lot more stable, and will correctly display HTTP errors if
|
||
|
|
needed. There's also the option to specify what subdirectory of a Git
|
||
|
|
repository to use for a build. This is for example useful when building
|
||
|
|
packages using a Git repository [containing multiple
|
||
|
|
PKGBUILDs](https://github.com/endeavouros-team/PKGBUILDS).
|
||
|
|
|
||
|
|
## What's to come?
|
||
|
|
|
||
|
|
My brain never stops, so I still have a lot of cool ideas I wish to implement
|
||
|
|
in the coming months.
|
||
|
|
|
||
|
|
For starters: better build awareness. The build system right now does not track
|
||
|
|
the progress of jobs. Once a job is dispatched to an agent, the main server
|
||
|
|
forgets about it and moves on. If a build fails due to unknown means causing
|
||
|
|
the logs to never be uploaded, it'll be as if the build never happened. I'd
|
||
|
|
like to change this. I want the main server to be aware of what jobs are
|
||
|
|
running, have failed, or have perhaps timed out. This information could then be
|
||
|
|
made available through the API, providing the user with valuable insight into
|
||
|
|
the workings of their Vieter instance.
|
||
|
|
|
||
|
|
Building on this idea, I wish to know what specific command caused the build to
|
||
|
|
fail. Was it makepkg, an HTTP error? And if it was makepkg, what error code?
|
||
|
|
can the build system respond to it by itself? The main goal is to provide a
|
||
|
|
deeper understanding into the workings of builds and the build system as a
|
||
|
|
whole.
|
||
|
|
|
||
|
|
Another big one to tackle is API access to the repositories. These are
|
||
|
|
currently only accessible through Pacman, but having this information available
|
||
|
|
as a convenient REST API, useable from the CLI tool, sounds like a valuable
|
||
|
|
asset. This would pave the way towards repository-level configuration.
|
||
|
|
|
||
|
|
Of course there's a lot more ideas, but the list would be too long to put here
|
||
|
|
;p
|
||
|
|
|
||
|
|
## Conclusion
|
||
|
|
|
||
|
|
As you can see, I still have *a lot* of ideas for Vieter. As usual however, I
|
||
|
|
can't predict when any of these features will get implemented. It all depends
|
||
|
|
on whether the uni life leaves me some time :)
|
||
|
|
|
||
|
|
If you're interested in Vieter, considering joining
|
||
|
|
[#vieter:rustybever.be](https://matrix.to/#/#vieter:rustybever.be) on Matrix!
|
||
|
|
The source code can be found on my personal
|
||
|
|
[Gitea](https://git.rustybever.be/vieter-v/vieter).
|