Compare commits
	
		
			8 Commits 
		
	
	
		
			64ef3cb847
			...
			ddccceb336
		
	
	| Author | SHA1 | Date | 
|---|---|---|
|  | ddccceb336 | |
|  | c15f4a482f | |
|  | c341d7a024 | |
|  | a9ddfd8ec8 | |
|  | 0ab39a334d | |
|  | d7d77afe09 | |
|  | 4ecf6a11c4 | |
|  | 5df5850044 | 
							
								
								
									
										37
									
								
								README.md
								
								
								
								
							
							
						
						
									
										37
									
								
								README.md
								
								
								
								
							|  | @ -22,13 +22,6 @@ a while now. I wanted a fast language that I could code while relaxing, without | |||
| having to exert too much mental effort & V seemed like the right choice for | ||||
| that. | ||||
| 
 | ||||
| ### Compiler | ||||
| 
 | ||||
| Vieter compiles with the standard Vlang compiler. However, I do maintain a | ||||
| [mirror](https://git.rustybever.be/Chewing_Bever/v). This is to ensure my CI | ||||
| does not break without reason, as I control when & how frequently the mirror is | ||||
| updated to reflect the official repository. | ||||
| 
 | ||||
| ## Features | ||||
| 
 | ||||
| * Arch repository server | ||||
|  | @ -41,20 +34,24 @@ updated to reflect the official repository. | |||
| 
 | ||||
| ## Building | ||||
| 
 | ||||
| In order to build Vieter, you'll need a couple of libraries: | ||||
| Besides a V installer, Vieter also requires the following libraries to work: | ||||
| 
 | ||||
| * An installation of V | ||||
| * gc | ||||
| * libarchive | ||||
| * openssl | ||||
| * sqlite3 | ||||
| 
 | ||||
| **NOTE**: if you encounter any issues compiling Vieter using the absolute | ||||
| latest version of V, it might be because my mirror is missing a specific commit | ||||
| that causes issues. For this reason, the `make v` command exists which will | ||||
| clone my compiler in the `v` directory & build it. Afterwards, you can use this | ||||
| compiler with make by prepending all make commands with `V_PATH=v/v`. If you do | ||||
| encounter this issue, please let me know so I can update my mirror & the | ||||
| codebase to fix it! | ||||
| ### Compiler | ||||
| 
 | ||||
| Vieter compiles with the standard Vlang compiler. However, I do maintain a | ||||
| [mirror](https://git.rustybever.be/vieter/v). This is to ensure my CI does not | ||||
| break without reason, as I control when & how frequently the mirror is updated | ||||
| to reflect the official repository. | ||||
| 
 | ||||
| If you encounter issues using the latest V compiler, try using my mirror | ||||
| instead. `make v` will clone the repository & build the mirror. Afterwards, | ||||
| prepending any make command with `V_PATH=v/v` tells make to use the locally | ||||
| compiled mirror instead. | ||||
| 
 | ||||
| ## Contributing | ||||
| 
 | ||||
|  | @ -70,7 +67,7 @@ If you wish to contribute to the project, please take note of the following: | |||
| 
 | ||||
| The `docs` directory contains a Hugo site consisting of all user & | ||||
| administrator documentation. `docs/api` on the other hand is a | ||||
| [slate](https://github.com/slatedocs/slate) project describing the HTTP web | ||||
| [Slate](https://github.com/slatedocs/slate) project describing the HTTP web | ||||
| API. | ||||
| 
 | ||||
| To modify the Hugo documentation, you'll need to install Hugo. Afterwards, you | ||||
|  | @ -81,8 +78,8 @@ can use the following commands inside the `docs` directory: | |||
| hugo | ||||
| 
 | ||||
| # Host an auto-refreshing web server with the documentation. Important to note | ||||
| is that the files will be at `http://localhost:1313/docs/vieter` instead of | ||||
| just `http://localhost:1313/docs/vieter` | ||||
| # is that the files will be at `http://localhost:1313/docs/vieter` instead of | ||||
| # just `http://localhost:1313/` | ||||
| hugo server | ||||
| ``` | ||||
| 
 | ||||
|  | @ -96,6 +93,6 @@ docker run \ | |||
|     -v $(pwd)/docs/api/source:/srv/slate/source slatedocs/slate serve | ||||
| ``` | ||||
| 
 | ||||
| This'll make the slate docs available at http://localhost:4567. Sadly, this | ||||
| This will make the Slate docs available at http://localhost:4567. Sadly, this | ||||
| server doesn't auto-refresh, so you'll have to manually refresh your browser | ||||
| every time you make a change. | ||||
|  |  | |||
|  | @ -112,7 +112,7 @@ Parameter | Description | |||
| url | URL of the Git repository. | ||||
| branch | Branch of the Git repository. | ||||
| repo | Vieter repository to publish built packages to. | ||||
| schedule | Cron build schedule | ||||
| schedule | Cron build schedule (syntax explained [here](https://rustybever.be/docs/vieter/usage/builds/schedule/)) | ||||
| arch | Comma-separated list of architectures to build package on. | ||||
| 
 | ||||
| ## Modify a repo | ||||
|  |  | |||
|  | @ -1,11 +1,12 @@ | |||
| # hugo server --minify --themesDir ... --baseURL=http://0.0.0.0:1313/theme/hugo-book/ | ||||
| 
 | ||||
| baseURL = 'https://rustybever.be/docs/vieter/' | ||||
| title = 'The Rusty Bever - Docs' | ||||
| title = 'Vieter - Docs' | ||||
| theme = 'hugo-book' | ||||
| 
 | ||||
| # Book configuration | ||||
| disablePathToLower = true | ||||
| # Doesn't work with docs as subdir | ||||
| enableGitInfo = true | ||||
| 
 | ||||
| # Needed for mermaid/katex shortcodes | ||||
|  | @ -28,16 +29,16 @@ enableGitInfo = true | |||
| 
 | ||||
| [menu] | ||||
| [[menu.after]] | ||||
|   name = "API Documentation" | ||||
|   url = "https://rustybever.be/docs/vieter/api" | ||||
|   name = "HTTP API Docs" | ||||
|   url = "https://rustybever.be/docs/vieter/api/" | ||||
|   weight = 10 | ||||
| [[menu.after]] | ||||
|   name = "Man Pages" | ||||
|   url = "https://rustybever.be/man/vieter/vieter.1.html" | ||||
|   weight = 20 | ||||
| [[menu.after]] | ||||
|   name = "Source" | ||||
|   url = "https://git.rustybever.be/Chewing_Bever/docs" | ||||
|   name = "Vieter" | ||||
|   url = "https://git.rustybever.be/vieter/vieter" | ||||
|   weight = 30 | ||||
| [[menu.after]] | ||||
|   name = "Hugo Theme" | ||||
|  | @ -69,14 +70,14 @@ enableGitInfo = true | |||
| 
 | ||||
|   # Set source repository location. | ||||
|   # Used for 'Last Modified' and 'Edit this page' links. | ||||
|   BookRepo = 'https://git.rustybever.be/Chewing_Bever/docs' | ||||
|   BookRepo = 'https://git.rustybever.be/vieter/vieter' | ||||
| 
 | ||||
|   # (Optional, default 'commit') Specifies commit portion of the link to the page's last modified | ||||
|   # commit hash for 'doc' page type. | ||||
|   # Requires 'BookRepo' param. | ||||
|   # Value used to construct a URL consisting of BookRepo/BookCommitPath/<commit-hash> | ||||
|   # Github uses 'commit', Bitbucket uses 'commits' | ||||
|   # BookCommitPath = 'commit' | ||||
|   BookCommitPath = 'src/commit' | ||||
| 
 | ||||
|   # Enable "Edit this page" links for 'doc' page type. | ||||
|   # Disabled by default. Uncomment to enable. Requires 'BookRepo' param. | ||||
|  |  | |||
|  | @ -1,27 +0,0 @@ | |||
| # Vieter CLI | ||||
| 
 | ||||
| I provide a simple CLI tool that currently only allows changing the Git | ||||
| repository API. Its usage is quite simple. | ||||
| 
 | ||||
| First, you need to create a file in your home directory called `.vieterrc` with | ||||
| the following content: | ||||
| 
 | ||||
| ```toml | ||||
| address = "https://example.com" | ||||
| api_key = "your-api-key" | ||||
| ``` | ||||
| 
 | ||||
| You can also use a different file or use environment variables, as described in | ||||
| [Configuration](/configuration). | ||||
| 
 | ||||
| Now you're ready to use the CLI tool. | ||||
| 
 | ||||
| ## Usage | ||||
| 
 | ||||
| * `vieter repos list` returns all repositories currently stored in the API. | ||||
| * `vieter repos add url branch repo arch...` adds the repository with the given | ||||
|   URL, branch, repo & arch to the API. | ||||
| * `vieter repos remove id` removes the repository with the given ID prefix. | ||||
| 
 | ||||
| You can always check `vieter -help` or `vieter repos -help` for more | ||||
| information about the commands. | ||||
|  | @ -9,12 +9,9 @@ documentation might not be relevant anymore for the latest release. | |||
| 
 | ||||
| ## Overview | ||||
| 
 | ||||
| Vieter has a few main features: | ||||
| 
 | ||||
| * It's a simple & lightweight implementation of an Arch repository server | ||||
| * It allows for uploading of built package archives | ||||
| * It supports a basic build system to periodically re-build packages & upload | ||||
|   them to the server | ||||
| Vieter consists of two main parts, namely an implementation of an Arch | ||||
| repository server & a scheduling system to periodically build Pacman packages & | ||||
| publish them to a repository. | ||||
| 
 | ||||
| {{< hint info >}} | ||||
| **Note**   | ||||
|  | @ -26,12 +23,12 @@ well. | |||
| 
 | ||||
| ### Why? | ||||
| 
 | ||||
| Vieter is my personal solution for a problem I've been facing for months: | ||||
| Vieter is my personal solution to a problem I've been facing for months: | ||||
| extremely long AUR package build times. I run EndeavourOS on both my laptops, | ||||
| one of which being a rather old MacBook Air. I really like being a beta-tester | ||||
| for projects & run development builds for multiple packages (nheko, | ||||
| newsflash...). The issue with this is that I have to regularly re-build these | ||||
| packages in order to stay up to date with development & these builds can take a | ||||
| newsflash...). Because of this, I have to regularly re-build these packages in | ||||
| order to stay up to date with development. However, these builds can take a | ||||
| really long time on the old MacBook. This project is a solution to that | ||||
| problem: instead of building the packages locally, I can build them | ||||
| automatically in the cloud & just download them whenever I update my system! | ||||
|  |  | |||
|  | @ -1,56 +0,0 @@ | |||
| # Builder | ||||
| 
 | ||||
| Vieter supports a basic build system that allows you to build the packages | ||||
| defined using the Git repositories API by running `vieter build`. For | ||||
| configuration, see [here](/configuration#builder). | ||||
| 
 | ||||
| ## How it works | ||||
| 
 | ||||
| The build system works in two stages. First it pulls down the | ||||
| `archlinux:latest` image from Docker Hub, runs `pacman -Syu` & configures a | ||||
| non-root build user. It then creates a new Docker image from this container. | ||||
| This is to prevent each build having to fully update the container's | ||||
| repositories. After the image has been created, each repository returned by | ||||
| `/api/repos` is built sequentially by starting up a new container with the | ||||
| previously created image as a base. Each container goes through the following steps: | ||||
| 
 | ||||
| 1. The repository is cloned | ||||
| 2. `makepkg --nobuild --syncdeps --needed --noconfirm` is ran to update the `pkgver` variable inside | ||||
|    the `PKGBUILD` file | ||||
| 3. A HEAD request is sent to the Vieter server to check whether the specific | ||||
|    version of the package is already present. If it is, the container exits. | ||||
| 4. `makepkg` is ran with `MAKEFLAGS="-j\$(nproc)` | ||||
| 5. Each produced package archive is uploaded to the Vieter instance's | ||||
|    repository, as defined in the API for that specific Git repo. | ||||
| 
 | ||||
| ## Cron image | ||||
| 
 | ||||
| The Vieter Docker image contains crond & a cron config that runs `vieter build` | ||||
| every night at 3AM. This value is currently hardcoded, but I wish to change | ||||
| that down the line (work is in progress). There's also some other caveats you | ||||
| should be aware of, namely that the image should be run as root & that the | ||||
| healthcheck will always fail, so you might have to disable it. This boils down | ||||
| to the following docker-compose file: | ||||
| 
 | ||||
| ```yaml | ||||
| version: '3' | ||||
| 
 | ||||
| services: | ||||
|   cron: | ||||
|     image: 'chewingbever/vieter:dev' | ||||
|     command: crond -f | ||||
|     user: root | ||||
| 
 | ||||
|     healthcheck: | ||||
|       disable: true | ||||
|          | ||||
|     environment: | ||||
|       - 'VIETER_API_KEY=some-key' | ||||
|       - 'VIETER_ADDRESS=https://example.com' | ||||
|     volumes: | ||||
|       - '/var/run/docker.sock:/var/run/docker.sock' | ||||
| ``` | ||||
| 
 | ||||
| Important to note is that the container also requires the host's Docker socket | ||||
| to be mounted as this is how it spawns the necessary containers, as well as a | ||||
| change to the container's command. | ||||
|  | @ -3,7 +3,7 @@ weight: 20 | |||
| --- | ||||
| # Configuration | ||||
| 
 | ||||
| All vieter operations by default try to read in the TOML file `~/.vieterrc` for | ||||
| By default, all vieter commands try to read in the TOML file `~/.vieterrc` for | ||||
| configuration. The location of this file can be changed by using the `-f` flag. | ||||
| 
 | ||||
| If the above file doesn't exist or you wish to override some of its settings, | ||||
|  | @ -19,53 +19,80 @@ the value in the environment variable is used. | |||
| {{< hint info >}} | ||||
| **Note**   | ||||
| All environment variables can also be provided from a file by appending them | ||||
| with `_FILE`. This for example allows you to provide the API key from a docker | ||||
| with `_FILE`. This for example allows you to provide the API key from a Docker | ||||
| secrets file. | ||||
| {{< /hint >}} | ||||
| 
 | ||||
| ## Modes | ||||
| ## Commands | ||||
| 
 | ||||
| The vieter binary can run in several "modes", indicated by the first argument | ||||
| passed to them. Each mode requires a different configuration. | ||||
| The first argument passed to Vieter determines which command you wish to use. | ||||
| Each of these can contain subcommands (e.g. `vieter repos list`), but all | ||||
| subcommands will use the same configuration. Below you can find the | ||||
| configuration variable required for each command. | ||||
| 
 | ||||
| ### Server | ||||
| ### `vieter server` | ||||
| 
 | ||||
| * `log_level`: defines how much logs to show. Valid values are one of `FATAL`, | ||||
|   `ERROR`, `WARN`, `INFO` or `DEBUG`. Defaults to `WARN` | ||||
| * `log_file`: log file to write logs to. Defaults to `vieter.log` in the | ||||
|   current directory. | ||||
| * `log_level`: log verbosity level. Value should be one of `FATAL`, `ERROR`, | ||||
|   `WARN`, `INFO` or `DEBUG`. | ||||
|     * Default: `WARN` | ||||
| * `log_file`: log file to write logs to. | ||||
|     * Default: `vieter.log` (in the current directory) | ||||
| * `pkg_dir`:  where Vieter should store the actual package archives. | ||||
| * `data_dir`: where Vieter stores the repositories, log file & database. | ||||
| * `api_key`: the API key to use when authenticating requests. | ||||
| * `default_arch`: architecture to always add packages of arch `any` to. | ||||
| * `default_arch`: this setting serves two main purposes: | ||||
|     * Packages with architecture `any` are always added to this architecture. | ||||
|       This prevents the server from being confused when an `any` package is | ||||
|       published as the very first package for a repository. | ||||
|     * Git repositories added without an `arch` value use this value instead. | ||||
| 
 | ||||
| ### Builder | ||||
| 
 | ||||
| ### `vieter cron` | ||||
| 
 | ||||
| * `log_level`: log verbosity level. Value should be one of `FATAL`, `ERROR`, | ||||
|   `WARN`, `INFO` or `DEBUG`. | ||||
|     * Default: `WARN` | ||||
| * `log_file`: log file to write logs to. | ||||
|     * Default: `vieter.log` (in `data_dir`) | ||||
| * `address`: *public* URL of the Vieter repository server to build for. From | ||||
|   this server the list of Git repositories is retrieved. All built packages are | ||||
|   published to this server. | ||||
| * `api_key`: API key of the above server. | ||||
| * `data_dir`: directory to store log file in. | ||||
| * `base_image`: Docker image to use when building a package. Any Pacman-based | ||||
|   distro image should work, as long as `/etc/pacman.conf` is used & | ||||
|   `base-devel` exists in the repositories. Make sure that the image supports | ||||
|   the architecture of your cron daemon. | ||||
|     * Default: `archlinux:base-devel` (only works on `x86_64`). If you require | ||||
|       `aarch64` support, consider using | ||||
|       [`menci/archlinuxarm:base-devel`](https://hub.docker.com/r/menci/archlinuxarm) | ||||
|       ([GitHub](https://github.com/Menci/docker-archlinuxarm)). This is the image | ||||
|       used for the Vieter CI builds. | ||||
| * `max_concurrent_builds`: how many builds to run at the same time. | ||||
|     * Default: `1` | ||||
| * `api_update_frequency`: how frequently (in minutes) to poll the Vieter | ||||
|   repository server for a new list of Git repositories to build. | ||||
|     * Default: `15` | ||||
| * `image_rebuild_frequency`: Vieter periodically builds a builder image using | ||||
|   the configured base image. This makes sure build containers do not have to | ||||
|   download a lot of packages when updating their system. This setting defines | ||||
|   how frequently (in minutes) to rebuild this builder image. | ||||
|     * Default: `1440` (every 24 hours) | ||||
| * `global_schedule`: build schedule for any Git repository that does not have a | ||||
|   schedule defined. For information about this syntax, see | ||||
|   [here](/usage/builds/schedule). | ||||
|     * Default: `0 3` (3AM every night) | ||||
| 
 | ||||
| ### `vieter logs` | ||||
| 
 | ||||
| * `api_key`: the API key to use when authenticating requests. | ||||
| * `address`: Base your URL of your Vieter instance, e.g. https://example.com | ||||
| * `base_image`: image to use when building a package. It should be an Archlinux | ||||
|   image. The default if not configured is `archlinux:base-devel`, but this | ||||
|   image only supports arm64. If you require aarch64 support as well, consider | ||||
|   using | ||||
|   [`menci/archlinuxarm:base-devel`](https://hub.docker.com/r/menci/archlinuxarm) | ||||
|   ([GH](https://github.com/Menci/docker-archlinuxarm)) | ||||
| * `address`: Base URL of your Vieter instance, e.g. https://example.com | ||||
| 
 | ||||
| ### Repos | ||||
| ### `vieter repos` | ||||
| 
 | ||||
| * `api_key`: the API key to use when authenticating requests. | ||||
| * `address`: Base your URL of your Vieter instance, e.g. https://example.com | ||||
| * `address`: Base URL of your Vieter instance, e.g. https://example.com | ||||
| * `base_image`: image to use when building a package using `vieter repos | ||||
|   build`. | ||||
|     * Default: `archlinux:base-devel` | ||||
| 
 | ||||
| ### Cron | ||||
| 
 | ||||
| * `log_level`: defines how much logs to show. Valid values are one of `FATAL`, | ||||
|   `ERROR`, `WARN`, `INFO` or `DEBUG`. Defaults to `WARN` | ||||
| * `api_key`: the API key to use when authenticating requests. | ||||
| * `address`: Base your URL of your Vieter instance, e.g. https://example.com. | ||||
|   This *must* be the publicly facing URL of your Vieter instance. | ||||
| * `data_dir`: where Vieter stores the log file. | ||||
| * `base_image`: Docker image from which to create the builder images. | ||||
| * `max_concurrent_builds`: amount of builds to run at once. | ||||
| * `api_update_frequency`: how frequenty to check for changes in the repo list. | ||||
| * `image_rebuild+frequency`: how frequently to rebuild the builder image | ||||
| * `global_schedule`: cron schedule to use for any repo without an individual | ||||
|   schedule | ||||
|  |  | |||
|  | @ -3,76 +3,100 @@ weight: 10 | |||
| --- | ||||
| # Installation | ||||
| 
 | ||||
| Vieter consists of a single binary, akin to busybox. The binary's behavior is | ||||
| determined by its CLI arguments, e.g. `vieter server` starts the repository | ||||
| server. | ||||
| 
 | ||||
| All installation solutions can be configured the same way, | ||||
| as described [here](/configuration). | ||||
| 
 | ||||
| ## Docker | ||||
| 
 | ||||
| Docker is the recommended way to install vieter. The images can be pulled from | ||||
| [`chewingbever/vieter`](https://hub.docker.com/r/chewingbever/vieter). You can | ||||
| either pull a release tag (e.g. `chewingbever/vieter:0.1.0-rc1`), or pull the | ||||
| `chewingbever/vieter:dev` tag. The latter is updated every time a new commit is | ||||
| pushed to the development branch. This branch will be the most up to date, but | ||||
| does not give any guarantees about stability, so beware! | ||||
| Docker images are published to the | ||||
| [`chewingbever/vieter`](https://hub.docker.com/r/chewingbever/vieter) Docker | ||||
| Hub repository. You can either pull a release tag (e.g. | ||||
| `chewingbever/vieter:0.1.0-rc1`), or pull the `chewingbever/vieter:dev` tag. | ||||
| The latter is updated every time a new commit is pushed to the development | ||||
| branch. This branch will be the most up to date, but does not give any | ||||
| guarantees about stability, so beware! | ||||
| 
 | ||||
| The simplest way to run the Docker image is using a plain Docker command: | ||||
| Thanks to the single-binary design of Vieter, this image can be used both for | ||||
| the repository server & the cron daemon. | ||||
| 
 | ||||
| ```sh | ||||
| docker run \ | ||||
|     --rm \ | ||||
|     -d \ | ||||
|     -v /path/to/data:/data \ | ||||
|     -e VIETER_API_KEY=changeme \ | ||||
|     -e VIETER_DEFAULT_ARCH=x86_64 \ | ||||
|     -p 8000:8000 \ | ||||
|     chewingbever/vieter:dev | ||||
| Below is an example compose file to set up both the repository server & the | ||||
| cron daemon: | ||||
| 
 | ||||
| ```yaml | ||||
| version: '3' | ||||
| 
 | ||||
| services: | ||||
|   server: | ||||
|     image: 'chewingbever/vieter:dev' | ||||
|     restart: 'always' | ||||
| 
 | ||||
|     environment: | ||||
|       - 'VIETER_API_KEY=secret' | ||||
|       - 'VIETER_DEFAULT_ARCH=x86_64' | ||||
|     volumes: | ||||
|       - 'data:/data' | ||||
| 
 | ||||
|   cron: | ||||
|     image: 'chewingbever/vieter:dev' | ||||
|     restart: 'always' | ||||
|     user: root | ||||
|     command: 'vieter cron' | ||||
| 
 | ||||
|     environment: | ||||
|       - 'VIETER_API_KEY=secret' | ||||
|       # MUST be public URL of Vieter repository | ||||
|       - 'VIETER_ADDRESS=https://example.com' | ||||
|       - 'VIETER_DEFAULT_ARCH=x86_64' | ||||
|       - 'VIETER_MAX_CONCURRENT_BUILDS=2' | ||||
|       - 'VIETER_GLOBAL_SCHEDULE=0 3' | ||||
|     volumes: | ||||
|       - '/var/run/docker.sock:/var/run/docker.sock' | ||||
| 
 | ||||
| volumes: | ||||
|   data: | ||||
| ``` | ||||
| 
 | ||||
| Here, you should change `/path/to/data` to the path on your host where you want | ||||
| vieter to store its files. | ||||
| If you do not require the build system, the repository server can be used | ||||
| independently as well. | ||||
| 
 | ||||
| The default configuration will store everything inside the `/data` directory. | ||||
| 
 | ||||
| Inside the container, the Vieter server runs on port 8000. This port should be | ||||
| exposed to the public accordingely. | ||||
| 
 | ||||
| For an overview of how to configure vieter & which environment variables can be | ||||
| used, see the [Configuration](/configuration) page. | ||||
| {{< hint info >}} | ||||
| **Note**   | ||||
| Builds are executed on the cron daemon's system using the host's Docker daemon. | ||||
| A cron daemon on a specific architecture will only build packages for that | ||||
| specific architecture. Therefore, if you wish to build packages for both | ||||
| `x86_64` & `aarch64`, you'll have to deploy two cron daemons, one on each | ||||
| architecture. Afterwards, any Git repositories enabled for those two | ||||
| architectures will build on both. | ||||
| {{< /hint >}} | ||||
| 
 | ||||
| ## Binary | ||||
| 
 | ||||
| On the [releases](https://git.rustybever.be/Chewing_Bever/vieter/releases) | ||||
| page, you can find statically compiled binaries for all released versions. You | ||||
| can download the binary for your host's architecture & run it that way. | ||||
| On the | ||||
| [releases](https://git.rustybever.be/vieter/vieter/releases) | ||||
| page, you can find statically compiled binaries for all | ||||
| released versions. This is the same binary as used inside | ||||
| the Docker images. | ||||
| 
 | ||||
| For more information about configuring the binary, check out the | ||||
| [Configuration](/configuration) page. | ||||
| ## Arch | ||||
| 
 | ||||
| I publish both development & release versions of Vieter to my personal | ||||
| repository, https://arch.r8r.be. Packages are available for `x86_64` & | ||||
| `aarch64`. To use the repository, add the following to your `pacman.conf`: | ||||
| 
 | ||||
| ``` | ||||
| [vieter] | ||||
| Server = https://arch.r8r.be/$repo/$arch | ||||
| SigLevel = Optional | ||||
| ``` | ||||
| 
 | ||||
| Afterwards, you can update your system & install the `vieter` package for the | ||||
| latest official release or `vieter-git` for the latest development release. | ||||
| 
 | ||||
| ## Building from source | ||||
| 
 | ||||
| Because the project is still in heavy development, it might be useful to build | ||||
| from source instead. Luckily, this process is very easy. You'll need make, | ||||
| libarchive & openssl; all of which should be present on an every-day Arch | ||||
| install. Then, after cloning the repository, you can use the following commands: | ||||
| 
 | ||||
| ```sh | ||||
| # Builds the compiler; should usually only be ran once. Vieter compiles using | ||||
| # the default compiler, but I maintain my own mirror to ensure nothing breaks | ||||
| # without me knowing. | ||||
| make v | ||||
| 
 | ||||
| # Build vieter | ||||
| # Alternatively, use `make prod` to build the production build. | ||||
| make | ||||
| ``` | ||||
| {{< hint info >}} | ||||
| **Note**   | ||||
| My version of the V compiler is also available on my Vieter instance, | ||||
| https://arch.r8r.be. It's in the `vieter` repository, with the package being | ||||
| named `vieter-v`. The compiler is available for both x86_64 & aarch64. | ||||
| {{< /hint >}} | ||||
| 
 | ||||
| ## My Vieter instance | ||||
| 
 | ||||
| Besides uploading development Docker images, my CI also publishes x86_64 & | ||||
| aarch64 packages to my personal Vieter instance, https://arch.r8r.be. If you'd | ||||
| like, you can use this repository as well by adding it to your Pacman | ||||
| configuration as described [here](/usage#configuring-pacman). Both the | ||||
| repository & the package are called `vieter`. | ||||
| The project [README](https://git.rustybever.be/vieter/vieter#building) contains | ||||
| instructions for building Vieter from source. | ||||
|  |  | |||
|  | @ -0,0 +1,3 @@ | |||
| --- | ||||
| weight: 100 | ||||
| --- | ||||
|  | @ -0,0 +1,81 @@ | |||
| # Builds In-depth | ||||
| 
 | ||||
| For those interested, this page describes how the build system works | ||||
| internally. | ||||
| 
 | ||||
| ## Builder image | ||||
| 
 | ||||
| Every cron daemon perodically creates a builder image that is then used as a | ||||
| base for all builds. This is done to prevent build containers having to pull | ||||
| down a bunch of updates when they update their system. | ||||
| 
 | ||||
| The build container is created by running the following commands inside a | ||||
| container started from the image defined in `base_image`: | ||||
| 
 | ||||
| ```sh | ||||
| # Update repos & install required packages | ||||
| pacman -Syu --needed --noconfirm base-devel git | ||||
| # Add a non-root user to run makepkg | ||||
| groupadd -g 1000 builder | ||||
| useradd -mg builder builder | ||||
| # Make sure they can use sudo without a password | ||||
| echo 'builder ALL=(ALL) NOPASSWD: ALL' >> /etc/sudoers | ||||
| # Create the directory for the builds & make it writeable for the | ||||
| # build user | ||||
| mkdir /build | ||||
| chown -R builder:builder /build | ||||
| ``` | ||||
| 
 | ||||
| This script updates the packages to their latest versions & creates a non-root | ||||
| user to use when running `makepkg`. | ||||
| 
 | ||||
| This script is base64-encoded & passed to the container as an environment | ||||
| variable. The container's entrypoint is set to `/bin/sh -c` & its command | ||||
| argument to `echo $BUILD_SCRIPT | base64 -d | /bin/sh -e`, with the | ||||
| `BUILD_SCRIPT` environment variable containing the base64-encoded script. | ||||
| 
 | ||||
| Once the container exits, a new Docker image is created from it. This image is | ||||
| then used as the base for any builds. | ||||
| 
 | ||||
| ## Running builds | ||||
| 
 | ||||
| Each build has its own Docker container, using the builder image as its base. | ||||
| The same base64-based technique as above is used, just with a different script. | ||||
| To make the build logs more clear, each command is appended by an echo command | ||||
| printing the next command to stdout. | ||||
| 
 | ||||
| Given the Git repository URL is `https://examplerepo.com` with branch `main`, | ||||
| the URL of the Vieter server is `https://example.com` and `vieter` is the | ||||
| repository we wish to publish to, we get the following script: | ||||
| 
 | ||||
| ```sh | ||||
| echo -e '+ echo -e '\''[vieter]\\nServer = https://example.com/$repo/$arch\\nSigLevel = Optional'\'' >> /etc/pacman.conf' | ||||
| echo -e '[vieter]\nServer = https://example.com/$repo/$arch\nSigLevel = Optional' >> /etc/pacman.conf | ||||
| echo -e '+ pacman -Syu --needed --noconfirm' | ||||
| pacman -Syu --needed --noconfirm | ||||
| echo -e '+ su builder' | ||||
| su builder | ||||
| echo -e '+ git clone --single-branch --depth 1 --branch main https://examplerepo.com repo' | ||||
| git clone --single-branch --depth 1 --branch main https://examplerepo.com repo | ||||
| echo -e '+ cd repo' | ||||
| cd repo | ||||
| echo -e '+ makepkg --nobuild --syncdeps --needed --noconfirm' | ||||
| makepkg --nobuild --syncdeps --needed --noconfirm | ||||
| echo -e '+ source PKGBUILD' | ||||
| source PKGBUILD | ||||
| echo -e '+ curl -s --head --fail https://example.com/vieter/x86_64/$pkgname-$pkgver-$pkgrel && exit 0' | ||||
| curl -s --head --fail https://example.com/vieter/x86_64/$pkgname-$pkgver-$pkgrel && exit 0 | ||||
| echo -e '+ [ "$(id -u)" == 0 ] && exit 0' | ||||
| [ "$(id -u)" == 0 ] && exit 0 | ||||
| echo -e '+ MAKEFLAGS="-j$(nproc)" makepkg -s --noconfirm --needed && for pkg in $(ls -1 *.pkg*); do curl -XPOST -T "$pkg" -H "X-API-KEY: $API_KEY" https://example.com/vieter/publish; done' | ||||
| MAKEFLAGS="-j$(nproc)" makepkg -s --noconfirm --needed && for pkg in $(ls -1 *.pkg*); do curl -XPOST -T "$pkg" -H "X-API-KEY: $API_KEY" https://example.com/vieter/publish; done | ||||
| ``` | ||||
| 
 | ||||
| This script: | ||||
| 
 | ||||
| 1. Adds the target repository as a repository in the build container | ||||
| 2. Updates mirrors & packages | ||||
| 3. Clones the Git repository | ||||
| 4. Runs `makepkg` without building to calculate `pkgver` | ||||
| 5. Checks whether the package version is already present on the server | ||||
| 6. If not, run `makepkg` & publish any generated package archives to the server | ||||
|  | @ -1,54 +0,0 @@ | |||
| --- | ||||
| weight: 30 | ||||
| --- | ||||
| # Usage | ||||
| 
 | ||||
| ## Starting the server | ||||
| 
 | ||||
| To start a server, either install it using Docker (see | ||||
| [Installation](/installation)) or run it locally by executing `vieter | ||||
| server`. See [Configuration](/configuration) for more information about | ||||
| configuring the binary. | ||||
| 
 | ||||
| ## Multiple repositories | ||||
| 
 | ||||
| Vieter works with multiple repositories. This means that a single Vieter server | ||||
| can serve multiple repositories in Pacman. It also automatically divides files | ||||
| with specific architectures among arch-repos. Arch-repos are the actual | ||||
| repositories you add to your `/etc/pacman.conf` file. See [Configuring | ||||
| Pacman](/usage#configuring-pacman) below for more info. | ||||
| 
 | ||||
| ## Adding packages | ||||
| 
 | ||||
| Using Vieter is currently very simple. If you wish to add a package to Vieter, | ||||
| build it using makepkg & POST that file to the `/<repo>/publish` endpoint of | ||||
| your server. This will add the package to the repository. Authentification | ||||
| requires you to add the API key as the `X-Api-Key` header. | ||||
| 
 | ||||
| All of this can be combined into a simple cURL call: | ||||
| 
 | ||||
| ``` | ||||
| curl -XPOST -H "X-API-KEY: your-key" -T some-package.pkg.tar.zst https://example.com/somerepo/publish | ||||
| ``` | ||||
| 
 | ||||
| `somerepo` is automatically created if it doesn't exist yet. | ||||
| 
 | ||||
| ## Configuring Pacman | ||||
| 
 | ||||
| Configuring Pacman to use a Vieter instance is very simple. In your | ||||
| `/etc/pacman.conf` file, add the following lines: | ||||
| 
 | ||||
| ``` | ||||
| [vieter] | ||||
| Server = https://example.com/$repo/$arch | ||||
| SigLevel = Optional | ||||
| ``` | ||||
| 
 | ||||
| Here, you see two important placeholder variables. `$repo` is replaced by the | ||||
| name within the square brackets, which in this case would be `vieter`. `$arch` | ||||
| is replaced by the output of `uname -m`. Because Vieter supports multiple | ||||
| repositories & architectures per repository, using this notation makes sure you | ||||
| always use the correct endpoint for fetching files. | ||||
| 
 | ||||
| I recommend placing this below all other repository entries, as the order | ||||
| decides which repository should be used if there's ever a naming conflict. | ||||
|  | @ -0,0 +1,3 @@ | |||
| --- | ||||
| weight: 30 | ||||
| --- | ||||
|  | @ -0,0 +1,51 @@ | |||
| --- | ||||
| weight: 20 | ||||
| --- | ||||
| # Building packages | ||||
| 
 | ||||
| The automatic build system is what makes Vieter very useful as a replacement | ||||
| for an AUR helper. It can perodically build packages & publish them to your | ||||
| personal Vieter repository server, removing the need to build the packages | ||||
| locally. | ||||
| 
 | ||||
| ## Adding builds | ||||
| 
 | ||||
| Before the cron system can start building your package, you need to add its | ||||
| info to the system. The Vieter repository server exposes an HTTP API for this | ||||
| (see the [HTTP API Docs](https://rustybever.be/docs/vieter/api/) for more | ||||
| info). For ease of use, the Vieter binary contains a CLI interface for | ||||
| interacting with this API (see [Configuration](/configuration) for | ||||
| configuration details). The [man | ||||
| pages](https://rustybever.be/man/vieter/vieter-repos.1.html) describe this in | ||||
| greater detail, but the basic usage is as follows: | ||||
| 
 | ||||
| ``` | ||||
| vieter repos add some-url some-branch some-repository | ||||
| ``` | ||||
| 
 | ||||
| Here, `some-url` is the URL of the Git repository containing the PKGBUILD. This | ||||
| URL is passed to `git clone`, meaning the repository should be public. Vieter | ||||
| expects the same format as an AUR Git repository, so you can directly use AUR | ||||
| URLs here. | ||||
| 
 | ||||
| `some-branch` is the branch of the Git repository the build should check out. | ||||
| If you're using an AUR package, this should be `master`. | ||||
| 
 | ||||
| Finally, `some-repo` is the repository to which the built package archives | ||||
| should be published. | ||||
| 
 | ||||
| The above command intentionally leaves out a few parameters to make the CLI | ||||
| more useable. For information on how to modify all parameters using the CLI, | ||||
| see | ||||
| [vieter-repos-edit(1)](https://rustybever.be/man/vieter/vieter-repos-edit.1.html). | ||||
| 
 | ||||
| ## Reading logs | ||||
| 
 | ||||
| The logs of each build are uploaded to the Vieter repository server, along with | ||||
| information about the exit code of the build container, when the build | ||||
| started/ended etc. These logs can then be accessed using the [HTTP | ||||
| API](https://rustybever.be/docs/vieter/api/). | ||||
| 
 | ||||
| For ease of use, the logs are also available using some CLI commands; see | ||||
| [vieter-logs(1)](https://rustybever.be/man/vieter/vieter-logs.1.html) for more | ||||
| information. | ||||
|  | @ -0,0 +1,42 @@ | |||
| # Cron schedule syntax | ||||
| 
 | ||||
| The Vieter cron daemon uses a subset of the cron expression syntax to schedule | ||||
| builds. | ||||
| 
 | ||||
| ## Format | ||||
| 
 | ||||
| `a b c d` | ||||
| 
 | ||||
| * `a`: minutes | ||||
| * `b`: hours | ||||
| * `c`: days | ||||
| * `d`: months | ||||
| 
 | ||||
| An expression consists of two to four sections. If less than four sections are | ||||
| provided, the parser will append `*` until there are four sections. This means | ||||
| that `0 3` is the same as `0 3 * *`. | ||||
| 
 | ||||
| Each section consists of one or more parts, separated by a comma. Each of these | ||||
| parts, in turn, can be one of the following (any letters are integers): | ||||
| 
 | ||||
| * `*`: allow all possible values. | ||||
| * `a`: only this value is allowed. | ||||
| * `*/n`: allow every n-th value. | ||||
| * `a/n`: allow every n-th value, starting at a in the list. | ||||
| * `a-b`: allow every value between a and b, bounds included. | ||||
| * `a-b/n`: allow every n-th value inside the list of values between a and b, | ||||
|   bounds included. | ||||
| 
 | ||||
| Each section can consist of as many of these parts as necessary. | ||||
| 
 | ||||
| ## Examples | ||||
| 
 | ||||
| * `0 3`: every day at 03:00AM. | ||||
| * `0 0 */7`: every 7th day of the month, at midnight. | ||||
| 
 | ||||
| ## CLI tool | ||||
| 
 | ||||
| The Vieter binary contains a command that shows you the next matching times for | ||||
| a given expression. This can be useful to understand the syntax. For more | ||||
| information, see | ||||
| [vieter-schedule(1)](https://rustybever.be/man/vieter/vieter-schedule.1.html). | ||||
|  | @ -0,0 +1,41 @@ | |||
| --- | ||||
| weight: 10 | ||||
| --- | ||||
| # Pacman repository | ||||
| 
 | ||||
| The part of Vieter that users will interact with the most is the Pacman | ||||
| repository aka `vieter server`. | ||||
| 
 | ||||
| ## Design overview | ||||
| 
 | ||||
| A Vieter repository server has support for multiple repositories, with each | ||||
| repository containing packages for multiple architectures. | ||||
| 
 | ||||
| If you wish to use these repositories on your system, add the following to | ||||
| `/etc/pacman.conf` for each repository you wish to use: | ||||
| 
 | ||||
| ``` | ||||
| [repo-name] | ||||
| Server = https://example.com/$repo/$arch | ||||
| SigLevel = Optional | ||||
| ``` | ||||
| 
 | ||||
| Here, `$repo` and `$arch` are not variables you have to fill in yourself. | ||||
| Rather, Pacman will substitute these when reading the config file. `$repo` is | ||||
| replaced by the name between the square brackets (in this case `repo-name`), | ||||
| and `$arch` is replaced by your system's architecture, e.g. `x86_64`. Of | ||||
| course, you can also fill in these values manually yourself, e.g. if you wish | ||||
| to use a different name inside the square brackets. | ||||
| 
 | ||||
| Important to note is that, when two repositories contain a package with the | ||||
| same name, Pacman will choose the one from the repository that's highest up in | ||||
| the `pacman.conf` file. Therefore, if you know your repository has packages | ||||
| with the same name as ones from the official repositories, it might be better | ||||
| to place the repository below the official repositories to avoid overwriting | ||||
| official packages. | ||||
| 
 | ||||
| ## Publishing packages | ||||
| 
 | ||||
| Packages can be easily published using a single HTTP POST request. Check out | ||||
| the [HTTP API docs](https://rustybever.be/docs/vieter/api/) for more info on | ||||
| these routes, including example cURL commands. | ||||
		Loading…
	
		Reference in New Issue