forked from vieter-v/vieter
Merge pull request 'Rework of Documentation' (#233) from Chewing_Bever/vieter:docs-rework into dev
Reviewed-on: vieter/vieter#233main
commit
440d1753da
37
README.md
37
README.md
|
|
@ -22,13 +22,6 @@ a while now. I wanted a fast language that I could code while relaxing, without
|
|||
having to exert too much mental effort & V seemed like the right choice for
|
||||
that.
|
||||
|
||||
### Compiler
|
||||
|
||||
Vieter compiles with the standard Vlang compiler. However, I do maintain a
|
||||
[mirror](https://git.rustybever.be/Chewing_Bever/v). This is to ensure my CI
|
||||
does not break without reason, as I control when & how frequently the mirror is
|
||||
updated to reflect the official repository.
|
||||
|
||||
## Features
|
||||
|
||||
* Arch repository server
|
||||
|
|
@ -41,20 +34,24 @@ updated to reflect the official repository.
|
|||
|
||||
## Building
|
||||
|
||||
In order to build Vieter, you'll need a couple of libraries:
|
||||
Besides a V installer, Vieter also requires the following libraries to work:
|
||||
|
||||
* An installation of V
|
||||
* gc
|
||||
* libarchive
|
||||
* openssl
|
||||
* sqlite3
|
||||
|
||||
**NOTE**: if you encounter any issues compiling Vieter using the absolute
|
||||
latest version of V, it might be because my mirror is missing a specific commit
|
||||
that causes issues. For this reason, the `make v` command exists which will
|
||||
clone my compiler in the `v` directory & build it. Afterwards, you can use this
|
||||
compiler with make by prepending all make commands with `V_PATH=v/v`. If you do
|
||||
encounter this issue, please let me know so I can update my mirror & the
|
||||
codebase to fix it!
|
||||
### Compiler
|
||||
|
||||
Vieter compiles with the standard Vlang compiler. However, I do maintain a
|
||||
[mirror](https://git.rustybever.be/vieter/v). This is to ensure my CI does not
|
||||
break without reason, as I control when & how frequently the mirror is updated
|
||||
to reflect the official repository.
|
||||
|
||||
If you encounter issues using the latest V compiler, try using my mirror
|
||||
instead. `make v` will clone the repository & build the mirror. Afterwards,
|
||||
prepending any make command with `V_PATH=v/v` tells make to use the locally
|
||||
compiled mirror instead.
|
||||
|
||||
## Contributing
|
||||
|
||||
|
|
@ -70,7 +67,7 @@ If you wish to contribute to the project, please take note of the following:
|
|||
|
||||
The `docs` directory contains a Hugo site consisting of all user &
|
||||
administrator documentation. `docs/api` on the other hand is a
|
||||
[slate](https://github.com/slatedocs/slate) project describing the HTTP web
|
||||
[Slate](https://github.com/slatedocs/slate) project describing the HTTP web
|
||||
API.
|
||||
|
||||
To modify the Hugo documentation, you'll need to install Hugo. Afterwards, you
|
||||
|
|
@ -81,8 +78,8 @@ can use the following commands inside the `docs` directory:
|
|||
hugo
|
||||
|
||||
# Host an auto-refreshing web server with the documentation. Important to note
|
||||
is that the files will be at `http://localhost:1313/docs/vieter` instead of
|
||||
just `http://localhost:1313/docs/vieter`
|
||||
# is that the files will be at `http://localhost:1313/docs/vieter` instead of
|
||||
# just `http://localhost:1313/`
|
||||
hugo server
|
||||
```
|
||||
|
||||
|
|
@ -96,6 +93,6 @@ docker run \
|
|||
-v $(pwd)/docs/api/source:/srv/slate/source slatedocs/slate serve
|
||||
```
|
||||
|
||||
This'll make the slate docs available at http://localhost:4567. Sadly, this
|
||||
This will make the Slate docs available at http://localhost:4567. Sadly, this
|
||||
server doesn't auto-refresh, so you'll have to manually refresh your browser
|
||||
every time you make a change.
|
||||
|
|
|
|||
|
|
@ -112,7 +112,7 @@ Parameter | Description
|
|||
url | URL of the Git repository.
|
||||
branch | Branch of the Git repository.
|
||||
repo | Vieter repository to publish built packages to.
|
||||
schedule | Cron build schedule
|
||||
schedule | Cron build schedule (syntax explained [here](https://rustybever.be/docs/vieter/usage/builds/schedule/))
|
||||
arch | Comma-separated list of architectures to build package on.
|
||||
|
||||
## Modify a repo
|
||||
|
|
|
|||
|
|
@ -1,11 +1,12 @@
|
|||
# hugo server --minify --themesDir ... --baseURL=http://0.0.0.0:1313/theme/hugo-book/
|
||||
|
||||
baseURL = 'https://rustybever.be/docs/vieter/'
|
||||
title = 'The Rusty Bever - Docs'
|
||||
title = 'Vieter - Docs'
|
||||
theme = 'hugo-book'
|
||||
|
||||
# Book configuration
|
||||
disablePathToLower = true
|
||||
# Doesn't work with docs as subdir
|
||||
enableGitInfo = true
|
||||
|
||||
# Needed for mermaid/katex shortcodes
|
||||
|
|
@ -28,16 +29,16 @@ enableGitInfo = true
|
|||
|
||||
[menu]
|
||||
[[menu.after]]
|
||||
name = "API Documentation"
|
||||
url = "https://rustybever.be/docs/vieter/api"
|
||||
name = "HTTP API Docs"
|
||||
url = "https://rustybever.be/docs/vieter/api/"
|
||||
weight = 10
|
||||
[[menu.after]]
|
||||
name = "Man Pages"
|
||||
url = "https://rustybever.be/man/vieter/vieter.1.html"
|
||||
weight = 20
|
||||
[[menu.after]]
|
||||
name = "Source"
|
||||
url = "https://git.rustybever.be/Chewing_Bever/docs"
|
||||
name = "Vieter"
|
||||
url = "https://git.rustybever.be/vieter/vieter"
|
||||
weight = 30
|
||||
[[menu.after]]
|
||||
name = "Hugo Theme"
|
||||
|
|
@ -69,14 +70,14 @@ enableGitInfo = true
|
|||
|
||||
# Set source repository location.
|
||||
# Used for 'Last Modified' and 'Edit this page' links.
|
||||
BookRepo = 'https://git.rustybever.be/Chewing_Bever/docs'
|
||||
BookRepo = 'https://git.rustybever.be/vieter/vieter'
|
||||
|
||||
# (Optional, default 'commit') Specifies commit portion of the link to the page's last modified
|
||||
# commit hash for 'doc' page type.
|
||||
# Requires 'BookRepo' param.
|
||||
# Value used to construct a URL consisting of BookRepo/BookCommitPath/<commit-hash>
|
||||
# Github uses 'commit', Bitbucket uses 'commits'
|
||||
# BookCommitPath = 'commit'
|
||||
BookCommitPath = 'src/commit'
|
||||
|
||||
# Enable "Edit this page" links for 'doc' page type.
|
||||
# Disabled by default. Uncomment to enable. Requires 'BookRepo' param.
|
||||
|
|
|
|||
|
|
@ -1,27 +0,0 @@
|
|||
# Vieter CLI
|
||||
|
||||
I provide a simple CLI tool that currently only allows changing the Git
|
||||
repository API. Its usage is quite simple.
|
||||
|
||||
First, you need to create a file in your home directory called `.vieterrc` with
|
||||
the following content:
|
||||
|
||||
```toml
|
||||
address = "https://example.com"
|
||||
api_key = "your-api-key"
|
||||
```
|
||||
|
||||
You can also use a different file or use environment variables, as described in
|
||||
[Configuration](/configuration).
|
||||
|
||||
Now you're ready to use the CLI tool.
|
||||
|
||||
## Usage
|
||||
|
||||
* `vieter repos list` returns all repositories currently stored in the API.
|
||||
* `vieter repos add url branch repo arch...` adds the repository with the given
|
||||
URL, branch, repo & arch to the API.
|
||||
* `vieter repos remove id` removes the repository with the given ID prefix.
|
||||
|
||||
You can always check `vieter -help` or `vieter repos -help` for more
|
||||
information about the commands.
|
||||
|
|
@ -9,12 +9,9 @@ documentation might not be relevant anymore for the latest release.
|
|||
|
||||
## Overview
|
||||
|
||||
Vieter has a few main features:
|
||||
|
||||
* It's a simple & lightweight implementation of an Arch repository server
|
||||
* It allows for uploading of built package archives
|
||||
* It supports a basic build system to periodically re-build packages & upload
|
||||
them to the server
|
||||
Vieter consists of two main parts, namely an implementation of an Arch
|
||||
repository server & a scheduling system to periodically build Pacman packages &
|
||||
publish them to a repository.
|
||||
|
||||
{{< hint info >}}
|
||||
**Note**
|
||||
|
|
@ -26,12 +23,12 @@ well.
|
|||
|
||||
### Why?
|
||||
|
||||
Vieter is my personal solution for a problem I've been facing for months:
|
||||
Vieter is my personal solution to a problem I've been facing for months:
|
||||
extremely long AUR package build times. I run EndeavourOS on both my laptops,
|
||||
one of which being a rather old MacBook Air. I really like being a beta-tester
|
||||
for projects & run development builds for multiple packages (nheko,
|
||||
newsflash...). The issue with this is that I have to regularly re-build these
|
||||
packages in order to stay up to date with development & these builds can take a
|
||||
newsflash...). Because of this, I have to regularly re-build these packages in
|
||||
order to stay up to date with development. However, these builds can take a
|
||||
really long time on the old MacBook. This project is a solution to that
|
||||
problem: instead of building the packages locally, I can build them
|
||||
automatically in the cloud & just download them whenever I update my system!
|
||||
|
|
|
|||
|
|
@ -1,56 +0,0 @@
|
|||
# Builder
|
||||
|
||||
Vieter supports a basic build system that allows you to build the packages
|
||||
defined using the Git repositories API by running `vieter build`. For
|
||||
configuration, see [here](/configuration#builder).
|
||||
|
||||
## How it works
|
||||
|
||||
The build system works in two stages. First it pulls down the
|
||||
`archlinux:latest` image from Docker Hub, runs `pacman -Syu` & configures a
|
||||
non-root build user. It then creates a new Docker image from this container.
|
||||
This is to prevent each build having to fully update the container's
|
||||
repositories. After the image has been created, each repository returned by
|
||||
`/api/repos` is built sequentially by starting up a new container with the
|
||||
previously created image as a base. Each container goes through the following steps:
|
||||
|
||||
1. The repository is cloned
|
||||
2. `makepkg --nobuild --syncdeps --needed --noconfirm` is ran to update the `pkgver` variable inside
|
||||
the `PKGBUILD` file
|
||||
3. A HEAD request is sent to the Vieter server to check whether the specific
|
||||
version of the package is already present. If it is, the container exits.
|
||||
4. `makepkg` is ran with `MAKEFLAGS="-j\$(nproc)`
|
||||
5. Each produced package archive is uploaded to the Vieter instance's
|
||||
repository, as defined in the API for that specific Git repo.
|
||||
|
||||
## Cron image
|
||||
|
||||
The Vieter Docker image contains crond & a cron config that runs `vieter build`
|
||||
every night at 3AM. This value is currently hardcoded, but I wish to change
|
||||
that down the line (work is in progress). There's also some other caveats you
|
||||
should be aware of, namely that the image should be run as root & that the
|
||||
healthcheck will always fail, so you might have to disable it. This boils down
|
||||
to the following docker-compose file:
|
||||
|
||||
```yaml
|
||||
version: '3'
|
||||
|
||||
services:
|
||||
cron:
|
||||
image: 'chewingbever/vieter:dev'
|
||||
command: crond -f
|
||||
user: root
|
||||
|
||||
healthcheck:
|
||||
disable: true
|
||||
|
||||
environment:
|
||||
- 'VIETER_API_KEY=some-key'
|
||||
- 'VIETER_ADDRESS=https://example.com'
|
||||
volumes:
|
||||
- '/var/run/docker.sock:/var/run/docker.sock'
|
||||
```
|
||||
|
||||
Important to note is that the container also requires the host's Docker socket
|
||||
to be mounted as this is how it spawns the necessary containers, as well as a
|
||||
change to the container's command.
|
||||
|
|
@ -3,7 +3,7 @@ weight: 20
|
|||
---
|
||||
# Configuration
|
||||
|
||||
All vieter operations by default try to read in the TOML file `~/.vieterrc` for
|
||||
By default, all vieter commands try to read in the TOML file `~/.vieterrc` for
|
||||
configuration. The location of this file can be changed by using the `-f` flag.
|
||||
|
||||
If the above file doesn't exist or you wish to override some of its settings,
|
||||
|
|
@ -19,53 +19,80 @@ the value in the environment variable is used.
|
|||
{{< hint info >}}
|
||||
**Note**
|
||||
All environment variables can also be provided from a file by appending them
|
||||
with `_FILE`. This for example allows you to provide the API key from a docker
|
||||
with `_FILE`. This for example allows you to provide the API key from a Docker
|
||||
secrets file.
|
||||
{{< /hint >}}
|
||||
|
||||
## Modes
|
||||
## Commands
|
||||
|
||||
The vieter binary can run in several "modes", indicated by the first argument
|
||||
passed to them. Each mode requires a different configuration.
|
||||
The first argument passed to Vieter determines which command you wish to use.
|
||||
Each of these can contain subcommands (e.g. `vieter repos list`), but all
|
||||
subcommands will use the same configuration. Below you can find the
|
||||
configuration variable required for each command.
|
||||
|
||||
### Server
|
||||
### `vieter server`
|
||||
|
||||
* `log_level`: defines how much logs to show. Valid values are one of `FATAL`,
|
||||
`ERROR`, `WARN`, `INFO` or `DEBUG`. Defaults to `WARN`
|
||||
* `log_file`: log file to write logs to. Defaults to `vieter.log` in the
|
||||
current directory.
|
||||
* `log_level`: log verbosity level. Value should be one of `FATAL`, `ERROR`,
|
||||
`WARN`, `INFO` or `DEBUG`.
|
||||
* Default: `WARN`
|
||||
* `log_file`: log file to write logs to.
|
||||
* Default: `vieter.log` (in the current directory)
|
||||
* `pkg_dir`: where Vieter should store the actual package archives.
|
||||
* `data_dir`: where Vieter stores the repositories, log file & database.
|
||||
* `api_key`: the API key to use when authenticating requests.
|
||||
* `default_arch`: architecture to always add packages of arch `any` to.
|
||||
* `default_arch`: this setting serves two main purposes:
|
||||
* Packages with architecture `any` are always added to this architecture.
|
||||
This prevents the server from being confused when an `any` package is
|
||||
published as the very first package for a repository.
|
||||
* Git repositories added without an `arch` value use this value instead.
|
||||
|
||||
### Builder
|
||||
|
||||
* `api_key`: the API key to use when authenticating requests.
|
||||
* `address`: Base your URL of your Vieter instance, e.g. https://example.com
|
||||
* `base_image`: image to use when building a package. It should be an Archlinux
|
||||
image. The default if not configured is `archlinux:base-devel`, but this
|
||||
image only supports arm64. If you require aarch64 support as well, consider
|
||||
using
|
||||
### `vieter cron`
|
||||
|
||||
* `log_level`: log verbosity level. Value should be one of `FATAL`, `ERROR`,
|
||||
`WARN`, `INFO` or `DEBUG`.
|
||||
* Default: `WARN`
|
||||
* `log_file`: log file to write logs to.
|
||||
* Default: `vieter.log` (in `data_dir`)
|
||||
* `address`: *public* URL of the Vieter repository server to build for. From
|
||||
this server the list of Git repositories is retrieved. All built packages are
|
||||
published to this server.
|
||||
* `api_key`: API key of the above server.
|
||||
* `data_dir`: directory to store log file in.
|
||||
* `base_image`: Docker image to use when building a package. Any Pacman-based
|
||||
distro image should work, as long as `/etc/pacman.conf` is used &
|
||||
`base-devel` exists in the repositories. Make sure that the image supports
|
||||
the architecture of your cron daemon.
|
||||
* Default: `archlinux:base-devel` (only works on `x86_64`). If you require
|
||||
`aarch64` support, consider using
|
||||
[`menci/archlinuxarm:base-devel`](https://hub.docker.com/r/menci/archlinuxarm)
|
||||
([GH](https://github.com/Menci/docker-archlinuxarm))
|
||||
([GitHub](https://github.com/Menci/docker-archlinuxarm)). This is the image
|
||||
used for the Vieter CI builds.
|
||||
* `max_concurrent_builds`: how many builds to run at the same time.
|
||||
* Default: `1`
|
||||
* `api_update_frequency`: how frequently (in minutes) to poll the Vieter
|
||||
repository server for a new list of Git repositories to build.
|
||||
* Default: `15`
|
||||
* `image_rebuild_frequency`: Vieter periodically builds a builder image using
|
||||
the configured base image. This makes sure build containers do not have to
|
||||
download a lot of packages when updating their system. This setting defines
|
||||
how frequently (in minutes) to rebuild this builder image.
|
||||
* Default: `1440` (every 24 hours)
|
||||
* `global_schedule`: build schedule for any Git repository that does not have a
|
||||
schedule defined. For information about this syntax, see
|
||||
[here](/usage/builds/schedule).
|
||||
* Default: `0 3` (3AM every night)
|
||||
|
||||
### Repos
|
||||
### `vieter logs`
|
||||
|
||||
* `api_key`: the API key to use when authenticating requests.
|
||||
* `address`: Base your URL of your Vieter instance, e.g. https://example.com
|
||||
* `address`: Base URL of your Vieter instance, e.g. https://example.com
|
||||
|
||||
### Cron
|
||||
### `vieter repos`
|
||||
|
||||
* `log_level`: defines how much logs to show. Valid values are one of `FATAL`,
|
||||
`ERROR`, `WARN`, `INFO` or `DEBUG`. Defaults to `WARN`
|
||||
* `api_key`: the API key to use when authenticating requests.
|
||||
* `address`: Base your URL of your Vieter instance, e.g. https://example.com.
|
||||
This *must* be the publicly facing URL of your Vieter instance.
|
||||
* `data_dir`: where Vieter stores the log file.
|
||||
* `base_image`: Docker image from which to create the builder images.
|
||||
* `max_concurrent_builds`: amount of builds to run at once.
|
||||
* `api_update_frequency`: how frequenty to check for changes in the repo list.
|
||||
* `image_rebuild+frequency`: how frequently to rebuild the builder image
|
||||
* `global_schedule`: cron schedule to use for any repo without an individual
|
||||
schedule
|
||||
* `address`: Base URL of your Vieter instance, e.g. https://example.com
|
||||
* `base_image`: image to use when building a package using `vieter repos
|
||||
build`.
|
||||
* Default: `archlinux:base-devel`
|
||||
|
||||
|
|
|
|||
|
|
@ -3,76 +3,100 @@ weight: 10
|
|||
---
|
||||
# Installation
|
||||
|
||||
Vieter consists of a single binary, akin to busybox. The binary's behavior is
|
||||
determined by its CLI arguments, e.g. `vieter server` starts the repository
|
||||
server.
|
||||
|
||||
All installation solutions can be configured the same way,
|
||||
as described [here](/configuration).
|
||||
|
||||
## Docker
|
||||
|
||||
Docker is the recommended way to install vieter. The images can be pulled from
|
||||
[`chewingbever/vieter`](https://hub.docker.com/r/chewingbever/vieter). You can
|
||||
either pull a release tag (e.g. `chewingbever/vieter:0.1.0-rc1`), or pull the
|
||||
`chewingbever/vieter:dev` tag. The latter is updated every time a new commit is
|
||||
pushed to the development branch. This branch will be the most up to date, but
|
||||
does not give any guarantees about stability, so beware!
|
||||
Docker images are published to the
|
||||
[`chewingbever/vieter`](https://hub.docker.com/r/chewingbever/vieter) Docker
|
||||
Hub repository. You can either pull a release tag (e.g.
|
||||
`chewingbever/vieter:0.1.0-rc1`), or pull the `chewingbever/vieter:dev` tag.
|
||||
The latter is updated every time a new commit is pushed to the development
|
||||
branch. This branch will be the most up to date, but does not give any
|
||||
guarantees about stability, so beware!
|
||||
|
||||
The simplest way to run the Docker image is using a plain Docker command:
|
||||
Thanks to the single-binary design of Vieter, this image can be used both for
|
||||
the repository server & the cron daemon.
|
||||
|
||||
```sh
|
||||
docker run \
|
||||
--rm \
|
||||
-d \
|
||||
-v /path/to/data:/data \
|
||||
-e VIETER_API_KEY=changeme \
|
||||
-e VIETER_DEFAULT_ARCH=x86_64 \
|
||||
-p 8000:8000 \
|
||||
chewingbever/vieter:dev
|
||||
Below is an example compose file to set up both the repository server & the
|
||||
cron daemon:
|
||||
|
||||
```yaml
|
||||
version: '3'
|
||||
|
||||
services:
|
||||
server:
|
||||
image: 'chewingbever/vieter:dev'
|
||||
restart: 'always'
|
||||
|
||||
environment:
|
||||
- 'VIETER_API_KEY=secret'
|
||||
- 'VIETER_DEFAULT_ARCH=x86_64'
|
||||
volumes:
|
||||
- 'data:/data'
|
||||
|
||||
cron:
|
||||
image: 'chewingbever/vieter:dev'
|
||||
restart: 'always'
|
||||
user: root
|
||||
command: 'vieter cron'
|
||||
|
||||
environment:
|
||||
- 'VIETER_API_KEY=secret'
|
||||
# MUST be public URL of Vieter repository
|
||||
- 'VIETER_ADDRESS=https://example.com'
|
||||
- 'VIETER_DEFAULT_ARCH=x86_64'
|
||||
- 'VIETER_MAX_CONCURRENT_BUILDS=2'
|
||||
- 'VIETER_GLOBAL_SCHEDULE=0 3'
|
||||
volumes:
|
||||
- '/var/run/docker.sock:/var/run/docker.sock'
|
||||
|
||||
volumes:
|
||||
data:
|
||||
```
|
||||
|
||||
Here, you should change `/path/to/data` to the path on your host where you want
|
||||
vieter to store its files.
|
||||
If you do not require the build system, the repository server can be used
|
||||
independently as well.
|
||||
|
||||
The default configuration will store everything inside the `/data` directory.
|
||||
|
||||
Inside the container, the Vieter server runs on port 8000. This port should be
|
||||
exposed to the public accordingely.
|
||||
|
||||
For an overview of how to configure vieter & which environment variables can be
|
||||
used, see the [Configuration](/configuration) page.
|
||||
{{< hint info >}}
|
||||
**Note**
|
||||
Builds are executed on the cron daemon's system using the host's Docker daemon.
|
||||
A cron daemon on a specific architecture will only build packages for that
|
||||
specific architecture. Therefore, if you wish to build packages for both
|
||||
`x86_64` & `aarch64`, you'll have to deploy two cron daemons, one on each
|
||||
architecture. Afterwards, any Git repositories enabled for those two
|
||||
architectures will build on both.
|
||||
{{< /hint >}}
|
||||
|
||||
## Binary
|
||||
|
||||
On the [releases](https://git.rustybever.be/Chewing_Bever/vieter/releases)
|
||||
page, you can find statically compiled binaries for all released versions. You
|
||||
can download the binary for your host's architecture & run it that way.
|
||||
On the
|
||||
[releases](https://git.rustybever.be/vieter/vieter/releases)
|
||||
page, you can find statically compiled binaries for all
|
||||
released versions. This is the same binary as used inside
|
||||
the Docker images.
|
||||
|
||||
For more information about configuring the binary, check out the
|
||||
[Configuration](/configuration) page.
|
||||
## Arch
|
||||
|
||||
I publish both development & release versions of Vieter to my personal
|
||||
repository, https://arch.r8r.be. Packages are available for `x86_64` &
|
||||
`aarch64`. To use the repository, add the following to your `pacman.conf`:
|
||||
|
||||
```
|
||||
[vieter]
|
||||
Server = https://arch.r8r.be/$repo/$arch
|
||||
SigLevel = Optional
|
||||
```
|
||||
|
||||
Afterwards, you can update your system & install the `vieter` package for the
|
||||
latest official release or `vieter-git` for the latest development release.
|
||||
|
||||
## Building from source
|
||||
|
||||
Because the project is still in heavy development, it might be useful to build
|
||||
from source instead. Luckily, this process is very easy. You'll need make,
|
||||
libarchive & openssl; all of which should be present on an every-day Arch
|
||||
install. Then, after cloning the repository, you can use the following commands:
|
||||
|
||||
```sh
|
||||
# Builds the compiler; should usually only be ran once. Vieter compiles using
|
||||
# the default compiler, but I maintain my own mirror to ensure nothing breaks
|
||||
# without me knowing.
|
||||
make v
|
||||
|
||||
# Build vieter
|
||||
# Alternatively, use `make prod` to build the production build.
|
||||
make
|
||||
```
|
||||
{{< hint info >}}
|
||||
**Note**
|
||||
My version of the V compiler is also available on my Vieter instance,
|
||||
https://arch.r8r.be. It's in the `vieter` repository, with the package being
|
||||
named `vieter-v`. The compiler is available for both x86_64 & aarch64.
|
||||
{{< /hint >}}
|
||||
|
||||
## My Vieter instance
|
||||
|
||||
Besides uploading development Docker images, my CI also publishes x86_64 &
|
||||
aarch64 packages to my personal Vieter instance, https://arch.r8r.be. If you'd
|
||||
like, you can use this repository as well by adding it to your Pacman
|
||||
configuration as described [here](/usage#configuring-pacman). Both the
|
||||
repository & the package are called `vieter`.
|
||||
The project [README](https://git.rustybever.be/vieter/vieter#building) contains
|
||||
instructions for building Vieter from source.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,3 @@
|
|||
---
|
||||
weight: 100
|
||||
---
|
||||
|
|
@ -0,0 +1,81 @@
|
|||
# Builds In-depth
|
||||
|
||||
For those interested, this page describes how the build system works
|
||||
internally.
|
||||
|
||||
## Builder image
|
||||
|
||||
Every cron daemon perodically creates a builder image that is then used as a
|
||||
base for all builds. This is done to prevent build containers having to pull
|
||||
down a bunch of updates when they update their system.
|
||||
|
||||
The build container is created by running the following commands inside a
|
||||
container started from the image defined in `base_image`:
|
||||
|
||||
```sh
|
||||
# Update repos & install required packages
|
||||
pacman -Syu --needed --noconfirm base-devel git
|
||||
# Add a non-root user to run makepkg
|
||||
groupadd -g 1000 builder
|
||||
useradd -mg builder builder
|
||||
# Make sure they can use sudo without a password
|
||||
echo 'builder ALL=(ALL) NOPASSWD: ALL' >> /etc/sudoers
|
||||
# Create the directory for the builds & make it writeable for the
|
||||
# build user
|
||||
mkdir /build
|
||||
chown -R builder:builder /build
|
||||
```
|
||||
|
||||
This script updates the packages to their latest versions & creates a non-root
|
||||
user to use when running `makepkg`.
|
||||
|
||||
This script is base64-encoded & passed to the container as an environment
|
||||
variable. The container's entrypoint is set to `/bin/sh -c` & its command
|
||||
argument to `echo $BUILD_SCRIPT | base64 -d | /bin/sh -e`, with the
|
||||
`BUILD_SCRIPT` environment variable containing the base64-encoded script.
|
||||
|
||||
Once the container exits, a new Docker image is created from it. This image is
|
||||
then used as the base for any builds.
|
||||
|
||||
## Running builds
|
||||
|
||||
Each build has its own Docker container, using the builder image as its base.
|
||||
The same base64-based technique as above is used, just with a different script.
|
||||
To make the build logs more clear, each command is appended by an echo command
|
||||
printing the next command to stdout.
|
||||
|
||||
Given the Git repository URL is `https://examplerepo.com` with branch `main`,
|
||||
the URL of the Vieter server is `https://example.com` and `vieter` is the
|
||||
repository we wish to publish to, we get the following script:
|
||||
|
||||
```sh
|
||||
echo -e '+ echo -e '\''[vieter]\\nServer = https://example.com/$repo/$arch\\nSigLevel = Optional'\'' >> /etc/pacman.conf'
|
||||
echo -e '[vieter]\nServer = https://example.com/$repo/$arch\nSigLevel = Optional' >> /etc/pacman.conf
|
||||
echo -e '+ pacman -Syu --needed --noconfirm'
|
||||
pacman -Syu --needed --noconfirm
|
||||
echo -e '+ su builder'
|
||||
su builder
|
||||
echo -e '+ git clone --single-branch --depth 1 --branch main https://examplerepo.com repo'
|
||||
git clone --single-branch --depth 1 --branch main https://examplerepo.com repo
|
||||
echo -e '+ cd repo'
|
||||
cd repo
|
||||
echo -e '+ makepkg --nobuild --syncdeps --needed --noconfirm'
|
||||
makepkg --nobuild --syncdeps --needed --noconfirm
|
||||
echo -e '+ source PKGBUILD'
|
||||
source PKGBUILD
|
||||
echo -e '+ curl -s --head --fail https://example.com/vieter/x86_64/$pkgname-$pkgver-$pkgrel && exit 0'
|
||||
curl -s --head --fail https://example.com/vieter/x86_64/$pkgname-$pkgver-$pkgrel && exit 0
|
||||
echo -e '+ [ "$(id -u)" == 0 ] && exit 0'
|
||||
[ "$(id -u)" == 0 ] && exit 0
|
||||
echo -e '+ MAKEFLAGS="-j$(nproc)" makepkg -s --noconfirm --needed && for pkg in $(ls -1 *.pkg*); do curl -XPOST -T "$pkg" -H "X-API-KEY: $API_KEY" https://example.com/vieter/publish; done'
|
||||
MAKEFLAGS="-j$(nproc)" makepkg -s --noconfirm --needed && for pkg in $(ls -1 *.pkg*); do curl -XPOST -T "$pkg" -H "X-API-KEY: $API_KEY" https://example.com/vieter/publish; done
|
||||
```
|
||||
|
||||
This script:
|
||||
|
||||
1. Adds the target repository as a repository in the build container
|
||||
2. Updates mirrors & packages
|
||||
3. Clones the Git repository
|
||||
4. Runs `makepkg` without building to calculate `pkgver`
|
||||
5. Checks whether the package version is already present on the server
|
||||
6. If not, run `makepkg` & publish any generated package archives to the server
|
||||
|
|
@ -1,54 +0,0 @@
|
|||
---
|
||||
weight: 30
|
||||
---
|
||||
# Usage
|
||||
|
||||
## Starting the server
|
||||
|
||||
To start a server, either install it using Docker (see
|
||||
[Installation](/installation)) or run it locally by executing `vieter
|
||||
server`. See [Configuration](/configuration) for more information about
|
||||
configuring the binary.
|
||||
|
||||
## Multiple repositories
|
||||
|
||||
Vieter works with multiple repositories. This means that a single Vieter server
|
||||
can serve multiple repositories in Pacman. It also automatically divides files
|
||||
with specific architectures among arch-repos. Arch-repos are the actual
|
||||
repositories you add to your `/etc/pacman.conf` file. See [Configuring
|
||||
Pacman](/usage#configuring-pacman) below for more info.
|
||||
|
||||
## Adding packages
|
||||
|
||||
Using Vieter is currently very simple. If you wish to add a package to Vieter,
|
||||
build it using makepkg & POST that file to the `/<repo>/publish` endpoint of
|
||||
your server. This will add the package to the repository. Authentification
|
||||
requires you to add the API key as the `X-Api-Key` header.
|
||||
|
||||
All of this can be combined into a simple cURL call:
|
||||
|
||||
```
|
||||
curl -XPOST -H "X-API-KEY: your-key" -T some-package.pkg.tar.zst https://example.com/somerepo/publish
|
||||
```
|
||||
|
||||
`somerepo` is automatically created if it doesn't exist yet.
|
||||
|
||||
## Configuring Pacman
|
||||
|
||||
Configuring Pacman to use a Vieter instance is very simple. In your
|
||||
`/etc/pacman.conf` file, add the following lines:
|
||||
|
||||
```
|
||||
[vieter]
|
||||
Server = https://example.com/$repo/$arch
|
||||
SigLevel = Optional
|
||||
```
|
||||
|
||||
Here, you see two important placeholder variables. `$repo` is replaced by the
|
||||
name within the square brackets, which in this case would be `vieter`. `$arch`
|
||||
is replaced by the output of `uname -m`. Because Vieter supports multiple
|
||||
repositories & architectures per repository, using this notation makes sure you
|
||||
always use the correct endpoint for fetching files.
|
||||
|
||||
I recommend placing this below all other repository entries, as the order
|
||||
decides which repository should be used if there's ever a naming conflict.
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
---
|
||||
weight: 30
|
||||
---
|
||||
|
|
@ -0,0 +1,51 @@
|
|||
---
|
||||
weight: 20
|
||||
---
|
||||
# Building packages
|
||||
|
||||
The automatic build system is what makes Vieter very useful as a replacement
|
||||
for an AUR helper. It can perodically build packages & publish them to your
|
||||
personal Vieter repository server, removing the need to build the packages
|
||||
locally.
|
||||
|
||||
## Adding builds
|
||||
|
||||
Before the cron system can start building your package, you need to add its
|
||||
info to the system. The Vieter repository server exposes an HTTP API for this
|
||||
(see the [HTTP API Docs](https://rustybever.be/docs/vieter/api/) for more
|
||||
info). For ease of use, the Vieter binary contains a CLI interface for
|
||||
interacting with this API (see [Configuration](/configuration) for
|
||||
configuration details). The [man
|
||||
pages](https://rustybever.be/man/vieter/vieter-repos.1.html) describe this in
|
||||
greater detail, but the basic usage is as follows:
|
||||
|
||||
```
|
||||
vieter repos add some-url some-branch some-repository
|
||||
```
|
||||
|
||||
Here, `some-url` is the URL of the Git repository containing the PKGBUILD. This
|
||||
URL is passed to `git clone`, meaning the repository should be public. Vieter
|
||||
expects the same format as an AUR Git repository, so you can directly use AUR
|
||||
URLs here.
|
||||
|
||||
`some-branch` is the branch of the Git repository the build should check out.
|
||||
If you're using an AUR package, this should be `master`.
|
||||
|
||||
Finally, `some-repo` is the repository to which the built package archives
|
||||
should be published.
|
||||
|
||||
The above command intentionally leaves out a few parameters to make the CLI
|
||||
more useable. For information on how to modify all parameters using the CLI,
|
||||
see
|
||||
[vieter-repos-edit(1)](https://rustybever.be/man/vieter/vieter-repos-edit.1.html).
|
||||
|
||||
## Reading logs
|
||||
|
||||
The logs of each build are uploaded to the Vieter repository server, along with
|
||||
information about the exit code of the build container, when the build
|
||||
started/ended etc. These logs can then be accessed using the [HTTP
|
||||
API](https://rustybever.be/docs/vieter/api/).
|
||||
|
||||
For ease of use, the logs are also available using some CLI commands; see
|
||||
[vieter-logs(1)](https://rustybever.be/man/vieter/vieter-logs.1.html) for more
|
||||
information.
|
||||
|
|
@ -0,0 +1,42 @@
|
|||
# Cron schedule syntax
|
||||
|
||||
The Vieter cron daemon uses a subset of the cron expression syntax to schedule
|
||||
builds.
|
||||
|
||||
## Format
|
||||
|
||||
`a b c d`
|
||||
|
||||
* `a`: minutes
|
||||
* `b`: hours
|
||||
* `c`: days
|
||||
* `d`: months
|
||||
|
||||
An expression consists of two to four sections. If less than four sections are
|
||||
provided, the parser will append `*` until there are four sections. This means
|
||||
that `0 3` is the same as `0 3 * *`.
|
||||
|
||||
Each section consists of one or more parts, separated by a comma. Each of these
|
||||
parts, in turn, can be one of the following (any letters are integers):
|
||||
|
||||
* `*`: allow all possible values.
|
||||
* `a`: only this value is allowed.
|
||||
* `*/n`: allow every n-th value.
|
||||
* `a/n`: allow every n-th value, starting at a in the list.
|
||||
* `a-b`: allow every value between a and b, bounds included.
|
||||
* `a-b/n`: allow every n-th value inside the list of values between a and b,
|
||||
bounds included.
|
||||
|
||||
Each section can consist of as many of these parts as necessary.
|
||||
|
||||
## Examples
|
||||
|
||||
* `0 3`: every day at 03:00AM.
|
||||
* `0 0 */7`: every 7th day of the month, at midnight.
|
||||
|
||||
## CLI tool
|
||||
|
||||
The Vieter binary contains a command that shows you the next matching times for
|
||||
a given expression. This can be useful to understand the syntax. For more
|
||||
information, see
|
||||
[vieter-schedule(1)](https://rustybever.be/man/vieter/vieter-schedule.1.html).
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
---
|
||||
weight: 10
|
||||
---
|
||||
# Pacman repository
|
||||
|
||||
The part of Vieter that users will interact with the most is the Pacman
|
||||
repository aka `vieter server`.
|
||||
|
||||
## Design overview
|
||||
|
||||
A Vieter repository server has support for multiple repositories, with each
|
||||
repository containing packages for multiple architectures.
|
||||
|
||||
If you wish to use these repositories on your system, add the following to
|
||||
`/etc/pacman.conf` for each repository you wish to use:
|
||||
|
||||
```
|
||||
[repo-name]
|
||||
Server = https://example.com/$repo/$arch
|
||||
SigLevel = Optional
|
||||
```
|
||||
|
||||
Here, `$repo` and `$arch` are not variables you have to fill in yourself.
|
||||
Rather, Pacman will substitute these when reading the config file. `$repo` is
|
||||
replaced by the name between the square brackets (in this case `repo-name`),
|
||||
and `$arch` is replaced by your system's architecture, e.g. `x86_64`. Of
|
||||
course, you can also fill in these values manually yourself, e.g. if you wish
|
||||
to use a different name inside the square brackets.
|
||||
|
||||
Important to note is that, when two repositories contain a package with the
|
||||
same name, Pacman will choose the one from the repository that's highest up in
|
||||
the `pacman.conf` file. Therefore, if you know your repository has packages
|
||||
with the same name as ones from the official repositories, it might be better
|
||||
to place the repository below the official repositories to avoid overwriting
|
||||
official packages.
|
||||
|
||||
## Publishing packages
|
||||
|
||||
Packages can be easily published using a single HTTP POST request. Check out
|
||||
the [HTTP API docs](https://rustybever.be/docs/vieter/api/) for more info on
|
||||
these routes, including example cURL commands.
|
||||
Loading…
Reference in New Issue