vieter-v
/
vieter
2
1
Fork 5
Code Issues 84 Pull Requests 4 Packages Projects Releases 12 Wiki Activity

Some breaking API changes #245

Merged
Jef Roosens merged 8 commits from Chewing_Bever/vieter:api-changes into dev 2022-06-16 18:17:53 +02:00
Conversation 0 Commits 8 Files Changed 31
31 changed files with 366 additions and 336 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
CHANGELOG.md
View File

@ -7,6 +7,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased](https://git.rustybever.be/vieter/vieter/src/branch/dev)
### Added
* Server port can now be configured
### Changed
* Moved all API routes under `/v1` namespace
* Renamed `vieter repos` to `vieter targets`
* Renamed `/api/v1/repos` namespace to `/api/v1/targets`
## [0.3.0](https://git.rustybever.be/vieter/vieter/src/tag/0.3.0)
Nothing besides bumping the versions.

10
docs/api/source/includes/_logs.md
View File

@ -24,7 +24,7 @@ curl \
"data": [
{
"id": 1,
"repo_id": 3,
"target_id": 3,
"start_time": 1652008554,
"end_time": 1652008559,
"arch": "x86_64",
@ -46,7 +46,7 @@ Parameter | Description
--------- | -----------
limit | Maximum amount of results to return.
offset | Offset of results.
repo | Only return builds published to this repository.
target | Only return builds for this target id.
before | Only return logs started before this time (UTC epoch)
after | Only return logs started after this time (UTC epoch)
arch | Only return logs built on this architecture
@ -58,7 +58,7 @@ exit_codes | Comma-separated list of exit codes to limit result to; using `!` as
```shell
curl \
-H 'X-Api-Key: secret' \
https://example.com/api/logs/15
https://example.com/api/logs/1
```
> JSON output format
@ -68,7 +68,7 @@ curl \
"message": "",
"data": {
"id": 1,
"repo_id": 3,
"target_id": 3,
"start_time": 1652008554,
"end_time": 1652008559,
"arch": "x86_64",
@ -129,11 +129,11 @@ Publish a new build log to the server.
Parameter | Description
--------- | -----------
id | ID of requested log
startTime | Start time of the build (UTC epoch)
endTime | End time of the build (UTC epoch)
arch | Architecture on which the build was done
exitCode | Exit code of the build container
target | id of target this build is for
### Request body

51
docs/api/source/includes/_git.md → docs/api/source/includes/_targets.md
View File

@ -1,4 +1,4 @@
# Git Repositories
# Targets
<aside class="notice">
@ -6,15 +6,14 @@ All routes in this section require authentication.
</aside>
Endpoints for interacting with the list of Git repositories stored on the
server.
Endpoints for interacting with the list of targets stored on the server.
## List repos
## List targets
```shell
curl \
-H 'X-Api-Key: secret' \
https://example.com/api/repos?offset=10&limit=20
https://example.com/api/targets?offset=10&limit=20
```
> JSON output format
@ -32,7 +31,7 @@ curl \
"arch": [
{
"id": 1,
"repo_id": 1,
"target_id": 1,
"value": "x86_64"
}
]
@ -41,11 +40,11 @@ curl \
}
```
Retrieve a list of Git repositories.
Retrieve a list of targets.
### HTTP Request
`GET /api/repos`
`GET /api/targets`
### Query Parameters
@ -53,14 +52,14 @@ Parameter | Description
--------- | -----------
limit | Maximum amount of results to return.
offset | Offset of results.
repo | Limit results to repositories that publish to the given repo.
repo | Limit results to targets that publish to the given repo.
## Get a repo
## Get specific target
```shell
curl \
-H 'X-Api-Key: secret' \
https://example.com/api/repos/15
https://example.com/api/targets/1
```
> JSON output format
@ -77,7 +76,7 @@ curl \
"arch": [
{
"id": 1,
"repo_id": 1,
"target_id": 1,
"value": "x86_64"
}
]
@ -85,25 +84,25 @@ curl \
}
```
Get info about a specific Git repository.
Get info about a specific target.
### HTTP Request
`GET /api/repos/:id`
`GET /api/targets/:id`
### URL Parameters
Parameter | Description
--------- | -----------
id | ID of requested repo
id | id of requested target
## Create a new repo
## Create a new target
Create a new Git repository with the given data.
Create a new target with the given data.
### HTTP Request
`POST /api/repos`
`POST /api/targets`
### Query Parameters
@ -115,19 +114,19 @@ repo | Vieter repository to publish built packages to.
schedule | Cron build schedule (syntax explained [here](https://rustybever.be/docs/vieter/usage/builds/schedule/))
arch | Comma-separated list of architectures to build package on.
## Modify a repo
## Modify a target
Modify the data of an existing Git repository.
Modify the data of an existing target.
### HTTP Request
`PATCH /api/repos/:id`
`PATCH /api/targets/:id`
### URL Parameters
Parameter | Description
--------- | -----------
id | ID of requested repo
id | id of target to modify
### Query Parameters
@ -139,16 +138,16 @@ repo | Vieter repository to publish built packages to.
schedule | Cron build schedule
arch | Comma-separated list of architectures to build package on.
## Remove a repo
## Remove a target
Remove a Git repository from the server.
Remove a target from the server.
### HTTP Request
`DELETE /api/repos/:id`
`DELETE /api/targets/:id`
### URL Parameters
Parameter | Description
--------- | -----------
id | ID of repo to remove
id | id of target to remove

2
docs/api/source/index.html.md
View File

@ -9,7 +9,7 @@ toc_footers:
includes:
- repository
- git
- targets
- logs
search: true

9
docs/content/configuration.md
View File

@ -26,7 +26,7 @@ secrets file.
## Commands
The first argument passed to Vieter determines which command you wish to use.
Each of these can contain subcommands (e.g. `vieter repos list`), but all
Each of these can contain subcommands (e.g. `vieter targets list`), but all
subcommands will use the same configuration. Below you can find the
configuration variable required for each command.
@ -45,7 +45,8 @@ configuration variable required for each command.
This prevents the server from being confused when an `any` package is
published as the very first package for a repository.
* Git repositories added without an `arch` value use this value instead.
* `port`: HTTP port to run on
* Default: `8000`
### `vieter cron`
@ -88,11 +89,11 @@ configuration variable required for each command.
* `api_key`: the API key to use when authenticating requests.
* `address`: Base URL of your Vieter instance, e.g. https://example.com
### `vieter repos`
### `vieter targets`
* `api_key`: the API key to use when authenticating requests.
* `address`: Base URL of your Vieter instance, e.g. https://example.com
* `base_image`: image to use when building a package using `vieter repos
* `base_image`: image to use when building a package using `vieter targets
build`.
* Default: `archlinux:base-devel`

6
docs/content/usage/builds/_index.md
View File

@ -16,11 +16,11 @@ info to the system. The Vieter repository server exposes an HTTP API for this
info). For ease of use, the Vieter binary contains a CLI interface for
interacting with this API (see [Configuration](/configuration) for
configuration details). The [man
pages](https://rustybever.be/man/vieter/vieter-repos.1.html) describe this in
pages](https://rustybever.be/man/vieter/vieter-targets.1.html) describe this in
greater detail, but the basic usage is as follows:
```
vieter repos add some-url some-branch some-repository
vieter targets add some-url some-branch some-repository
```
Here, `some-url` is the URL of the Git repository containing the PKGBUILD. This
@ -37,7 +37,7 @@ should be published.
The above command intentionally leaves out a few parameters to make the CLI
more useable. For information on how to modify all parameters using the CLI,
see
[vieter-repos-edit(1)](https://rustybever.be/man/vieter/vieter-repos-edit.1.html).
[vieter-targets-edit(1)](https://rustybever.be/man/vieter/vieter-targets-edit.1.html).
## Reading logs

6
src/build/build.v
View File

@ -6,7 +6,7 @@ import time
import os
import strings
import util
import models { GitRepo }
import models { Target }
const (
container_build_dir = '/build'
@ -91,9 +91,9 @@ pub:
}
// build_repo builds, packages & publishes a given Arch package based on the
// provided GitRepo. The base image ID should be of an image previously created
// provided target. The base image ID should be of an image previously created
// by create_build_image. It returns the logs of the container.
pub fn build_repo(address string, api_key string, base_image_id string, repo &GitRepo) ?BuildResult {
pub fn build_repo(address string, api_key string, base_image_id string, repo &Target) ?BuildResult {
mut dd := docker.new_conn()?
defer {

6
src/build/shell.v
View File

@ -1,6 +1,6 @@
module build
import models { GitRepo }
import models { Target }
// escape_shell_string escapes any characters that could be interpreted
// incorrectly by a shell. The resulting value should be safe to use inside an
@ -22,8 +22,8 @@ pub fn echo_commands(cmds []string) []string {
return out
}
// create_build_script generates a shell script that builds a given GitRepo.
fn create_build_script(address string, repo &GitRepo, build_arch string) string {
// create_build_script generates a shell script that builds a given Target.
fn create_build_script(address string, repo &Target, build_arch string) string {
repo_url := '$address/$repo.repo'
commands := echo_commands([

6
src/build/shell_test.v
View File

@ -1,15 +1,15 @@
module build
import models { GitRepo }
import models { Target }
fn test_create_build_script() {
repo := GitRepo{
target := Target{
id: 1
url: 'https://examplerepo.com'
branch: 'main'
repo: 'vieter'
}
build_script := create_build_script('https://example.com', repo, 'x86_64')
build_script := create_build_script('https://example.com', target, 'x86_64')
expected := $embed_file('build_script.sh')
assert build_script == expected.to_string().trim_space()

73
src/client/git.v
View File

@ -1,73 +0,0 @@
module client
import models { GitRepo, GitRepoFilter }
import net.http { Method }
import response { Response }
// get_git_repos returns a list of GitRepo's, given a filter object.
pub fn (c &Client) get_git_repos(filter GitRepoFilter) ?[]GitRepo {
params := models.params_from(filter)
data := c.send_request<[]GitRepo>(Method.get, '/api/repos', params)?
return data.data
}
// get_all_git_repos retrieves *all* GitRepo's from the API using the default
// limit.
pub fn (c &Client) get_all_git_repos() ?[]GitRepo {
mut repos := []GitRepo{}
mut offset := u64(0)
for {
sub_repos := c.get_git_repos(offset: offset)?
if sub_repos.len == 0 {
break
}
repos << sub_repos
offset += u64(sub_repos.len)
}
return repos
}
// get_git_repo returns the repo for a specific ID.
pub fn (c &Client) get_git_repo(id int) ?GitRepo {
data := c.send_request<GitRepo>(Method.get, '/api/repos/$id', {})?
return data.data
}
// add_git_repo adds a new repo to the server.
pub fn (c &Client) add_git_repo(url string, branch string, repo string, arch []string) ?Response<string> {
mut params := {
'url': url
'branch': branch
'repo': repo
}
if arch.len > 0 {
params['arch'] = arch.join(',')
}
data := c.send_request<string>(Method.post, '/api/repos', params)?
return data
}
// remove_git_repo removes the repo with the given ID from the server.
pub fn (c &Client) remove_git_repo(id int) ?Response<string> {
data := c.send_request<string>(Method.delete, '/api/repos/$id', {})?
return data
}
// patch_git_repo sends a PATCH request to the given repo with the params as
// payload.
pub fn (c &Client) patch_git_repo(id int, params map[string]string) ?Response<string> {
data := c.send_request<string>(Method.patch, '/api/repos/$id', params)?
return data
}

20
src/client/logs.v
View File

@ -8,47 +8,47 @@ import time
// get_build_logs returns all build logs.
pub fn (c &Client) get_build_logs(filter BuildLogFilter) ?Response<[]BuildLog> {
params := models.params_from(filter)
data := c.send_request<[]BuildLog>(Method.get, '/api/logs', params)?
data := c.send_request<[]BuildLog>(Method.get, '/api/v1/logs', params)?
return data
}
// get_build_logs_for_repo returns all build logs for a given repo.
pub fn (c &Client) get_build_logs_for_repo(repo_id int) ?Response<[]BuildLog> {
// get_build_logs_for_target returns all build logs for a given target.
pub fn (c &Client) get_build_logs_for_target(target_id int) ?Response<[]BuildLog> {
params := {
'repo': repo_id.str()
'repo': target_id.str()
}
data := c.send_request<[]BuildLog>(Method.get, '/api/logs', params)?
data := c.send_request<[]BuildLog>(Method.get, '/api/v1/logs', params)?
return data
}
// get_build_log returns a specific build log.
pub fn (c &Client) get_build_log(id int) ?Response<BuildLog> {
data := c.send_request<BuildLog>(Method.get, '/api/logs/$id', {})?
data := c.send_request<BuildLog>(Method.get, '/api/v1/logs/$id', {})?
return data
}
// get_build_log_content returns the contents of the build log file.
pub fn (c &Client) get_build_log_content(id int) ?string {
data := c.send_request_raw_response(Method.get, '/api/logs/$id/content', {}, '')?
data := c.send_request_raw_response(Method.get, '/api/v1/logs/$id/content', {}, '')?
return data
}
// add_build_log adds a new build log to the server.
pub fn (c &Client) add_build_log(repo_id int, start_time time.Time, end_time time.Time, arch string, exit_code int, content string) ?Response<string> {
pub fn (c &Client) add_build_log(target_id int, start_time time.Time, end_time time.Time, arch string, exit_code int, content string) ?Response<string> {
params := {
'repo': repo_id.str()
'target': target_id.str()
'startTime': start_time.unix_time().str()
'endTime': end_time.unix_time().str()
'arch': arch
'exitCode': exit_code.str()
}
data := c.send_request_with_body<string>(Method.post, '/api/logs', params, content)?
data := c.send_request_with_body<string>(Method.post, '/api/v1/logs', params, content)?
return data
}

73
src/client/targets.v 100644
View File

@ -0,0 +1,73 @@
module client
import models { Target, TargetFilter }
import net.http { Method }
import response { Response }
// get_targets returns a list of targets, given a filter object.
pub fn (c &Client) get_targets(filter TargetFilter) ?[]Target {
params := models.params_from(filter)
data := c.send_request<[]Target>(Method.get, '/api/v1/targets', params)?
return data.data
}
// get_all_targets retrieves *all* targs from the API using the default
// limit.
pub fn (c &Client) get_all_targets() ?[]Target {
mut targets := []Target{}
mut offset := u64(0)
for {
sub_targets := c.get_targets(offset: offset)?
if sub_targets.len == 0 {
break
}
targets << sub_targets
offset += u64(sub_targets.len)
}
return targets
}
// get_target returns the target for a specific id.
pub fn (c &Client) get_target(id int) ?Target {
data := c.send_request<Target>(Method.get, '/api/v1/targets/$id', {})?
return data.data
}
// add_target adds a new target to the server.
pub fn (c &Client) add_target(url string, branch string, repo string, arch []string) ?Response<string> {
mut params := {
'url': url
'branch': branch
'repo': repo
}
if arch.len > 0 {
params['arch'] = arch.join(',')
}
data := c.send_request<string>(Method.post, '/api/v1/targets', params)?
return data
}
// remove_target removes the target with the given id from the server.
pub fn (c &Client) remove_target(id int) ?Response<string> {
data := c.send_request<string>(Method.delete, '/api/v1/targets/$id', {})?
return data
}
// patch_target sends a PATCH request to the given target with the params as
// payload.
pub fn (c &Client) patch_target(id int, params map[string]string) ?Response<string> {
data := c.send_request<string>(Method.patch, '/api/v1/targets/$id', params)?
return data
}

22
src/console/logs/logs.v
View File

@ -12,7 +12,7 @@ struct Config {
api_key string [required]
}
// cmd returns the cli module that handles the build repos API.
// cmd returns the cli module that handles the build logs API.
pub fn cmd() cli.Command {
return cli.Command{
name: 'logs'
@ -33,8 +33,8 @@ pub fn cmd() cli.Command {
flag: cli.FlagType.int
},
cli.Flag{
name: 'repo'
description: 'Only return logs for this repo id.'
name: 'target'
description: 'Only return logs for this target id.'
flag: cli.FlagType.int
},
cli.Flag{
@ -79,9 +79,9 @@ pub fn cmd() cli.Command {
filter.offset = u64(offset)
}
repo_id := cmd.flags.get_int('repo')?
if repo_id != 0 {
filter.repo = repo_id
target_id := cmd.flags.get_int('target')?
if target_id != 0 {
filter.target = target_id
}
tz_offset := time.offset()
@ -168,10 +168,10 @@ pub fn cmd() cli.Command {
// print_log_list prints a list of logs.
fn print_log_list(logs []BuildLog) ? {
data := logs.map([it.id.str(), it.repo_id.str(), it.start_time.local().str(),
data := logs.map([it.id.str(), it.target_id.str(), it.start_time.local().str(),
it.exit_code.str()])
println(console.pretty_table(['id', 'repo', 'start time', 'exit code'], data)?)
println(console.pretty_table(['id', 'target', 'start time', 'exit code'], data)?)
}
// list prints a list of all build logs.
@ -182,10 +182,10 @@ fn list(conf Config, filter BuildLogFilter) ? {
print_log_list(logs)?
}
// list prints a list of all build logs for a given repo.
fn list_for_repo(conf Config, repo_id int) ? {
// list prints a list of all build logs for a given target.
fn list_for_target(conf Config, target_id int) ? {
c := client.new(conf.address, conf.api_key)
logs := c.get_build_logs_for_repo(repo_id)?.data
logs := c.get_build_logs_for_target(target_id)?.data
print_log_list(logs)?
}

6
src/console/git/build.v → src/console/targets/build.v
View File

@ -1,14 +1,14 @@
module git
module targets
import client
import docker
import os
import build
// build builds every Git repo in the server's list.
// build locally builds the target with the given id.
fn build(conf Config, repo_id int) ? {
c := client.new(conf.address, conf.api_key)
repo := c.get_git_repo(repo_id)?
repo := c.get_target(repo_id)?
build_arch := os.uname().machine

36
src/console/git/git.v → src/console/targets/targets.v
View File

@ -1,11 +1,11 @@
module git
module targets
import cli
import vieter.vconf
import cron.expression { parse_expression }
import client
import console
import models { GitRepoFilter }
import models { TargetFilter }
struct Config {
address string [required]
@ -16,12 +16,12 @@ struct Config {
// cmd returns the cli submodule that handles the repos API interaction
pub fn cmd() cli.Command {
return cli.Command{
name: 'repos'
description: 'Interact with the repos API.'
name: 'targets'
description: 'Interact with the targets API.'
commands: [
cli.Command{
name: 'list'
description: 'List the current repos.'
description: 'List the current targets.'
flags: [
cli.Flag{
name: 'limit'
@ -35,7 +35,7 @@ pub fn cmd() cli.Command {
},
cli.Flag{
name: 'repo'
description: 'Only return Git repos that publish to this repo.'
description: 'Only return targets that publish to this repo.'
flag: cli.FlagType.string
},
]
@ -43,7 +43,7 @@ pub fn cmd() cli.Command {
config_file := cmd.flags.get_string('config-file')?
conf := vconf.load<Config>(prefix: 'VIETER_', default_path: config_file)?
mut filter := GitRepoFilter{}
mut filter := TargetFilter{}
limit := cmd.flags.get_int('limit')?
if limit != 0 {
@ -67,7 +67,7 @@ pub fn cmd() cli.Command {
name: 'add'
required_args: 3
usage: 'url branch repo'
description: 'Add a new repository.'
description: 'Add a new Git repository target.'
execute: fn (cmd cli.Command) ? {
config_file := cmd.flags.get_string('config-file')?
conf := vconf.load<Config>(prefix: 'VIETER_', default_path: config_file)?
@ -79,7 +79,7 @@ pub fn cmd() cli.Command {
name: 'remove'
required_args: 1
usage: 'id'
description: 'Remove a repository that matches the given ID prefix.'
description: 'Remove a target that matches the given id.'
execute: fn (cmd cli.Command) ? {
config_file := cmd.flags.get_string('config-file')?
conf := vconf.load<Config>(prefix: 'VIETER_', default_path: config_file)?
@ -91,7 +91,7 @@ pub fn cmd() cli.Command {
name: 'info'
required_args: 1
usage: 'id'
description: 'Show detailed information for the repo matching the ID prefix.'
description: 'Show detailed information for the target matching the id.'
execute: fn (cmd cli.Command) ? {
config_file := cmd.flags.get_string('config-file')?
conf := vconf.load<Config>(prefix: 'VIETER_', default_path: config_file)?
@ -103,7 +103,7 @@ pub fn cmd() cli.Command {
name: 'edit'
required_args: 1
usage: 'id'
description: 'Edit the repository that matches the given ID prefix.'
description: 'Edit the Git repository target that matches the given id.'
flags: [
cli.Flag{
name: 'url'
@ -152,7 +152,7 @@ pub fn cmd() cli.Command {
name: 'build'
required_args: 1
usage: 'id'
description: 'Build the repo with the given id & publish it.'
description: 'Build the target with the given id & publish it.'
execute: fn (cmd cli.Command) ? {
config_file := cmd.flags.get_string('config-file')?
conf := vconf.load<Config>(prefix: 'VIETER_', default_path: config_file)?
@ -168,9 +168,9 @@ pub fn cmd() cli.Command {
// ID. If multiple or none are found, an error is raised.
// list prints out a list of all repositories.
fn list(conf Config, filter GitRepoFilter) ? {
fn list(conf Config, filter TargetFilter) ? {
c := client.new(conf.address, conf.api_key)
repos := c.get_git_repos(filter)?
repos := c.get_targets(filter)?
data := repos.map([it.id.str(), it.url, it.branch, it.repo])
println(console.pretty_table(['id', 'url', 'branch', 'repo'], data)?)
@ -179,7 +179,7 @@ fn list(conf Config, filter GitRepoFilter) ? {
// add adds a new repository to the server's list.
fn add(conf Config, url string, branch string, repo string) ? {
c := client.new(conf.address, conf.api_key)
res := c.add_git_repo(url, branch, repo, [])?
res := c.add_target(url, branch, repo, [])?
println(res.message)
}
@ -191,7 +191,7 @@ fn remove(conf Config, id string) ? {
if id_int != 0 {
c := client.new(conf.address, conf.api_key)
res := c.remove_git_repo(id_int)?
res := c.remove_target(id_int)?
println(res.message)
}
}
@ -209,7 +209,7 @@ fn patch(conf Config, id string, params map[string]string) ? {
id_int := id.int()
if id_int != 0 {
c := client.new(conf.address, conf.api_key)
res := c.patch_git_repo(id_int, params)?
res := c.patch_target(id_int, params)?
println(res.message)
}
@ -224,6 +224,6 @@ fn info(conf Config, id string) ? {
}
c := client.new(conf.address, conf.api_key)
repo := c.get_git_repo(id_int)?
repo := c.get_target(id_int)?
println(repo)
}

10
src/cron/daemon/daemon.v
View File

@ -9,7 +9,7 @@ import build
import docker
import os
import client
import models { GitRepo }
import models { Target }
const (
// How many seconds to wait before retrying to update API if failed
@ -20,7 +20,7 @@ const (
struct ScheduledBuild {
pub:
repo GitRepo
repo Target
timestamp time.Time
}
@ -38,7 +38,7 @@ mut:
api_update_frequency int
image_rebuild_frequency int
// Repos currently loaded from API.
repos []GitRepo
repos []Target
// At what point to update the list of repositories.
api_update_timestamp time.Time
image_build_timestamp time.Time
@ -149,7 +149,7 @@ pub fn (mut d Daemon) run() {
}
// schedule_build adds the next occurence of the given repo build to the queue.
fn (mut d Daemon) schedule_build(repo GitRepo) {
fn (mut d Daemon) schedule_build(repo Target) {
ce := if repo.schedule != '' {
parse_expression(repo.schedule) or {
// TODO This shouldn't return an error if the expression is empty.
@ -178,7 +178,7 @@ fn (mut d Daemon) schedule_build(repo GitRepo) {
fn (mut d Daemon) renew_repos() {
d.linfo('Renewing repos...')
mut new_repos := d.client.get_all_git_repos() or {
mut new_repos := d.client.get_all_targets() or {
d.lerror('Failed to renew repos. Retrying in ${daemon.api_update_retry_timeout}s...')
d.api_update_timestamp = time.now().add_seconds(daemon.api_update_retry_timeout)

8
src/db/db.v
View File

@ -13,8 +13,12 @@ struct MigrationVersion {
}
const (
migrations_up = [$embed_file('migrations/001-initial/up.sql')]
migrations_down = [$embed_file('migrations/001-initial/down.sql')]
migrations_up = [
$embed_file('migrations/001-initial/up.sql'),
$embed_file('migrations/002-rename-to-targets/up.sql'),
]
migrations_down = [$embed_file('migrations/001-initial/down.sql'),
$embed_file('migrations/002-rename-to-targets/down.sql')]
)
// init initializes a database & adds the correct tables.

99
src/db/git.v
View File

@ -1,99 +0,0 @@
module db
import models { GitRepo, GitRepoArch, GitRepoFilter }
// get_git_repos returns all GitRepo's in the database.
pub fn (db &VieterDb) get_git_repos(filter GitRepoFilter) []GitRepo {
// This seems to currently be blocked by a bug in the ORM, I'll have to ask
// around.
if filter.repo != '' {
res := sql db.conn {
select from GitRepo where repo == filter.repo order by id limit filter.limit offset filter.offset
}
return res
}
res := sql db.conn {
select from GitRepo order by id limit filter.limit offset filter.offset
}
return res
}
// get_git_repo tries to return a specific GitRepo.
pub fn (db &VieterDb) get_git_repo(repo_id int) ?GitRepo {
res := sql db.conn {
select from GitRepo where id == repo_id
}
// If a select statement fails, it returns a zeroed object. By
// checking one of the required fields, we can see whether the query
// returned a result or not.
if res.id == 0 {
return none
}
return res
}
// add_git_repo inserts the given GitRepo into the database.
pub fn (db &VieterDb) add_git_repo(repo GitRepo) {
sql db.conn {
insert repo into GitRepo
}
}
// delete_git_repo deletes the repo with the given ID from the database.
pub fn (db &VieterDb) delete_git_repo(repo_id int) {
sql db.conn {
delete from GitRepo where id == repo_id
delete from GitRepoArch where repo_id == repo_id
}
}
// update_git_repo updates any non-array values for a given GitRepo.
pub fn (db &VieterDb) update_git_repo(repo_id int, params map[string]string) {
mut values := []string{}
// TODO does this allow for SQL injection?
$for field in GitRepo.fields {
if field.name in params {
// Any fields that are array types require their own update method
$if field.typ is string {
values << "$field.name = '${params[field.name]}'"
}
}
}
values_str := values.join(', ')
// I think this is actual SQL & not the ORM language
query := 'update GitRepo set $values_str where id == $repo_id'
db.conn.exec_none(query)
}
// update_git_repo_archs updates a given GitRepo's arch value.
pub fn (db &VieterDb) update_git_repo_archs(repo_id int, archs []GitRepoArch) {
archs_with_id := archs.map(GitRepoArch{
...it
repo_id: repo_id
})
sql db.conn {
delete from GitRepoArch where repo_id == repo_id
}
for arch in archs_with_id {
sql db.conn {
insert arch into GitRepoArch
}
}
}
// git_repo_exists is a utility function that checks whether a repo with the
// given id exists.
pub fn (db &VieterDb) git_repo_exists(repo_id int) bool {
db.get_git_repo(repo_id) or { return false }
return true
}

12
src/db/logs.v
View File

@ -7,8 +7,8 @@ import time
pub fn (db &VieterDb) get_build_logs(filter BuildLogFilter) []BuildLog {
mut where_parts := []string{}
if filter.repo != 0 {
where_parts << 'repo_id == $filter.repo'
if filter.target != 0 {
where_parts << 'target_id == $filter.target'
}
if filter.before != time.Time{} {
@ -55,11 +55,11 @@ pub fn (db &VieterDb) get_build_logs(filter BuildLogFilter) []BuildLog {
return res
}
// get_build_logs_for_repo returns all BuildLog's in the database for a given
// repo.
pub fn (db &VieterDb) get_build_logs_for_repo(repo_id int) []BuildLog {
// get_build_logs_for_target returns all BuildLog's in the database for a given
// target.
pub fn (db &VieterDb) get_build_logs_for_target(target_id int) []BuildLog {
res := sql db.conn {
select from BuildLog where repo_id == repo_id order by id
select from BuildLog where target_id == target_id order by id
}
return res

5
src/db/migrations/002-rename-to-targets/down.sql 100644
View File

@ -0,0 +1,5 @@
ALTER TABLE Target RENAME TO GitRepo;
ALTER TABLE TargetArch RENAME TO GitRepoArch;
ALTER TABLE GitRepoArch RENAME COLUMN target_id TO repo_id;
ALTER TABLE BuildLog RENAME COLUMN target_id TO repo_id;

5
src/db/migrations/002-rename-to-targets/up.sql 100644
View File

@ -0,0 +1,5 @@
ALTER TABLE GitRepo RENAME TO Target;
ALTER TABLE GitRepoArch RENAME TO TargetArch;
ALTER TABLE TargetArch RENAME COLUMN repo_id TO target_id;
ALTER TABLE BuildLog RENAME COLUMN repo_id TO target_id;

99
src/db/targets.v 100644
View File

@ -0,0 +1,99 @@
module db
import models { Target, TargetArch, TargetFilter }
// get_targets returns all targets in the database.
pub fn (db &VieterDb) get_targets(filter TargetFilter) []Target {
// This seems to currently be blocked by a bug in the ORM, I'll have to ask
// around.
if filter.repo != '' {
res := sql db.conn {
select from Target where repo == filter.repo order by id limit filter.limit offset filter.offset
}
return res
}
res := sql db.conn {
select from Target order by id limit filter.limit offset filter.offset
}
return res
}
// get_target tries to return a specific target.
pub fn (db &VieterDb) get_target(target_id int) ?Target {
res := sql db.conn {
select from Target where id == target_id
}
// If a select statement fails, it returns a zeroed object. By
// checking one of the required fields, we can see whether the query
// returned a result or not.
if res.id == 0 {
return none
}
return res
}
// add_target inserts the given target into the database.
pub fn (db &VieterDb) add_target(repo Target) {
sql db.conn {
insert repo into Target
}
}
// delete_target deletes the target with the given id from the database.
pub fn (db &VieterDb) delete_target(target_id int) {
sql db.conn {
delete from Target where id == target_id
delete from TargetArch where target_id == target_id
}
}
// update_target updates any non-array values for a given target.
pub fn (db &VieterDb) update_target(target_id int, params map[string]string) {
mut values := []string{}
// TODO does this allow for SQL injection?
$for field in Target.fields {
if field.name in params {
// Any fields that are array types require their own update method
$if field.typ is string {
values << "$field.name = '${params[field.name]}'"
}
}
}
values_str := values.join(', ')
// I think this is actual SQL & not the ORM language
query := 'update Target set $values_str where id == $target_id'
db.conn.exec_none(query)
}
// update_target_archs updates a given target's arch value.
pub fn (db &VieterDb) update_target_archs(target_id int, archs []TargetArch) {
archs_with_id := archs.map(TargetArch{
...it
target_id: target_id
})
sql db.conn {
delete from TargetArch where target_id == target_id
}
for arch in archs_with_id {
sql db.conn {
insert arch into TargetArch
}
}
}
// target_exists is a utility function that checks whether a target with the
// given id exists.
pub fn (db &VieterDb) target_exists(target_id int) bool {
db.get_target(target_id) or { return false }
return true
}

4
src/main.v
View File

@ -3,7 +3,7 @@ module main
import os
import server
import cli
import console.git
import console.targets
import console.logs
import console.schedule
import console.man
@ -26,7 +26,7 @@ fn main() {
]
commands: [
server.cmd(),
git.cmd(),
targets.cmd(),
cron.cmd(),
logs.cmd(),
schedule.cmd(),

6
src/models/logs.v
View File

@ -5,7 +5,7 @@ import time
pub struct BuildLog {
pub mut:
id int [primary; sql: serial]
repo_id int [nonull]
target_id int [nonull]
start_time time.Time [nonull]
end_time time.Time [nonull]
arch string [nonull]
@ -16,7 +16,7 @@ pub mut:
pub fn (bl &BuildLog) str() string {
mut parts := [
'id: $bl.id',
'repo id: $bl.repo_id',
'target id: $bl.target_id',
'start time: $bl.start_time.local()',
'end time: $bl.end_time.local()',
'duration: ${bl.end_time - bl.start_time}',
@ -33,7 +33,7 @@ pub struct BuildLogFilter {
pub mut:
limit u64 = 25
offset u64
repo int
target int
before time.Time
after time.Time
arch string

4
src/models/models.v
View File

@ -23,8 +23,8 @@ pub fn patch_from_params<T>(mut o T, params map[string]string) ? {
o.$(field.name) = params[field.name].int()
} $else $if field.typ is u64 {
o.$(field.name) = params[field.name].u64()
} $else $if field.typ is []GitRepoArch {
o.$(field.name) = params[field.name].split(',').map(GitRepoArch{ value: it })
} $else $if field.typ is []TargetArch {
o.$(field.name) = params[field.name].split(',').map(TargetArch{ value: it })
} $else $if field.typ is time.Time {
o.$(field.name) = time.unix(params[field.name].int())
} $else $if field.typ is []string {

18
src/models/git.v → src/models/targets.v
View File

@ -1,18 +1,18 @@
module models
pub struct GitRepoArch {
pub struct TargetArch {
pub:
id int [primary; sql: serial]
repo_id int [nonull]
value string [nonull]
id int [primary; sql: serial]
target_id int [nonull]
value string [nonull]
}
// str returns a string representation.
pub fn (gra &GitRepoArch) str() string {
pub fn (gra &TargetArch) str() string {
return gra.value
}
pub struct GitRepo {
pub struct Target {
pub mut:
id int [primary; sql: serial]
// URL of the Git repository
@ -25,11 +25,11 @@ pub mut:
schedule string
// On which architectures the package is allowed to be built. In reality,
// this controls which builders will periodically build the image.
arch []GitRepoArch [fkey: 'repo_id']
arch []TargetArch [fkey: 'target_id']
}
// str returns a string representation.
pub fn (gr &GitRepo) str() string {
pub fn (gr &Target) str() string {
mut parts := [
'id: $gr.id',
'url: $gr.url',
@ -44,7 +44,7 @@ pub fn (gr &GitRepo) str() string {
}
[params]
pub struct GitRepoFilter {
pub struct TargetFilter {
pub mut:
limit u64 = 25
offset u64

6
src/server/README.md 100644
View File

@ -0,0 +1,6 @@
This module contains the Vieter HTTP server, consisting of the repository
implementation & the REST API.
**NOTE**: vweb defines the priority order of routes by the file names in this
module. Therefore, it's very important that all API routes are defined in files
prefixed with `api_`, as this is before the word `routes` alphabetically.

36
src/server/logs.v → src/server/api_logs.v
View File

@ -10,10 +10,10 @@ import os
import util
import models { BuildLog, BuildLogFilter }
// get_logs returns all build logs in the database. A 'repo' query param can
// v1_get_logs returns all build logs in the database. A 'target' query param can
// optionally be added to limit the list of build logs to that repository.
['/api/logs'; get]
fn (mut app App) get_logs() web.Result {
['/api/v1/logs'; get]
fn (mut app App) v1_get_logs() web.Result {
if !app.is_authorized() {
return app.json(http.Status.unauthorized, new_response('Unauthorized.'))
}
@ -26,9 +26,9 @@ fn (mut app App) get_logs() web.Result {
return app.json(http.Status.ok, new_data_response(logs))
}
// get_single_log returns the build log with the given id.
['/api/logs/:id'; get]
fn (mut app App) get_single_log(id int) web.Result {
// v1_get_single_log returns the build log with the given id.
['/api/v1/logs/:id'; get]
fn (mut app App) v1_get_single_log(id int) web.Result {
if !app.is_authorized() {
return app.json(http.Status.unauthorized, new_response('Unauthorized.'))
}
@ -38,16 +38,16 @@ fn (mut app App) get_single_log(id int) web.Result {
return app.json(http.Status.ok, new_data_response(log))
}
// get_log_content returns the actual build log file for the given id.
['/api/logs/:id/content'; get]
fn (mut app App) get_log_content(id int) web.Result {
// v1_get_log_content returns the actual build log file for the given id.
['/api/v1/logs/:id/content'; get]
fn (mut app App) v1_get_log_content(id int) web.Result {
if !app.is_authorized() {
return app.json(http.Status.unauthorized, new_response('Unauthorized.'))
}
log := app.db.get_build_log(id) or { return app.not_found() }
file_name := log.start_time.custom_format('YYYY-MM-DD_HH-mm-ss')
full_path := os.join_path(app.conf.data_dir, logs_dir_name, log.repo_id.str(), log.arch,
full_path := os.join_path(app.conf.data_dir, logs_dir_name, log.target_id.str(), log.arch,
file_name)
return app.file(full_path)
@ -62,9 +62,9 @@ fn parse_query_time(query string) ?time.Time {
return t
}
// post_log adds a new log to the database.
['/api/logs'; post]
fn (mut app App) post_log() web.Result {
// v1_post_log adds a new log to the database.
['/api/v1/logs'; post]
fn (mut app App) v1_post_log() web.Result {
if !app.is_authorized() {
return app.json(http.Status.unauthorized, new_response('Unauthorized.'))
}
@ -96,15 +96,15 @@ fn (mut app App) post_log() web.Result {
arch := app.query['arch']
repo_id := app.query['repo'].int()
target_id := app.query['target'].int()
if !app.db.git_repo_exists(repo_id) {
return app.json(http.Status.bad_request, new_response('Unknown Git repo.'))
if !app.db.target_exists(target_id) {
return app.json(http.Status.bad_request, new_response('Unknown target.'))
}
// Store log in db
log := BuildLog{
repo_id: repo_id
target_id: target_id
start_time: start_time
end_time: end_time
arch: arch
@ -113,7 +113,7 @@ fn (mut app App) post_log() web.Result {
app.db.add_build_log(log)
repo_logs_dir := os.join_path(app.conf.data_dir, logs_dir_name, repo_id.str(), arch)
repo_logs_dir := os.join_path(app.conf.data_dir, logs_dir_name, target_id.str(), arch)
// Create the logs directory of it doesn't exist
if !os.exists(repo_logs_dir) {

50
src/server/git.v → src/server/api_targets.v
View File

@ -4,38 +4,38 @@ import web
import net.http
import response { new_data_response, new_response }
import db
import models { GitRepo, GitRepoArch, GitRepoFilter }
import models { Target, TargetArch, TargetFilter }
// get_repos returns the current list of repos.
['/api/repos'; get]
fn (mut app App) get_repos() web.Result {
// v1_get_targets returns the current list of targets.
['/api/v1/targets'; get]
fn (mut app App) v1_get_targets() web.Result {
if !app.is_authorized() {
return app.json(http.Status.unauthorized, new_response('Unauthorized.'))
}
filter := models.from_params<GitRepoFilter>(app.query) or {
filter := models.from_params<TargetFilter>(app.query) or {
return app.json(http.Status.bad_request, new_response('Invalid query parameters.'))
}
repos := app.db.get_git_repos(filter)
repos := app.db.get_targets(filter)
return app.json(http.Status.ok, new_data_response(repos))
}
// get_single_repo returns the information for a single repo.
['/api/repos/:id'; get]
fn (mut app App) get_single_repo(id int) web.Result {
// v1_get_single_target returns the information for a single target.
['/api/v1/targets/:id'; get]
fn (mut app App) v1_get_single_target(id int) web.Result {
if !app.is_authorized() {
return app.json(http.Status.unauthorized, new_response('Unauthorized.'))
}
repo := app.db.get_git_repo(id) or { return app.not_found() }
repo := app.db.get_target(id) or { return app.not_found() }
return app.json(http.Status.ok, new_data_response(repo))
}
// post_repo creates a new repo from the provided query string.
['/api/repos'; post]
fn (mut app App) post_repo() web.Result {
// v1_post_target creates a new target from the provided query string.
['/api/v1/targets'; post]
fn (mut app App) v1_post_target() web.Result {
if !app.is_authorized() {
return app.json(http.Status.unauthorized, new_response('Unauthorized.'))
}
@ -48,40 +48,40 @@ fn (mut app App) post_repo() web.Result {
params['arch'] = app.conf.default_arch
}
new_repo := models.from_params<GitRepo>(params) or {
new_repo := models.from_params<Target>(params) or {
return app.json(http.Status.bad_request, new_response(err.msg()))
}
app.db.add_git_repo(new_repo)
app.db.add_target(new_repo)
return app.json(http.Status.ok, new_response('Repo added successfully.'))
}
// delete_repo removes a given repo from the server's list.
['/api/repos/:id'; delete]
fn (mut app App) delete_repo(id int) web.Result {
// v1_delete_target removes a given target from the server's list.
['/api/v1/targets/:id'; delete]
fn (mut app App) v1_delete_target(id int) web.Result {
if !app.is_authorized() {
return app.json(http.Status.unauthorized, new_response('Unauthorized.'))
}
app.db.delete_git_repo(id)
app.db.delete_target(id)
return app.json(http.Status.ok, new_response('Repo removed successfully.'))
}
// patch_repo updates a repo's data with the given query params.
['/api/repos/:id'; patch]
fn (mut app App) patch_repo(id int) web.Result {
// v1_patch_target updates a target's data with the given query params.
['/api/v1/targets/:id'; patch]
fn (mut app App) v1_patch_target(id int) web.Result {
if !app.is_authorized() {
return app.json(http.Status.unauthorized, new_response('Unauthorized.'))
}
app.db.update_git_repo(id, app.query)
app.db.update_target(id, app.query)
if 'arch' in app.query {
arch_objs := app.query['arch'].split(',').map(GitRepoArch{ value: it })
arch_objs := app.query['arch'].split(',').map(TargetArch{ value: it })
app.db.update_git_repo_archs(id, arch_objs)
app.db.update_target_archs(id, arch_objs)
}
return app.json(http.Status.ok, new_response('Repo updated successfully.'))

1
src/server/cli.v
View File

@ -10,6 +10,7 @@ pub:
data_dir string
api_key string
default_arch string
port int = 8000
}
// cmd returns the cli submodule that handles starting the server

3
src/server/server.v
View File

@ -8,7 +8,6 @@ import util
import db
const (
port = 8000
log_file_name = 'vieter.log'
repo_dir_name = 'repos'
db_file_name = 'vieter.sqlite'
@ -77,5 +76,5 @@ pub fn server(conf Config) ? {
conf: conf
repo: repo
db: db
}, server.port)
}, conf.port)
}