GreekStapler
/
vieter
forked from vieter-v/vieter
1
0
Fork 0
Code Issues Pull Requests Releases Activity

Merge pull request 'Some breaking API changes' (#245) from Chewing_Bever/vieter:api-changes into dev 

Reviewed-on: vieter/vieter#245
main
Jef Roosens 2022-06-16 18:17:53 +02:00
parent 3a5ac5d32b 3d38df6d03
commit f81039d2bb
31 changed files with 366 additions and 336 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
CHANGELOG.md
View File

@ -7,6 +7,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased](https://git.rustybever.be/vieter/vieter/src/branch/dev)
### Added
* Server port can now be configured
### Changed
* Moved all API routes under `/v1` namespace
* Renamed `vieter repos` to `vieter targets`
* Renamed `/api/v1/repos` namespace to `/api/v1/targets`
## [0.3.0](https://git.rustybever.be/vieter/vieter/src/tag/0.3.0)
Nothing besides bumping the versions.

10
docs/api/source/includes/_logs.md
View File

@ -24,7 +24,7 @@ curl \
"data": [
{
"id": 1,
"repo_id": 3,
"target_id": 3,
"start_time": 1652008554,
"end_time": 1652008559,
"arch": "x86_64",
@ -46,7 +46,7 @@ Parameter | Description
--------- | -----------
limit | Maximum amount of results to return.
offset | Offset of results.
repo | Only return builds published to this repository.
target | Only return builds for this target id.
before | Only return logs started before this time (UTC epoch)
after | Only return logs started after this time (UTC epoch)
arch | Only return logs built on this architecture
@ -58,7 +58,7 @@ exit_codes | Comma-separated list of exit codes to limit result to; using `!` as
```shell
curl \
-H 'X-Api-Key: secret' \
https://example.com/api/logs/15
https://example.com/api/logs/1
```
> JSON output format
@ -68,7 +68,7 @@ curl \
"message": "",
"data": {
"id": 1,
"repo_id": 3,
"target_id": 3,
"start_time": 1652008554,
"end_time": 1652008559,
"arch": "x86_64",
@ -129,11 +129,11 @@ Publish a new build log to the server.
Parameter | Description
--------- | -----------
id | ID of requested log
startTime | Start time of the build (UTC epoch)
endTime | End time of the build (UTC epoch)
arch | Architecture on which the build was done
exitCode | Exit code of the build container
target | id of target this build is for
### Request body

51
docs/api/source/includes/_git.md → docs/api/source/includes/_targets.md
View File

@ -1,4 +1,4 @@
# Git Repositories
# Targets
<aside class="notice">
@ -6,15 +6,14 @@ All routes in this section require authentication.
</aside>
Endpoints for interacting with the list of Git repositories stored on the
server.
Endpoints for interacting with the list of targets stored on the server.
## List repos
## List targets
```shell
curl \
-H 'X-Api-Key: secret' \
https://example.com/api/repos?offset=10&limit=20
https://example.com/api/targets?offset=10&limit=20
```
> JSON output format
@ -32,7 +31,7 @@ curl \
"arch": [
{
"id": 1,
"repo_id": 1,
"target_id": 1,
"value": "x86_64"
}
]
@ -41,11 +40,11 @@ curl \
}
```
Retrieve a list of Git repositories.
Retrieve a list of targets.
### HTTP Request
`GET /api/repos`
`GET /api/targets`
### Query Parameters
@ -53,14 +52,14 @@ Parameter | Description
--------- | -----------
limit | Maximum amount of results to return.
offset | Offset of results.
repo | Limit results to repositories that publish to the given repo.
repo | Limit results to targets that publish to the given repo.
## Get a repo
## Get specific target
```shell
curl \
-H 'X-Api-Key: secret' \
https://example.com/api/repos/15
https://example.com/api/targets/1
```
> JSON output format
@ -77,7 +76,7 @@ curl \
"arch": [
{
"id": 1,
"repo_id": 1,
"target_id": 1,
"value": "x86_64"
}
]
@ -85,25 +84,25 @@ curl \
}
```
Get info about a specific Git repository.
Get info about a specific target.
### HTTP Request
`GET /api/repos/:id`
`GET /api/targets/:id`
### URL Parameters
Parameter | Description
--------- | -----------
id | ID of requested repo
id | id of requested target
## Create a new repo
## Create a new target
Create a new Git repository with the given data.
Create a new target with the given data.
### HTTP Request
`POST /api/repos`
`POST /api/targets`
### Query Parameters
@ -115,19 +114,19 @@ repo | Vieter repository to publish built packages to.
schedule | Cron build schedule (syntax explained [here](https://rustybever.be/docs/vieter/usage/builds/schedule/))
arch | Comma-separated list of architectures to build package on.
## Modify a repo
## Modify a target
Modify the data of an existing Git repository.
Modify the data of an existing target.
### HTTP Request
`PATCH /api/repos/:id`
`PATCH /api/targets/:id`
### URL Parameters
Parameter | Description
--------- | -----------
id | ID of requested repo
id | id of target to modify
### Query Parameters
@ -139,16 +138,16 @@ repo | Vieter repository to publish built packages to.
schedule | Cron build schedule
arch | Comma-separated list of architectures to build package on.
## Remove a repo
## Remove a target
Remove a Git repository from the server.
Remove a target from the server.
### HTTP Request
`DELETE /api/repos/:id`
`DELETE /api/targets/:id`
### URL Parameters
Parameter | Description
--------- | -----------
id | ID of repo to remove
id | id of target to remove

2
docs/api/source/index.html.md
View File

@ -9,7 +9,7 @@ toc_footers:
includes:
- repository
- git
- targets
- logs
search: true

9
docs/content/configuration.md
View File

@ -26,7 +26,7 @@ secrets file.
## Commands
The first argument passed to Vieter determines which command you wish to use.
Each of these can contain subcommands (e.g. `vieter repos list`), but all
Each of these can contain subcommands (e.g. `vieter targets list`), but all
subcommands will use the same configuration. Below you can find the
configuration variable required for each command.
@ -45,7 +45,8 @@ configuration variable required for each command.
This prevents the server from being confused when an `any` package is
published as the very first package for a repository.
* Git repositories added without an `arch` value use this value instead.
* `port`: HTTP port to run on
* Default: `8000`
### `vieter cron`
@ -88,11 +89,11 @@ configuration variable required for each command.
* `api_key`: the API key to use when authenticating requests.
* `address`: Base URL of your Vieter instance, e.g. https://example.com
### `vieter repos`
### `vieter targets`
* `api_key`: the API key to use when authenticating requests.
* `address`: Base URL of your Vieter instance, e.g. https://example.com
* `base_image`: image to use when building a package using `vieter repos
* `base_image`: image to use when building a package using `vieter targets
build`.
* Default: `archlinux:base-devel`

6
docs/content/usage/builds/_index.md
View File

@ -16,11 +16,11 @@ info to the system. The Vieter repository server exposes an HTTP API for this
info). For ease of use, the Vieter binary contains a CLI interface for
interacting with this API (see [Configuration](/configuration) for
configuration details). The [man
pages](https://rustybever.be/man/vieter/vieter-repos.1.html) describe this in
pages](https://rustybever.be/man/vieter/vieter-targets.1.html) describe this in
greater detail, but the basic usage is as follows:
```
vieter repos add some-url some-branch some-repository
vieter targets add some-url some-branch some-repository
```
Here, `some-url` is the URL of the Git repository containing the PKGBUILD. This
@ -37,7 +37,7 @@ should be published.
The above command intentionally leaves out a few parameters to make the CLI
more useable. For information on how to modify all parameters using the CLI,
see
[vieter-repos-edit(1)](https://rustybever.be/man/vieter/vieter-repos-edit.1.html).
[vieter-targets-edit(1)](https://rustybever.be/man/vieter/vieter-targets-edit.1.html).
## Reading logs

6
src/build/build.v
View File

@ -6,7 +6,7 @@ import time
import os
import strings
import util
import models { GitRepo }
import models { Target }
const (
container_build_dir = '/build'
@ -91,9 +91,9 @@ pub:
}
// build_repo builds, packages & publishes a given Arch package based on the
// provided GitRepo. The base image ID should be of an image previously created
// provided target. The base image ID should be of an image previously created
// by create_build_image. It returns the logs of the container.
pub fn build_repo(address string, api_key string, base_image_id string, repo &GitRepo) ?BuildResult {
pub fn build_repo(address string, api_key string, base_image_id string, repo &Target) ?BuildResult {
mut dd := docker.new_conn()?
defer {

6
src/build/shell.v
View File

@ -1,6 +1,6 @@
module build
import models { GitRepo }
import models { Target }
// escape_shell_string escapes any characters that could be interpreted
// incorrectly by a shell. The resulting value should be safe to use inside an
@ -22,8 +22,8 @@ pub fn echo_commands(cmds []string) []string {
return out
}
// create_build_script generates a shell script that builds a given GitRepo.
fn create_build_script(address string, repo &GitRepo, build_arch string) string {
// create_build_script generates a shell script that builds a given Target.
fn create_build_script(address string, repo &Target, build_arch string) string {
repo_url := '$address/$repo.repo'
commands := echo_commands([

6
src/build/shell_test.v
View File

@ -1,15 +1,15 @@
module build
import models { GitRepo }
import models { Target }
fn test_create_build_script() {
repo := GitRepo{
target := Target{
id: 1
url: 'https://examplerepo.com'
branch: 'main'
repo: 'vieter'
}
build_script := create_build_script('https://example.com', repo, 'x86_64')
build_script := create_build_script('https://example.com', target, 'x86_64')
expected := $embed_file('build_script.sh')
assert build_script == expected.to_string().trim_space()

73
src/client/git.v
View File

@ -1,73 +0,0 @@
module client
import models { GitRepo, GitRepoFilter }
import net.http { Method }
import response { Response }
// get_git_repos returns a list of GitRepo's, given a filter object.
pub fn (c &Client) get_git_repos(filter GitRepoFilter) ?[]GitRepo {
params := models.params_from(filter)
data := c.send_request<[]GitRepo>(Method.get, '/api/repos', params)?
return data.data
}
// get_all_git_repos retrieves *all* GitRepo's from the API using the default
// limit.
pub fn (c &Client) get_all_git_repos() ?[]GitRepo {
mut repos := []GitRepo{}
mut offset := u64(0)
for {
sub_repos := c.get_git_repos(offset: offset)?
if sub_repos.len == 0 {
break
}
repos << sub_repos
offset += u64(sub_repos.len)
}
return repos
}
// get_git_repo returns the repo for a specific ID.
pub fn (c &Client) get_git_repo(id int) ?GitRepo {
data := c.send_request<GitRepo>(Method.get, '/api/repos/$id', {})?
return data.data
}
// add_git_repo adds a new repo to the server.
pub fn (c &Client) add_git_repo(url string, branch string, repo string, arch []string) ?Response<string> {
mut params := {
'url': url
'branch': branch
'repo': repo
}
if arch.len > 0 {
params['arch'] = arch.join(',')
}
data := c.send_request<string>(Method.post, '/api/repos', params)?
return data
}
// remove_git_repo removes the repo with the given ID from the server.
pub fn (c &Client) remove_git_repo(id int) ?Response<string> {
data := c.send_request<string>(Method.delete, '/api/repos/$id', {})?
return data
}
// patch_git_repo sends a PATCH request to the given repo with the params as
// payload.
pub fn (c &Client) patch_git_repo(id int, params map[string]string) ?Response<string> {
data := c.send_request<string>(Method.patch, '/api/repos/$id', params)?
return data
}

20
src/client/logs.v
View File

@ -8,47 +8,47 @@ import time
// get_build_logs returns all build logs.
pub fn (c &Client) get_build_logs(filter BuildLogFilter) ?Response<[]BuildLog> {
params := models.params_from(filter)
data := c.send_request<[]BuildLog>(Method.get, '/api/logs', params)?
data := c.send_request<[]BuildLog>(Method.get, '/api/v1/logs', params)?
return data
}
// get_build_logs_for_repo returns all build logs for a given repo.
pub fn (c &Client) get_build_logs_for_repo(repo_id int) ?Response<[]BuildLog> {
// get_build_logs_for_target returns all build logs for a given target.
pub fn (c &Client) get_build_logs_for_target(target_id int) ?Response<[]BuildLog> {
params := {
'repo': repo_id.str()
'repo': target_id.str()
}
data := c.send_request<[]BuildLog>(Method.get, '/api/logs', params)?
data := c.send_request<[]BuildLog>(Method.get, '/api/v1/logs', params)?
return data
}
// get_build_log returns a specific build log.
pub fn (c &Client) get_build_log(id int) ?Response<BuildLog> {
data := c.send_request<BuildLog>(Method.get, '/api/logs/$id', {})?
data := c.send_request<BuildLog>(Method.get, '/api/v1/logs/$id', {})?
return data
}
// get_build_log_content returns the contents of the build log file.
pub fn (c &Client) get_build_log_content(id int) ?string {
data := c.send_request_raw_response(Method.get, '/api/logs/$id/content', {}, '')?
data := c.send_request_raw_response(Method.get, '/api/v1/logs/$id/content', {}, '')?
return data
}
// add_build_log adds a new build log to the server.
pub fn (c &Client) add_build_log(repo_id int, start_time time.Time, end_time time.Time, arch string, exit_code int, content string) ?Response<string> {
pub fn (c &Client) add_build_log(target_id int, start_time time.Time, end_time time.Time, arch string, exit_code int, content string) ?Response<string> {
params := {
'repo': repo_id.str()
'target': target_id.str()
'startTime': start_time.unix_time().str()
'endTime': end_time.unix_time().str()
'arch': arch
'exitCode': exit_code.str()
}
data := c.send_request_with_body<string>(Method.post, '/api/logs', params, content)?
data := c.send_request_with_body<string>(Method.post, '/api/v1/logs', params, content)?
return data
}

73
src/client/targets.v 100644
View File

@ -0,0 +1,73 @@
module client
import models { Target, TargetFilter }
import net.http { Method }
import response { Response }
// get_targets returns a list of targets, given a filter object.
pub fn (c &Client) get_targets(filter TargetFilter) ?[]Target {
params := models.params_from(filter)
data := c.send_request<[]Target>(Method.get, '/api/v1/targets', params)?
return data.data
}
// get_all_targets retrieves *all* targs from the API using the default
// limit.
pub fn (c &Client) get_all_targets() ?[]Target {
mut targets := []Target{}
mut offset := u64(0)
for {
sub_targets := c.get_targets(offset: offset)?
if sub_targets.len == 0 {
break
}
targets << sub_targets
offset += u64(sub_targets.len)
}
return targets
}
// get_target returns the target for a specific id.
pub fn (c &Client) get_target(id int) ?Target {
data := c.send_request<Target>(Method.get, '/api/v1/targets/$id', {})?
return data.data
}
// add_target adds a new target to the server.
pub fn (c &Client) add_target(url string, branch string, repo string, arch []string) ?Response<string> {
mut params := {
'url': url
'branch': branch
'repo': repo
}
if arch.len > 0 {
params['arch'] = arch.join(',')
}
data := c.send_request<string>(Method.post, '/api/v1/targets', params)?
return data
}
// remove_target removes the target with the given id from the server.
pub fn (c &Client) remove_target(id int) ?Response<string> {
data := c.send_request<string>(Method.delete, '/api/v1/targets/$id', {})?
return data
}
// patch_target sends a PATCH request to the given target with the params as
// payload.
pub fn (c &Client) patch_target(id int, params map[string]string) ?Response<string> {
data := c.send_request<string>(Method.patch, '/api/v1/targets/$id', params)?
return data
}

22
src/console/logs/logs.v
View File

@ -12,7 +12,7 @@ struct Config {
api_key string [required]
}
// cmd returns the cli module that handles the build repos API.
// cmd returns the cli module that handles the build logs API.
pub fn cmd() cli.Command {
return cli.Command{
name: 'logs'
@ -33,8 +33,8 @@ pub fn cmd() cli.Command {
flag: cli.FlagType.int
},
cli.Flag{
name: 'repo'
description: 'Only return logs for this repo id.'
name: 'target'
description: 'Only return logs for this target id.'
flag: cli.FlagType.int
},
cli.Flag{
@ -79,9 +79,9 @@ pub fn cmd() cli.Command {
filter.offset = u64(offset)
}
repo_id := cmd.flags.get_int('repo')?
if repo_id != 0 {
filter.repo = repo_id
target_id := cmd.flags.get_int('target')?
if target_id != 0 {
filter.target = target_id
}
tz_offset := time.offset()
@ -168,10 +168,10 @@ pub fn cmd() cli.Command {
// print_log_list prints a list of logs.
fn print_log_list(logs []BuildLog) ? {
data := logs.map([it.id.str(), it.repo_id.str(), it.start_time.local().str(),
data := logs.map([it.id.str(), it.target_id.str(), it.start_time.local().str(),
it.exit_code.str()])
println(console.pretty_table(['id', 'repo', 'start time', 'exit code'], data)?)
println(console.pretty_table(['id', 'target', 'start time', 'exit code'], data)?)
}
// list prints a list of all build logs.
@ -182,10 +182,10 @@ fn list(conf Config, filter BuildLogFilter) ? {
print_log_list(logs)?
}
// list prints a list of all build logs for a given repo.
fn list_for_repo(conf Config, repo_id int) ? {
// list prints a list of all build logs for a given target.
fn list_for_target(conf Config, target_id int) ? {
c := client.new(conf.address, conf.api_key)
logs := c.get_build_logs_for_repo(repo_id)?.data
logs := c.get_build_logs_for_target(target_id)?.data
print_log_list(logs)?
}

6
src/console/git/build.v → src/console/targets/build.v
View File

@ -1,14 +1,14 @@
module git
module targets
import client
import docker
import os
import build
// build builds every Git repo in the server's list.
// build locally builds the target with the given id.
fn build(conf Config, repo_id int) ? {
c := client.new(conf.address, conf.api_key)
repo := c.get_git_repo(repo_id)?
repo := c.get_target(repo_id)?
build_arch := os.uname().machine

36
src/console/git/git.v → src/console/targets/targets.v
View File

@ -1,11 +1,11 @@
module git
module targets
import cli
import vieter.vconf
import cron.expression { parse_expression }
import client
import console
import models { GitRepoFilter }
import models { TargetFilter }
struct Config {
address string [required]
@ -16,12 +16,12 @@ struct Config {
// cmd returns the cli submodule that handles the repos API interaction
pub fn cmd() cli.Command {
return cli.Command{
name: 'repos'
description: 'Interact with the repos API.'
name: 'targets'
description: 'Interact with the targets API.'
commands: [
cli.Command{
name: 'list'
description: 'List the current repos.'
description: 'List the current targets.'
flags: [
cli.Flag{
name: 'limit'
@ -35,7 +35,7 @@ pub fn cmd() cli.Command {
},
cli.Flag{
name: 'repo'
description: 'Only return Git repos that publish to this repo.'
description: 'Only return targets that publish to this repo.'
flag: cli.FlagType.string
},
]
@ -43,7 +43,7 @@ pub fn cmd() cli.Command {
config_file := cmd.flags.get_string('config-file')?
conf := vconf.load<Config>(prefix: 'VIETER_', default_path: config_file)?
mut filter := GitRepoFilter{}
mut filter := TargetFilter{}
limit := cmd.flags.get_int('limit')?
if limit != 0 {
@ -67,7 +67,7 @@ pub fn cmd() cli.Command {
name: 'add'
required_args: 3
usage: 'url branch repo'
description: 'Add a new repository.'
description: 'Add a new Git repository target.'
execute: fn (cmd cli.Command) ? {
config_file := cmd.flags.get_string('config-file')?
conf := vconf.load<Config>(prefix: 'VIETER_', default_path: config_file)?
@ -79,7 +79,7 @@ pub fn cmd() cli.Command {
name: 'remove'
required_args: 1
usage: 'id'
description: 'Remove a repository that matches the given ID prefix.'
description: 'Remove a target that matches the given id.'
execute: fn (cmd cli.Command) ? {
config_file := cmd.flags.get_string('config-file')?
conf := vconf.load<Config>(prefix: 'VIETER_', default_path: config_file)?
@ -91,7 +91,7 @@ pub fn cmd() cli.Command {
name: 'info'
required_args: 1
usage: 'id'
description: 'Show detailed information for the repo matching the ID prefix.'
description: 'Show detailed information for the target matching the id.'
execute: fn (cmd cli.Command) ? {
config_file := cmd.flags.get_string('config-file')?
conf := vconf.load<Config>(prefix: 'VIETER_', default_path: config_file)?
@ -103,7 +103,7 @@ pub fn cmd() cli.Command {
name: 'edit'
required_args: 1
usage: 'id'
description: 'Edit the repository that matches the given ID prefix.'
description: 'Edit the Git repository target that matches the given id.'
flags: [
cli.Flag{
name: 'url'
@ -152,7 +152,7 @@ pub fn cmd() cli.Command {
name: 'build'
required_args: 1
usage: 'id'
description: 'Build the repo with the given id & publish it.'
description: 'Build the target with the given id & publish it.'
execute: fn (cmd cli.Command) ? {
config_file := cmd.flags.get_string('config-file')?
conf := vconf.load<Config>(prefix: 'VIETER_', default_path: config_file)?
@ -168,9 +168,9 @@ pub fn cmd() cli.Command {
// ID. If multiple or none are found, an error is raised.
// list prints out a list of all repositories.
fn list(conf Config, filter GitRepoFilter) ? {
fn list(conf Config, filter TargetFilter) ? {
c := client.new(conf.address, conf.api_key)
repos := c.get_git_repos(filter)?
repos := c.get_targets(filter)?
data := repos.map([it.id.str(), it.url, it.branch, it.repo])
println(console.pretty_table(['id', 'url', 'branch', 'repo'], data)?)
@ -179,7 +179,7 @@ fn list(conf Config, filter GitRepoFilter) ? {
// add adds a new repository to the server's list.
fn add(conf Config, url string, branch string, repo string) ? {
c := client.new(conf.address, conf.api_key)
res := c.add_git_repo(url, branch, repo, [])?
res := c.add_target(url, branch, repo, [])?
println(res.message)
}
@ -191,7 +191,7 @@ fn remove(conf Config, id string) ? {
if id_int != 0 {
c := client.new(conf.address, conf.api_key)
res := c.remove_git_repo(id_int)?
res := c.remove_target(id_int)?
println(res.message)
}
}
@ -209,7 +209,7 @@ fn patch(conf Config, id string, params map[string]string) ? {
id_int := id.int()
if id_int != 0 {
c := client.new(conf.address, conf.api_key)
res := c.patch_git_repo(id_int, params)?
res := c.patch_target(id_int, params)?
println(res.message)
}
@ -224,6 +224,6 @@ fn info(conf Config, id string) ? {
}
c := client.new(conf.address, conf.api_key)
repo := c.get_git_repo(id_int)?
repo := c.get_target(id_int)?
println(repo)
}

10
src/cron/daemon/daemon.v
View File

@ -9,7 +9,7 @@ import build
import docker
import os
import client
import models { GitRepo }
import models { Target }
const (
// How many seconds to wait before retrying to update API if failed
@ -20,7 +20,7 @@ const (
struct ScheduledBuild {
pub:
repo GitRepo
repo Target
timestamp time.Time
}
@ -38,7 +38,7 @@ mut:
api_update_frequency int
image_rebuild_frequency int
// Repos currently loaded from API.
repos []GitRepo
repos []Target
// At what point to update the list of repositories.
api_update_timestamp time.Time
image_build_timestamp time.Time
@ -149,7 +149,7 @@ pub fn (mut d Daemon) run() {
}
// schedule_build adds the next occurence of the given repo build to the queue.
fn (mut d Daemon) schedule_build(repo GitRepo) {
fn (mut d Daemon) schedule_build(repo Target) {
ce := if repo.schedule != '' {
parse_expression(repo.schedule) or {
// TODO This shouldn't return an error if the expression is empty.
@ -178,7 +178,7 @@ fn (mut d Daemon) schedule_build(repo GitRepo) {
fn (mut d Daemon) renew_repos() {
d.linfo('Renewing repos...')
mut new_repos := d.client.get_all_git_repos() or {
mut new_repos := d.client.get_all_targets() or {
d.lerror('Failed to renew repos. Retrying in ${daemon.api_update_retry_timeout}s...')
d.api_update_timestamp = time.now().add_seconds(daemon.api_update_retry_timeout)

8
src/db/db.v
View File

@ -13,8 +13,12 @@ struct MigrationVersion {
}
const (
migrations_up = [$embed_file('migrations/001-initial/up.sql')]
migrations_down = [$embed_file('migrations/001-initial/down.sql')]
migrations_up = [
$embed_file('migrations/001-initial/up.sql'),
$embed_file('migrations/002-rename-to-targets/up.sql'),
]
migrations_down = [$embed_file('migrations/001-initial/down.sql'),
$embed_file('migrations/002-rename-to-targets/down.sql')]
)
// init initializes a database & adds the correct tables.

99
src/db/git.v
View File

@ -1,99 +0,0 @@
module db
import models { GitRepo, GitRepoArch, GitRepoFilter }
// get_git_repos returns all GitRepo's in the database.
pub fn (db &VieterDb) get_git_repos(filter GitRepoFilter) []GitRepo {
// This seems to currently be blocked by a bug in the ORM, I'll have to ask
// around.
if filter.repo != '' {
res := sql db.conn {
select from GitRepo where repo == filter.repo order by id limit filter.limit offset filter.offset
}
return res
}
res := sql db.conn {
select from GitRepo order by id limit filter.limit offset filter.offset
}
return res
}
// get_git_repo tries to return a specific GitRepo.
pub fn (db &VieterDb) get_git_repo(repo_id int) ?GitRepo {
res := sql db.conn {
select from GitRepo where id == repo_id
}
// If a select statement fails, it returns a zeroed object. By
// checking one of the required fields, we can see whether the query
// returned a result or not.
if res.id == 0 {
return none
}
return res
}
// add_git_repo inserts the given GitRepo into the database.
pub fn (db &VieterDb) add_git_repo(repo GitRepo) {
sql db.conn {
insert repo into GitRepo
}
}
// delete_git_repo deletes the repo with the given ID from the database.
pub fn (db &VieterDb) delete_git_repo(repo_id int) {
sql db.conn {
delete from GitRepo where id == repo_id
delete from GitRepoArch where repo_id == repo_id
}
}
// update_git_repo updates any non-array values for a given GitRepo.
pub fn (db &VieterDb) update_git_repo(repo_id int, params map[string]string) {
mut values := []string{}
// TODO does this allow for SQL injection?
$for field in GitRepo.fields {
if field.name in params {
// Any fields that are array types require their own update method
$if field.typ is string {
values << "$field.name = '${params[field.name]}'"
}
}
}
values_str := values.join(', ')
// I think this is actual SQL & not the ORM language
query := 'update GitRepo set $values_str where id == $repo_id'
db.conn.exec_none(query)
}
// update_git_repo_archs updates a given GitRepo's arch value.
pub fn (db &VieterDb) update_git_repo_archs(repo_id int, archs []GitRepoArch) {
archs_with_id := archs.map(GitRepoArch{
...it
repo_id: repo_id
})
sql db.conn {
delete from GitRepoArch where repo_id == repo_id
}
for arch in archs_with_id {
sql db.conn {
insert arch into GitRepoArch
}
}
}
// git_repo_exists is a utility function that checks whether a repo with the
// given id exists.
pub fn (db &VieterDb) git_repo_exists(repo_id int) bool {
db.get_git_repo(repo_id) or { return false }
return true
}

12
src/db/logs.v
View File

@ -7,8 +7,8 @@ import time
pub fn (db &VieterDb) get_build_logs(filter BuildLogFilter) []BuildLog {
mut where_parts := []string{}
if filter.repo != 0 {
where_parts << 'repo_id == $filter.repo'
if filter.target != 0 {
where_parts << 'target_id == $filter.target'
}
if filter.before != time.Time{} {
@ -55,11 +55,11 @@ pub fn (db &VieterDb) get_build_logs(filter BuildLogFilter) []BuildLog {
return res
}
// get_build_logs_for_repo returns all BuildLog's in the database for a given
// repo.
pub fn (db &VieterDb) get_build_logs_for_repo(repo_id int) []BuildLog {
// get_build_logs_for_target returns all BuildLog's in the database for a given
// target.
pub fn (db &VieterDb) get_build_logs_for_target(target_id int) []BuildLog {
res := sql db.conn {
select from BuildLog where repo_id == repo_id order by id
select from BuildLog where target_id == target_id order by id
}
return res

5
src/db/migrations/002-rename-to-targets/down.sql 100644
View File

@ -0,0 +1,5 @@
ALTER TABLE Target RENAME TO GitRepo;
ALTER TABLE TargetArch RENAME TO GitRepoArch;
ALTER TABLE GitRepoArch RENAME COLUMN target_id TO repo_id;
ALTER TABLE BuildLog RENAME COLUMN target_id TO repo_id;

5
src/db/migrations/002-rename-to-targets/up.sql 100644
View File

@ -0,0 +1,5 @@
ALTER TABLE GitRepo RENAME TO Target;
ALTER TABLE GitRepoArch RENAME TO TargetArch;
ALTER TABLE TargetArch RENAME COLUMN repo_id TO target_id;
ALTER TABLE BuildLog RENAME COLUMN repo_id TO target_id;

99
src/db/targets.v 100644
View File

@ -0,0 +1,99 @@
module db
import models { Target, TargetArch, TargetFilter }
// get_targets returns all targets in the database.
pub fn (db &VieterDb) get_targets(filter TargetFilter) []Target {
// This seems to currently be blocked by a bug in the ORM, I'll have to ask
// around.
if filter.repo != '' {
res := sql db.conn {
select from Target where repo == filter.repo order by id limit filter.limit offset filter.offset
}
return res
}
res := sql db.conn {
select from Target order by id limit filter.limit offset filter.offset
}
return res
}
// get_target tries to return a specific target.
pub fn (db &VieterDb) get_target(target_id int) ?Target {
res := sql db.conn {
select from Target where id == target_id
}
// If a select statement fails, it returns a zeroed object. By
// checking one of the required fields, we can see whether the query
// returned a result or not.
if res.id == 0 {
return none
}
return res
}
// add_target inserts the given target into the database.
pub fn (db &VieterDb) add_target(repo Target) {
sql db.conn {
insert repo into Target
}
}
// delete_target deletes the target with the given id from the database.
pub fn (db &VieterDb) delete_target(target_id int) {
sql db.conn {
delete from Target where id == target_id
delete from TargetArch where target_id == target_id
}
}
// update_target updates any non-array values for a given target.
pub fn (db &VieterDb) update_target(target_id int, params map[string]string) {
mut values := []string{}
// TODO does this allow for SQL injection?
$for field in Target.fields {
if field.name in params {
// Any fields that are array types require their own update method
$if field.typ is string {
values << "$field.name = '${params[field.name]}'"
}
}
}
values_str := values.join(', ')
// I think this is actual SQL & not the ORM language
query := 'update Target set $values_str where id == $target_id'
db.conn.exec_none(query)
}
// update_target_archs updates a given target's arch value.
pub fn (db &VieterDb) update_target_archs(target_id int, archs []TargetArch) {
archs_with_id := archs.map(TargetArch{
...it
target_id: target_id
})
sql db.conn {
delete from TargetArch where target_id == target_id
}
for arch in archs_with_id {
sql db.conn {
insert arch into TargetArch
}
}
}
// target_exists is a utility function that checks whether a target with the
// given id exists.
pub fn (db &VieterDb) target_exists(target_id int) bool {
db.get_target(target_id) or { return false }
return true
}

4
src/main.v
View File

@ -3,7 +3,7 @@ module main
import os
import server
import cli
import console.git
import console.targets
import console.logs
import console.schedule
import console.man
@ -26,7 +26,7 @@ fn main() {
]
commands: [
server.cmd(),
git.cmd(),
targets.cmd(),
cron.cmd(),
logs.cmd(),
schedule.cmd(),

6
src/models/logs.v
View File

@ -5,7 +5,7 @@ import time
pub struct BuildLog {
pub mut:
id int [primary; sql: serial]
repo_id int [nonull]
target_id int [nonull]
start_time time.Time [nonull]
end_time time.Time [nonull]
arch string [nonull]
@ -16,7 +16,7 @@ pub mut:
pub fn (bl &BuildLog) str() string {
mut parts := [
'id: $bl.id',
'repo id: $bl.repo_id',
'target id: $bl.target_id',
'start time: $bl.start_time.local()',
'end time: $bl.end_time.local()',
'duration: ${bl.end_time - bl.start_time}',
@ -33,7 +33,7 @@ pub struct BuildLogFilter {
pub mut:
limit u64 = 25
offset u64
repo int
target int
before time.Time
after time.Time
arch string

4
src/models/models.v
View File

@ -23,8 +23,8 @@ pub fn patch_from_params<T>(mut o T, params map[string]string) ? {
o.$(field.name) = params[field.name].int()
} $else $if field.typ is u64 {
o.$(field.name) = params[field.name].u64()
} $else $if field.typ is []GitRepoArch {
o.$(field.name) = params[field.name].split(',').map(GitRepoArch{ value: it })
} $else $if field.typ is []TargetArch {
o.$(field.name) = params[field.name].split(',').map(TargetArch{ value: it })
} $else $if field.typ is time.Time {
o.$(field.name) = time.unix(params[field.name].int())
} $else $if field.typ is []string {

14
src/models/git.v → src/models/targets.v
View File

@ -1,18 +1,18 @@
module models
pub struct GitRepoArch {
pub struct TargetArch {
pub:
id int [primary; sql: serial]
repo_id int [nonull]
target_id int [nonull]
value string [nonull]
}
// str returns a string representation.
pub fn (gra &GitRepoArch) str() string {
pub fn (gra &TargetArch) str() string {
return gra.value
}
pub struct GitRepo {
pub struct Target {
pub mut:
id int [primary; sql: serial]
// URL of the Git repository
@ -25,11 +25,11 @@ pub mut:
schedule string
// On which architectures the package is allowed to be built. In reality,
// this controls which builders will periodically build the image.
arch []GitRepoArch [fkey: 'repo_id']
arch []TargetArch [fkey: 'target_id']
}
// str returns a string representation.
pub fn (gr &GitRepo) str() string {
pub fn (gr &Target) str() string {
mut parts := [
'id: $gr.id',
'url: $gr.url',
@ -44,7 +44,7 @@ pub fn (gr &GitRepo) str() string {
}
[params]
pub struct GitRepoFilter {
pub struct TargetFilter {
pub mut:
limit u64 = 25
offset u64

6
src/server/README.md 100644
View File

@ -0,0 +1,6 @@
This module contains the Vieter HTTP server, consisting of the repository
implementation & the REST API.
**NOTE**: vweb defines the priority order of routes by the file names in this
module. Therefore, it's very important that all API routes are defined in files
prefixed with `api_`, as this is before the word `routes` alphabetically.

36
src/server/logs.v → src/server/api_logs.v
View File

@ -10,10 +10,10 @@ import os
import util
import models { BuildLog, BuildLogFilter }
// get_logs returns all build logs in the database. A 'repo' query param can
// v1_get_logs returns all build logs in the database. A 'target' query param can
// optionally be added to limit the list of build logs to that repository.
['/api/logs'; get]
fn (mut app App) get_logs() web.Result {
['/api/v1/logs'; get]
fn (mut app App) v1_get_logs() web.Result {
if !app.is_authorized() {
return app.json(http.Status.unauthorized, new_response('Unauthorized.'))
}
@ -26,9 +26,9 @@ fn (mut app App) get_logs() web.Result {
return app.json(http.Status.ok, new_data_response(logs))
}
// get_single_log returns the build log with the given id.
['/api/logs/:id'; get]
fn (mut app App) get_single_log(id int) web.Result {
// v1_get_single_log returns the build log with the given id.
['/api/v1/logs/:id'; get]
fn (mut app App) v1_get_single_log(id int) web.Result {
if !app.is_authorized() {
return app.json(http.Status.unauthorized, new_response('Unauthorized.'))
}
@ -38,16 +38,16 @@ fn (mut app App) get_single_log(id int) web.Result {
return app.json(http.Status.ok, new_data_response(log))
}
// get_log_content returns the actual build log file for the given id.
['/api/logs/:id/content'; get]
fn (mut app App) get_log_content(id int) web.Result {
// v1_get_log_content returns the actual build log file for the given id.
['/api/v1/logs/:id/content'; get]
fn (mut app App) v1_get_log_content(id int) web.Result {
if !app.is_authorized() {
return app.json(http.Status.unauthorized, new_response('Unauthorized.'))
}
log := app.db.get_build_log(id) or { return app.not_found() }
file_name := log.start_time.custom_format('YYYY-MM-DD_HH-mm-ss')
full_path := os.join_path(app.conf.data_dir, logs_dir_name, log.repo_id.str(), log.arch,
full_path := os.join_path(app.conf.data_dir, logs_dir_name, log.target_id.str(), log.arch,
file_name)
return app.file(full_path)
@ -62,9 +62,9 @@ fn parse_query_time(query string) ?time.Time {
return t
}
// post_log adds a new log to the database.
['/api/logs'; post]
fn (mut app App) post_log() web.Result {
// v1_post_log adds a new log to the database.
['/api/v1/logs'; post]
fn (mut app App) v1_post_log() web.Result {
if !app.is_authorized() {
return app.json(http.Status.unauthorized, new_response('Unauthorized.'))
}
@ -96,15 +96,15 @@ fn (mut app App) post_log() web.Result {
arch := app.query['arch']
repo_id := app.query['repo'].int()
target_id := app.query['target'].int()
if !app.db.git_repo_exists(repo_id) {
return app.json(http.Status.bad_request, new_response('Unknown Git repo.'))
if !app.db.target_exists(target_id) {
return app.json(http.Status.bad_request, new_response('Unknown target.'))
}
// Store log in db
log := BuildLog{
repo_id: repo_id
target_id: target_id
start_time: start_time
end_time: end_time
arch: arch
@ -113,7 +113,7 @@ fn (mut app App) post_log() web.Result {
app.db.add_build_log(log)
repo_logs_dir := os.join_path(app.conf.data_dir, logs_dir_name, repo_id.str(), arch)
repo_logs_dir := os.join_path(app.conf.data_dir, logs_dir_name, target_id.str(), arch)
// Create the logs directory of it doesn't exist
if !os.exists(repo_logs_dir) {

50
src/server/git.v → src/server/api_targets.v
View File

@ -4,38 +4,38 @@ import web
import net.http
import response { new_data_response, new_response }
import db
import models { GitRepo, GitRepoArch, GitRepoFilter }
import models { Target, TargetArch, TargetFilter }
// get_repos returns the current list of repos.
['/api/repos'; get]
fn (mut app App) get_repos() web.Result {
// v1_get_targets returns the current list of targets.
['/api/v1/targets'; get]
fn (mut app App) v1_get_targets() web.Result {
if !app.is_authorized() {
return app.json(http.Status.unauthorized, new_response('Unauthorized.'))
}
filter := models.from_params<GitRepoFilter>(app.query) or {
filter := models.from_params<TargetFilter>(app.query) or {
return app.json(http.Status.bad_request, new_response('Invalid query parameters.'))
}
repos := app.db.get_git_repos(filter)
repos := app.db.get_targets(filter)
return app.json(http.Status.ok, new_data_response(repos))
}
// get_single_repo returns the information for a single repo.
['/api/repos/:id'; get]
fn (mut app App) get_single_repo(id int) web.Result {
// v1_get_single_target returns the information for a single target.
['/api/v1/targets/:id'; get]
fn (mut app App) v1_get_single_target(id int) web.Result {
if !app.is_authorized() {
return app.json(http.Status.unauthorized, new_response('Unauthorized.'))
}
repo := app.db.get_git_repo(id) or { return app.not_found() }
repo := app.db.get_target(id) or { return app.not_found() }
return app.json(http.Status.ok, new_data_response(repo))
}
// post_repo creates a new repo from the provided query string.
['/api/repos'; post]
fn (mut app App) post_repo() web.Result {
// v1_post_target creates a new target from the provided query string.
['/api/v1/targets'; post]
fn (mut app App) v1_post_target() web.Result {
if !app.is_authorized() {
return app.json(http.Status.unauthorized, new_response('Unauthorized.'))
}
@ -48,40 +48,40 @@ fn (mut app App) post_repo() web.Result {
params['arch'] = app.conf.default_arch
}
new_repo := models.from_params<GitRepo>(params) or {
new_repo := models.from_params<Target>(params) or {
return app.json(http.Status.bad_request, new_response(err.msg()))
}
app.db.add_git_repo(new_repo)
app.db.add_target(new_repo)
return app.json(http.Status.ok, new_response('Repo added successfully.'))
}
// delete_repo removes a given repo from the server's list.
['/api/repos/:id'; delete]
fn (mut app App) delete_repo(id int) web.Result {
// v1_delete_target removes a given target from the server's list.
['/api/v1/targets/:id'; delete]
fn (mut app App) v1_delete_target(id int) web.Result {
if !app.is_authorized() {
return app.json(http.Status.unauthorized, new_response('Unauthorized.'))
}
app.db.delete_git_repo(id)
app.db.delete_target(id)
return app.json(http.Status.ok, new_response('Repo removed successfully.'))
}
// patch_repo updates a repo's data with the given query params.
['/api/repos/:id'; patch]
fn (mut app App) patch_repo(id int) web.Result {
// v1_patch_target updates a target's data with the given query params.
['/api/v1/targets/:id'; patch]
fn (mut app App) v1_patch_target(id int) web.Result {
if !app.is_authorized() {
return app.json(http.Status.unauthorized, new_response('Unauthorized.'))
}
app.db.update_git_repo(id, app.query)
app.db.update_target(id, app.query)
if 'arch' in app.query {
arch_objs := app.query['arch'].split(',').map(GitRepoArch{ value: it })
arch_objs := app.query['arch'].split(',').map(TargetArch{ value: it })
app.db.update_git_repo_archs(id, arch_objs)
app.db.update_target_archs(id, arch_objs)
}
return app.json(http.Status.ok, new_response('Repo updated successfully.'))

1
src/server/cli.v
View File

@ -10,6 +10,7 @@ pub:
data_dir string
api_key string
default_arch string
port int = 8000
}
// cmd returns the cli submodule that handles starting the server

3
src/server/server.v
View File

@ -8,7 +8,6 @@ import util
import db
const (
port = 8000
log_file_name = 'vieter.log'
repo_dir_name = 'repos'
db_file_name = 'vieter.sqlite'
@ -77,5 +76,5 @@ pub fn server(conf Config) ? {
conf: conf
repo: repo
db: db
}, server.port)
}, conf.port)
}