87 lines
3.6 KiB
Markdown
87 lines
3.6 KiB
Markdown
# Roadmap
|
|
|
|
This file describes a broad roadmap of what's currently planned with the
|
|
project.
|
|
|
|
## Readable repository
|
|
|
|
The first goal is to make the server a fully functional Debian package
|
|
repository. This includes:
|
|
|
|
### Setting up a database
|
|
|
|
Hilde stores an index of the stored deb files in a PostgreSQL database. Any
|
|
data that can be calculated beforehand and/or stored efficiently inside a
|
|
database (e.g. hashes, package names & versions, descriptions etc.) will be
|
|
stored there. Any request that isn't a deb file will be queried and/or
|
|
generated from this database. For example, the `Packages` file can be
|
|
generated on the fly instead of storing it on disk. The goal is for the data
|
|
storage to only include deb files.
|
|
|
|
### Indexing the directory on startup
|
|
|
|
On startup, Hilde will scan the packages directory for deb files. These will
|
|
then all be parsed, and synced with the database.
|
|
|
|
### Functioning as a repository
|
|
|
|
Using this database backend, the server should fully mimic a Debian repository
|
|
server ([an example](http://ftp.debian.org/debian/)). As there's no real reason
|
|
not to, we should implement the
|
|
[repository spec](https://wiki.debian.org/DebianRepository) as complete as
|
|
possible.
|
|
|
|
## The ideas
|
|
|
|
Now that we have a repository that we can use, it's time to add some extra
|
|
functionality.
|
|
|
|
### Uploading versions
|
|
|
|
This is the initial reason I wanted to start this project. My goal is to add a
|
|
way to send POST requests to the server containing new versions of a new or
|
|
existing package. These packages can be from any distribution, architecture, or
|
|
name, and the server will just dynamically create the directories n stuff when
|
|
needed. This feature can then be used in CI pipelines to simplify uploading
|
|
versions to a repository.
|
|
|
|
### User management & JWT tokens
|
|
|
|
Considering we can't just make an upload endpoint publicly available, we'll
|
|
need a system to log in. There will be an admin user that can do everything,
|
|
but I want to add a detailed user management system. This system will have
|
|
fine-grained controls over what user can do what, and each user is able to
|
|
create JWT tokens that further narrows down the functionality. For example, a
|
|
user could have read & write access to the stable distribution's updates
|
|
section. Then they can create a token that only allows to write a certain
|
|
package, and use it in their CI pipeline for that particular package.
|
|
|
|
### Federation
|
|
|
|
I think the idea of being able to spread a service across multiple hosts is
|
|
really cool, so I've thought about adding this to Hilde as well. I have never
|
|
created anything like this before, but I think the idea is really cool. My
|
|
current approach would be as follows:
|
|
|
|
The nodes communicate using a common secret key. Each node only requires the
|
|
location of a single other node. When a new node connects to the network, it
|
|
asks the node it's been given about which nodes it knows. These nodes are then
|
|
added to the node's list of known nodes. These new nodes are then also
|
|
notified, and this process is repeated until all nodes are discovered. Each
|
|
node also keeps an index of which packages can be found on which node. This
|
|
makes it so that every node can be used as an entrypoint to the network, making
|
|
it easy to load balance.
|
|
|
|
Each node could maybe send a notification to all the other nodes when it
|
|
discovers/loses a node, or receives a new package.
|
|
|
|
When uploading a new package, we could query the system to see which node has
|
|
the most space left, as to more efficiently distribute the packages, if
|
|
possible.
|
|
|
|
### Mirroring repositories
|
|
|
|
It might be useful to add functionality to mirror an existing repo. This could
|
|
for example make a home setup fully self-sufficient, by mirroring the official
|
|
Debian repositories.
|