diff --git a/content/dev/otter/_index.md b/content/dev/otter/_index.md new file mode 100644 index 0000000..00b6ac5 --- /dev/null +++ b/content/dev/otter/_index.md @@ -0,0 +1,9 @@ +--- +title: "Otter" +summary: "Lightweight implementation of the Gpodder API" +type: "project" +params: + links: + - name: Source + url: 'https://git.rustybever.be/Chewing_Bever/otter' +--- diff --git a/content/dev/otter/devlog-1.md b/content/dev/otter/devlog-1.md new file mode 100644 index 0000000..53e44ca --- /dev/null +++ b/content/dev/otter/devlog-1.md @@ -0,0 +1,236 @@ +--- +title: "Gpodder, Rust and Domain Driven Design" +date: 2025-03-15T12:55:31+01:00 +--- + +Recently I've gotten into listening to podcasts more, notably the +[Severed](https://open.spotify.com/show/3yJnUXzFvX4p4QlsfVPuyZ) podcast going +in depth on every Severance episode. I use Kasts on my computers and AntennaPod +on my phone to listen, and I looked into how I could sync my subscriptions +between my devices. This search ended with [Gpodder.net](https://gpodder.net/), +a free and open-source web service that's supported by both AntennaPod and +Kasts! The server is also self-hostable, but it's quite the behemoth written in +Python, and I felt like starting another project, so I decided to implement my +own version of the API instead. That project is Otter. + +## The Gpodder API + +The API documentation for Gpodder is available on [Read The +Docs](https://gpoddernet.readthedocs.io/en/latest/). The documentation is okay, +but it lacks a lot of important information about how the API should behave. My +development process for this project consisted of trying to implement routes +via the documentation before testing it using Kasts. This took a lot of trial +and error, but I ended up getting most of the routes working the way most +clients expect (I hope). + +Currently Otter supports all routes required for clients to synchronize +subscriptions and play information for episodes. I've based some of my work on +[Opodsync](https://github.com/kd2org/opodsync), another implementation of the +same idea, written in PHP. + +## Domain Driven Design + +Usually the way I develop projects is very chaotic. I try out different things +to see what sticks and often end up with very interconnected and difficult to +separate components that join together into a big mess. Usually this does clean +up after some time, but I still have a tendency to develop too tightly +connected components. One example of this would be having ORM logic in web +route handlers, or jumping through hoops to be able to use the same structs for +the database operations and the web representation. For Otter, I took a step +back for once. + +I took some inspiration from domain driven design and clean architecture to +structure my codebase. The idea (at least, the way I understand/interpret it) +is to work with abstractions that cleanly separate various components. The +first step was designing said abstraction. I introduced the concept of a +"Gpodder repository", which is an object that provides all the necessary +methods required to function as the backend data store for a Gpodder server. +This repository is defined as a collection of traits: + +```rust +pub trait EpisodeActionRepository { + /// Insert the given episode actions into the datastore. + fn add_episode_actions(&self, user: &User, actions: Vec) + -> Result; + + /// Retrieve the list of episode actions for the given user. + fn episode_actions_for_user( + &self, + user: &User, + since: Option, + podcast: Option, + device: Option, + aggregated: bool, + ) -> Result<(i64, Vec), AuthErr>; +} +``` + +For example, the `EpisodeActionRepository` provides the methods required to +serve the Episode Actions API section of the API documentation. + +```rust +async fn get_episode_actions( + State(ctx): State, + Path(username): Path, + Extension(user): Extension, + Query(filter): Query, +) -> AppResult> { + if username.format != Format::Json { + return Err(AppError::NotFound); + } + + if *username != user.username { + return Err(AppError::BadRequest); + } + + Ok(tokio::task::spawn_blocking(move || { + ctx.repo.episode_actions_for_user( + &user, + filter.since, + filter.podcast, + filter.device, + filter.aggregated, + ) + }) + .await + .unwrap() + .map(|(timestamp, actions)| Json(EpisodeActionsResponse { timestamp, actions }))?) +} +``` + +In our web layer (here I'm using Axum), all that needs to be done is to call +the respective function on the repository and convert it to the correct +representation format, as required by the API. + +This complete separation of concerns makes development much less mentally +taxing in my opinion, as each component can be viewed on its own. The +repository abstraction defines models and methods required for a representation +layer (in this case, the web server) to access the various parts of the +repository without having to know anything about the internal workings. The +current implementation of the repository works using Sqlite, but thanks to the +abstraction, I could add a Postgres implementation as well by simply +reimplementing the abstraction. Rust's type system adds to the strength of this +design pattern by providing strong abstraction primitives via traits. + +## Configuration + +One other fun thing I'd like to mention is how Otter handles configuration +variables. In my eyes, the ideal application supports configuration as a +combination of a config file, environment variables and CLI arguments, with +environment variables overwriting config file variables, and CLI arguments +taking precedence above all. Using [Serde](https://serde.rs/), +[Clap](https://github.com/clap-rs/clap) and +[Figment](https://github.com/SergioBenitez/Figment), this can be done in a very +clean way. + +First, we define our configuration as a struct. This is both the expected +format of the configuration file, and the final configuration struct that we +wish to end up with: + +```rust +#[derive(Deserialize)] +pub struct Config { + #[serde(default = "default_data_dir")] + pub data_dir: PathBuf, + #[serde(default = "default_domain")] + pub domain: String, + #[serde(default = "default_port")] + pub port: u16, +} + +fn default_data_dir() -> PathBuf { + PathBuf::from("./data") +} + +fn default_domain() -> String { + "127.0.0.1".to_string() +} + +fn default_port() -> u16 { + 8080 +} +``` + +Important here is the `Deserialize` derive. The Figment library heavily relies +on Serde to function, and a `Deserialize` implementation is required for our +final configuration to be deserialized into this struct. We also define +sensible defaults where applicable. + +The second step is defining our CLI arguments using Clap: + +```rust +#[derive(Serialize, Args, Clone)] +pub struct ClapConfig { + #[arg( + short, + long = "config", + env = "OTTER_CONFIG_FILE", + value_name = "CONFIG_FILE" + )] + config_file: Option, + + #[serde(skip_serializing_if = "Option::is_none")] + #[arg(long = "data", value_name = "DATA_DIR")] + data_dir: Option, + + #[serde(skip_serializing_if = "Option::is_none")] + #[arg(short, long, value_name = "DOMAIN")] + domain: Option, + + #[serde(skip_serializing_if = "Option::is_none")] + #[arg(short, long, value_name = "PORT")] + port: Option, +} +``` + +Here we see two things of note. First, the `ClapConfig` implements `Serialize`, +not `Deserialize`. This is because we'll be using the `Serialize` +implementation later when combining the configuration sources. We also add a +`#[serde(skip_serializing_if = "Option::is_none")]` attribute to each variable +that can be overwritten. This tells Serde to ignore the variable altogether if +its value is `None` as a workaround for Figment not being able to ignore `None` +values. Important here is how the variables names in the `ClapConfig` struct +are the same as those in the `Config` struct. This is required for Figment to +be able to combine these configuration sources. + +Finally, we combine everything using Figment: + +```rust +let mut figment = Figment::new(); + +if let Some(config_path) = &self.config.config_file { + figment = figment.merge(Toml::file(config_path)); +} + +let config: crate::config::Config = figment + .merge(Env::prefixed("OTTER_")) + .merge(Serialized::defaults(self.config.clone())) + .extract().unwrap(); +``` + +We start by defining an empty figment. In the Figment library, a "figment" is a +combination of configuration providers that can be deserialized using Serde. A +provider is an object that can provide configuration variables from some +source, be it a TOML file or environment variables. Figments can be combined +with providers in various ways. Here, we make use of "merge", which overwrites +any existing configuration variables with the values received from the new +provider. + +We merge the empty figment with the `Toml` provider, which reads a TOML file +for configuration variables. Then, we merge again, first with the +`Env::prefixed("OTTER_")` provider, and then with the +`Serialized::defaults(self.config.clone())` provider. The `Env` provider reads +variables from environment variables, while `Serialized` provides variables by +serializing a struct, in this case our `ClapConfig` struct (hence why we need a +`Serialize` implementation). The result is a figment consisting of variables +from the config file, environment variables and CLI arguments. This can be +deserialized into our `Config` struct using `extract`. + +In my opinion, all of this combines into a very clean configuration system that +allows configuration variables from all convenient sources, without sacrificing +flexibility or resorting to boilerplate code. + +I hope to deploy a first build of Otter on my own servers in the near future to +test it and work out any kinks. Afterwards, I'll announce a `0.1` release! + +Thanks for reading.