Compression and Transfer-Encoding #51

New Issue

Jef Roosens · 2024-04-14T11:06:24+02:00

Jef Roosens commented

2024-04-14 11:06:24 +02:00

Ideally I'd like to keep the space required for the server to a minimum, so compressing the stored files sounds like a good idea. Additionally, using the Transfer-Encoding header would allow us to reduce the amount of bandwidth required for downloads as well.

When uploading using the CLI, the CLI could compress the files before sending them over the wire. When using cURL the server would compress the files once when they're received instead. Either way, the server would only store a compressed version of each uploaded file.

When downloading, the server would simply serve the file in its compressed form with the Transfer-Encoding header set. This would allow the client to decompress the file while it's being received, without having to send the uncompressed contents over the wire.

Additionally, we could also add a /:key.gz route that explicitely serves the compressed file as the content instead of having it be transparently decompressed by the client.

Ideally I'd like to keep the space required for the server to a minimum, so compressing the stored files sounds like a good idea. Additionally, using the Transfer-Encoding header would allow us to reduce the amount of bandwidth required for downloads as well. When uploading using the CLI, the CLI could compress the files before sending them over the wire. When using cURL the server would compress the files once when they're received instead. Either way, the server would only store a compressed version of each uploaded file. When downloading, the server would simply serve the file in its compressed form with the Transfer-Encoding header set. This would allow the client to decompress the file while it's being received, without having to send the uncompressed contents over the wire. Additionally, we could also add a `/:key.gz` route that explicitely serves the compressed file as the content instead of having it be transparently decompressed by the client.

Jef Roosens added the

idea

label 2024-04-14 11:06:24 +02:00

Jef Roosens commented

2024-04-14 11:07:44 +02:00

Poster

We might need to use the Content-Encoding header instead.

Sign in to join this conversation.