Implement defragment operation for lsm_store #28

New Issue

Jef Roosens · 2023-11-16T19:01:12+01:00

Jef Roosens commented

2023-11-16 19:01:12 +01:00

An active database will inevitably accumulate some internal fragmentation. To fight this, a defragment operation can be introduced when can e.g. run on startup.

Implementing this should be very simple: iterate over the idx file, write any entries with a length greater than zero to a new idx file, and write the respective data from the database to a new database file. The indexes of the idx file entries will need to be adjusted though, as the database file's offsets will have changed.

An active database will inevitably accumulate some internal fragmentation. To fight this, a defragment operation can be introduced when can e.g. run on startup. Implementing this should be very simple: iterate over the idx file, write any entries with a length greater than zero to a new idx file, and write the respective data from the database to a new database file. The indexes of the idx file entries will need to be adjusted though, as the database file's offsets will have changed.

Jef Roosens added the

Kind/Feature

label 2023-11-16 19:01:12 +01:00

Jef Roosens changed title from ~~Implement vacuum operation for `lsm_store`~~ to Implement defragment operation for `lsm_store`

2023-11-16 19:01:33 +01:00

Jef Roosens added the

Project/Lander

label 2024-08-26 10:41:54 +02:00

Sign in to join this conversation.