Memory management system #5

New Issue

Jef Roosens · 2022-12-07T14:53:05+01:00

Jef Roosens commented

2022-12-07 14:53:05 +01:00

I just had a rather cool idea.

First of all, I'd like to switch to my own binary format for data storage, where the trie just stores random data as void pointers. The application using the trie would then be responsible for casting the pointers, with the trie not caring about the contents at all. The literal bytes could then be stored inside the data file.

Now, the second idea is to allow configuring the maximum memory that the trie is allowed to use. The trie would keep track of its own internal memory usage, including all allocations etc. The main part where this setting would come into play is when storing the actual data in the trie. As long as the trie has space left, the data will be stored in memory. If there's no space left, only the file path or something of the entry is stored, with it being read from disk when requested.

This idea still needs some fletching out, but it could be cool. The important part here is that this memory management idea could hopefully be managed completely when adding & removing items, and not when searching, meaning search will stay just as fast.

I just had a rather cool idea. First of all, I'd like to switch to my own binary format for data storage, where the trie just stores random data as void pointers. The application using the trie would then be responsible for casting the pointers, with the trie not caring about the contents at all. The literal bytes could then be stored inside the data file. Now, the second idea is to allow configuring the maximum memory that the trie is allowed to use. The trie would keep track of its own internal memory usage, including all allocations etc. The main part where this setting would come into play is when storing the actual data in the trie. As long as the trie has space left, the data will be stored in memory. If there's no space left, only the file path or something of the entry is stored, with it being read from disk when requested. This idea still needs some fletching out, but it could be cool. The important part here is that this memory management idea could hopefully be managed completely when adding & removing items, and not when searching, meaning search will stay just as fast.

Jef Roosens added the

idea

label 2023-11-16 18:07:36 +01:00

Jef Roosens commented

2023-11-16 18:45:49 +01:00

Poster

I don't think this idea is very viable.

Jef Roosens closed this issue

2023-11-16 18:45:49 +01:00

Sign in to join this conversation.