Tha_14/tomato-testing
1
0
Fork 0
forked from Green-Sky/tomato
Code Pull Requests Actions Packages Releases Activity

change binary meta format and add zstd to metadata

Browse Source
This commit is contained in:
Green Sky
2024-02-19 12:32:09 +01:00
parent 4fb2b51b7d
commit 6aac44cda9
6 changed files with 160 additions and 67 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
src/fragment_store/README.md
View File

@@ -40,25 +40,26 @@ A Metadata json object can have arbitrary keys, some are predefined:
### Split Metadata
file magic bytes `SOLMET` (6 bytes)
msgpack array:
1 byte encryption type (`0x00` is none)
1 byte compression type (`0x00` is none)
...metadata here...
- `[0]`: file magic string `SOLMET` (6 bytes)
- `[1]`: uint8 encryption type (`0x00` is none)
- `[2]`: uint8 compression type (`0x00` is none, `0x01` is zstd)
- `[3]`: binary metadata (optionally compressed and encrypted)
note that the encryption and compression are for the metadata only.
The metadata itself contains encryption and compression info about the data.
### Split Data
(none) all the data is in the metadata file.
All the metadata is in the metadata file. (like encryption and compression)
This is mostly to allow direct storage for files in the Fragment store without excessive duplication.
Keep in mind to not use the actual file name as the data/meta file name.
### Single fragment
Note: this format is unused for now
file magic bytes `SOLFIL` (6 bytes)
1 byte encryption type (`0x00` is none)