Age | Commit message (Collapse) | Author | |
---|---|---|---|
2021-10-18 | update todo and add CLI optionswork/export | n-peugnet | |
2021-10-08 | add chunkSize parameter to cli | n-peugnet | |
2021-10-06 | refactor main and logger | n-peugnet | |
2021-10-01 | add some comments, summary, TODOs and infos | n-peugnet | |
2021-09-30 | properly manage relative symlinks will CI pass ?? | n-peugnet | |
2021-09-29 | support symlinks if it is internal to source | n-peugnet | |
adapt tests | |||
2021-09-29 | move old funcs to repo_test and add roundtrip test | n-peugnet | |
2021-09-29 | merge tryDeltaEncodeChunk and encodeTempChunk | n-peugnet | |
to reuse the sketch calculation | |||
2021-09-29 | add details and some todos | n-peugnet | |
2021-09-27 | add CLI with subcommands | n-peugnet | |
2021-09-23 | Testing with Fdelta as the differ | n-peugnet | |
2021-09-22 | fix mystical bug by making multiple passes | n-peugnet | |
We make ne passes while new blocks have been added. This way we are assured that the result will be the same on following runs. result : 161 /tmp/test-1/00001/recipe 161 /tmp/test-1/00001/files 0 /tmp/test-1/00001/hashes 40 /tmp/test-1/00001/chunks 482 /tmp/test-1/00001 9904052 /tmp/test-1/00000/recipe 5377304 /tmp/test-1/00000/files 2061331 /tmp/test-1/00000/hashes 207958371 /tmp/test-1/00000/chunks 225301178 /tmp/test-1/00000 225301740 /tmp/test-1 | |||
2021-09-22 | trying to fix the mystical bug | n-peugnet | |
If prev is not null and no match if found, always encode both remaining chunks. Previously some chunks of chunkSize could have been stored as TempChunks in the recipe instead of as StoredChunks with hashes and an Id. This did not fix the mystical bug. But it helped finding where it came from. | |||
2021-09-21 | store file listing incrementally | n-peugnet | |
2021-09-21 | store recipes incrementally | n-peugnet | |
2021-09-21 | change recipe into a generic slice patch/diff package | n-peugnet | |
2021-09-20 | update todolist | n-peugnet | |
2021-09-20 | store recipe compressed | n-peugnet | |
for now using zlib on a 556Mo dir: recipe: 11668558 o -> 5540831 o size divided by 2 | |||
2021-09-20 | remove errored files from fileList | n-peugnet | |
and add tests with non existing link | |||
2021-09-20 | store fileList compressed | n-peugnet | |
for now using zlib on a 556Mo dir: files: 1469447 o -> 207289 o size divided by 7 | |||
2021-09-15 | add perf output | n-peugnet | |
2021-09-15 | add storeWorker and use it to store chunk content and hashes | n-peugnet | |
2021-09-14 | perf: move hash parallelism from sketch to repo | n-peugnet | |
2021-09-14 | logger add SetOutput + basic tests | n-peugnet | |
2021-09-14 | logger add panic + colors & remove ln variants & use it | n-peugnet | |
2021-09-14 | add basic logger | n-peugnet | |
2021-09-13 | first tests on real data | n-peugnet | |
2021-09-13 | fix loadChunks by using cache instead of reading file | n-peugnet | |
This way there is only one place where we read chunks and where the read/write wrapper is used. This also allows to remove LoadedChunk as it is not used anymore. | |||
2021-09-13 | fix english typos | n-peugnet | |
2021-09-13 | add read write wrappers to compress chunks' content | n-peugnet | |
2021-09-10 | use moare the reader interface | n-peugnet | |
2021-09-09 | better commit and restore tests (almost complete) | n-peugnet | |
added some file path management function for prfixes and trailing slash | |||
2021-09-09 | first add of restore function to Repo | n-peugnet | |
2021-09-09 | move unused functions from repo to repo_test | n-peugnet | |
2021-09-08 | start using chunk cache | n-peugnet | |
2021-09-07 | update todo and readme | n-peugnet | |
2021-09-07 | udpate todo list | n-peugnet | |
2021-09-02 | hash and store chunks left by matchStream | n-peugnet | |
2021-09-02 | move maps inside repo | n-peugnet | |
2021-09-02 | find similar chunks while matching stream | n-peugnet | |
2021-09-01 | move some consts into repo | n-peugnet | |
2021-08-31 | join too small temp chunks with previous one if possible | n-peugnet | |
2021-08-30 | better diff test | n-peugnet | |
2021-08-30 | findSimilarChunk returns an Id if found | n-peugnet | |
2021-08-28 | refactor(chunks): use more interfaces | n-peugnet | |
2021-08-24 | refactor: extract chunk.og & add Reader getter | n-peugnet | |