Slashdot Mirror
Developer Shares A Recoverable Container Format That's File System Agnostic (github.com)
Long-time Slashdot reader MarcoPon writes: I created a thing: SeqBox. It's an archive/container format (and corresponding suite of tools) with some interesting and unique features. Basically an SBX file is composed of a series of sector-sized blocks with a small header with a recognizable signature, integrity check, info about the file they belong to, and a sequence number. The results of this encoding is the ability to recover an SBX container even if the file system is corrupted, completely lost or just unknown, no matter how much the file is fragmented.
The default block size used is 512 bytes, which is a suitable sub-multiple of every sector size used by most system after the CP/M days. One example of system that doesn't plays well with is Amiga Old File System (which use 488 bytes per blocks, IIRC). Actually it's the only FS/platform that I found not working, among the ones I managed to test (a bit over 20, I listed them in the readme, just above the tech spec).
SeqBox
So the only failure mode this protects from is corruption of metadata while every data block remains intact. On any sane filesystem, that sounds useless: the only cases this might happen are filesystems that can't handle unclean shutdown (FAT, ext2) or the disk lies about barriers. And those cameras that still use FAT have software you can't update, so you can't install that SBX thingy -- if you could, you'd be better off switching to a better filesystem.
In its present state, I'd suggest you scrap the whole project, it's a waste of time.
On the other hand, it would be an entirely different story if you added some form of erasure code that operates on amounts of data bigger than a single sector (most storage devices already have per-sector erasure codes).
The creatures outside looked from Alt-Right to Antifa; but already it was impossible to say which was which.
Unlike HDD controllers, SSD controller do wear-leveling, so there is no guarantee that your data will be written as as a contiguous block of memory (regardless of what the filesystem says), only that it will be in 4096 byte blocks. Recovering deleted data from a SSD is no simple task because it means you need to know or guess the controller behavior for wear-leveling in order to go back and find the order of previously written data. With this you would be able to just read the raw memory even after the controller has been reset and still be able to recover the data. I think it would be a nice option to have a filesystem be able to encode user files in something like this highly recoverable format. The only real problem is that the file has to be completely rewritten even if you only modify part it in order to differentiate the new version from the old version.
Anons need not reply. Questions end with a question mark.
That one yes, but not the others already saved. A fragmented JPEG instead is pretty difficult to recover if the file system is inconsistent. The usual recover tools would easily find the first fragment, and then proceed from there collecting sectors in sequence, which may or may not contain the right data.
SeqBox
They are two entirely different thing, but they could "help each other" in some ways.
PAR (or a RAR archive + recovery records, etc.) try to address the problem of losing some small parts of a file (due for example to physical errors), using some amount of redundancy. SeqBox try to address the issue of identifying and reassembling all part of a file, when they are all still on the physical media, but without the file systems indexes / structures to locate them (es. after a quick format, zero writes on the first sectors, etc.).
If you combine the two, creating an SBX container of a RAR + recovery records for example, you get both qualities.
SeqBox