[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Cryptography] Hierarchical bulk digest modes



You have basically described how the ZFS filesystem works for error detection and correction. 
It builds a Merkel hash tree of the per block hashes, stored in block pointers containing metadata (compression, encryption, hash algorithm ids, transaction birth id and information about the (upto) 3 copies of the block).
In addition to RAID it also stores copies (always a minimum 2 copies of metadata).
Block sizes are variable with 512 as the smallest and 1MB as the largest.
The hashes are also able to be used for inline deduplication.
When you have multiple copies you can not only detect but correct corruption by reading from an alternative copy and then issuing a repairing write.
ZFS is also copy on write, so it never overwrites live data.

When I added encryption to ZFS (in Solaris) I used AES in CCM or GCM. The IV and tag are stored in the blockpointer. 


Darren J Moffat
_______________________________________________
The cryptography mailing list
cryptography AT metzdowd.com
https://www.metzdowd.com/mailman/listinfo/cryptography