Ring Buffer Architecture (8 slots shown) for Parallel Compression
Slot 0
Empty
Slot 1
Writing...
Slot 2
Compressed
Slot 3
Compressing...
Slot 4
Uncompressed
Slot 5
Adding block...
Slot 6
Empty
Slot 7
Empty
NextToWrite=1
NextToCompress=4
NextToEmit=5
Worker Thread 2
Currently writing
Can also compress
Worker Thread 1
Currently compressing
Can also write
Emit Thread
Generates uncompressed blocks
Can help with compression
SST File
Invariant:
NextToWrite ≤ NextToCompress ≤ NextToEmit (modulo ring buffer size)