A developer demonstrates parsing compressed NDJSON files at 40 GB/s using zstd compression with multi-frame support and the simdjson library. By enabling independent decompression of multiple frames across threads, zstd and lz4 achieve 16x faster throughput than gzip while maintaining similar file sizes.
A developer achieved 40 GB/s parsing speed for compressed NDJSON files using zstd with multi-frame architecture and the simdjson library, processing 5 million records on a 64-core server. By enabling parallel decompression and parsing across independent frames, zstd and lz4 outperformed single-threaded gzip by 16x while maintaining similar compression ratios.
A developer demonstrated parsing compressed NDJSON files at 40 GB/s using zstd compression with multi-frame formatting and the simdjson library. By allowing parallel decompression and parsing across multiple threads, zstd and lz4 formats achieved 16x faster throughput than gzip while maintaining similar file sizes.
A developer demonstrates parsing compressed NDJSON files at 40 GB/s using multi-threaded decompression with zstd format. By writing compressed files as independent frames rather than single streams, multiple threads can decompress and parse simultaneously, achieving 16x speedup over gzip while keeping file sizes comparable.
SIMDjson 5.0 has been released with official support for static reflection, improved handling of large integers, and various performance optimizations. The new version shows measurable speed improvements over SIMDjson 4.6.4, continuing the library's track record of advancing JSON parsing performance.