A repository achieved 99.26% storage reduction while maintaining exact reconstruction across 100 versions. The post discusses technical implementation details related to version control optimization.
A developer created a binary schema language to compress JSON payloads by 80%, explaining how binary packing works by encoding data types and lengths with continuation bits to efficiently represent strings and integers in fewer bytes than ASCII representations.
A developer explores binary packing as an alternative to JSON, creating a custom binary schema language that reduces data payload sizes by 80%. The article explains fundamental concepts of binary encoding, type headers, variable-length integers, and continuation bits for efficient data serialization.
Glyd is a lossless AI compression technique that stores open-source LLM weights in 11 bits instead of 16, reducing GPU memory usage by 33% while maintaining bit-for-bit accuracy. The method decodes weights directly in GPU matrix operations without rounding, enabling the same models to run on less hardware, often faster, with negligible impact on model outputs.
An exploration of using gzip compression as a language model for text generation. The author demonstrates that gzip can generate text through beam search decoding, showing that compression and prediction are equivalent tasks. While gzip produces repetitive results compared to neural language models, it reveals interesting structural patterns when given proper search horizons.
Astra, a strong AI model, won a Kolmogorov Audio Compression challenge by reverse-engineering the synthesis code rather than creating a traditional compression algorithm, reducing 600MB of audio to 20KB.
DataHaskell/Dataframe implemented a Parquet writer for Haskell that enables efficient serialization and interoperability with the data science ecosystem. The writeParquet function provides simple usage with defaults, while writeParquetWithOptions allows fine-grained control over row group and page sizes.
Denizlihub is an all-in-one web toolkit offering PDF conversion, image editing, AI OCR, JSON formatting, and developer utilities that process files entirely in the browser without server uploads. The platform features popular tools for compressing PDFs, converting between document formats, removing backgrounds, generating QR codes, and various other productivity tasks.
Compaction.dev is a tool that reduces input and output tokens for Claude Code, Codex, and Cursor by compressing model-visible content locally before sending to providers and shaping output during generation. It operates within existing editors without requiring new tools, achieving ~14% input reduction and ~25% output reduction in demonstrated runs, with free and paid account options for different compression features.
LensVLM is an inference framework that enables Vision Language Models to process compressed images of text by selectively expanding relevant regions, maintaining accuracy at 4.3x compression while outperforming baselines up to 10.1x compression across text QA benchmarks. The approach combines learned tools for selective expansion with post-training to make visual compression robust, generalizing to multimodal document and code understanding tasks.
LensVLM is a 9B Vision Language Model that compresses long text documents into images, then selectively expands only relevant pages to answer queries. The model uses learned tools to decompress specific sections, supporting compression ratios up to 15x while maintaining question-answering capabilities.
Apple's LensVLM-9B is a Vision-Language Model framework that maintains text recognition accuracy in compressed images by selectively expanding relevant regions using learned tools, achieving 4.3x compression while matching full-text performance on text QA benchmarks.
Microsoft Edge's security team replaced the C implementation of Brotli compression in its network stack with a Rust implementation to improve memory safety. Brotli was chosen as the next rustification target because it processes untrusted network data at a critical point in the browser, has a mature Rust crate with production adoption, and is widely deployed on the modern web.
Ext-Turbovec is a PHP 8.3+ extension for in-process vector indexing and approximate nearest-neighbor search using quantized indexes. It implements Google Research's TurboQuant algorithm, offering 2–4 bit compression with SIMD search kernels that outperform FAISS on most configurations, enabling vector search within PHP workers without external databases or data leaving the machine.
Avhash is a BlurHash alternative that uses stripped AV1 frames as tiny image placeholders. It encodes image data more efficiently than BlurHash by removing container and header bytes, then regenerates them during decoding for browser-native AVIF rendering. The format supports quality tuning and uses base-88 encoding for HTML-safe attribute storage.
Researchers reformulate LLM block pruning as a constrained binary optimization problem mapped to an Ising glass spin system, enabling efficient ranking of pruned configurations without benchmarking each candidate. Unlike mean-field methods that treat blocks independently, this approach accounts for pairwise couplings between blocks, achieving 23 percentage points improvement over competing methods at 50% compression of Llama-3.3-70B-Instruct on MMLU.
ArrowSpace successfully processed 100,000-dimensional biomarker data from the Dorothea dataset, scaling to 800 samples in 41-110 seconds with over 600× memory compression. The system achieved 5-22% graph density through parameter sweeps, with TauMode computation being the critical bottleneck at 62% of runtime. Johnson-Lindenstrauss projection and k-NN connectivity emerged as key factors controlling build time and graph density respectively.
Grant Sanderson's concept of 'compression is intelligence' frames intelligence as the ability to distill fundamental principles from the real world and apply them broadly. The article parallels this with machine learning, where models must be prevented from memorizing specific patterns (overfitting) and instead learn to generalize, similar to how truly intelligent people grasp abstract concepts rather than memorizing facts.
SMPTE 2110 is a suite of standards from the Society of Motion Picture and Television Engineers that describes how to send digital media over IP networks for broadcast production and distribution. The standards, first published in 2017 with updates through 2022, define separate specifications for transporting video, audio, ancillary data, and metadata while maintaining synchronization through protocols like RTP and PTP. SMPTE 2110 prioritizes quality and flexibility over bandwidth efficiency in professional broadcast environments.
A researcher explores whether gzip, a standard compression utility, can perform language modeling by leveraging the mathematical equivalence between compression and prediction. Using beam search over byte sequences scored by gzip's compression length, they demonstrate that gzip can generate text continuations that show understanding of source material, despite producing imperfect output.