An article examining the mathematical equivalence between compression and prediction in information theory. The author argues that while the correspondence is valid, compression involves decisions about object representation, probability models, and decoder capabilities that exist outside the prediction-compression equivalence.
This article explains how to factorize the probability of a language model trajectory using the chain rule, applying assumptions about initial states, environment dynamics, and policy behavior. Starting from basic probability theory, it derives that trajectory probability decomposes into initial state distribution, policy action probabilities, and environment transition probabilities.
Simon Tatham analysed the 2048 puzzle game on a 3×3 board instead of the standard 4×4 grid to create a shorter version. He exhaustively computed all possible game states to determine optimal play and what target tile value would be reasonable for players to achieve, leveraging the small state space and monotonic nature of the game.
A 2018 physics paper argues that the Fermi paradox—the puzzle of why we observe no intelligent alien life despite mathematical models suggesting it should be common—dissolves when accounting for realistic uncertainties in parameters like the Drake equation. The authors show that when uncertainty distributions are properly modeled rather than assumed certain, there is substantial probability of no other intelligent life in the observable universe, eliminating the paradox.
Bayesian OCR methods can use document context like language to improve character recognition accuracy, but this approach has limitations. When applied at scale to large databases like Google's Ngram, contextual priors can introduce systematic errors—for example, incorrectly recognizing ambiguous characters as English words in English texts, as happened with the word 'grok' appearing before its 1961 coinage.