A hobbyist designed a TPU and optimized transformer inference on a $100 Cmod A7 FPGA board, implementing a small transformer model to predict the next token in addition sequences. The project analyzes ML performance through compute and memory communication bottlenecks, using matrix multiplications as the core workload with simplified operations like ReLU activation for hardware feasibility.