New research has exposed a potential flaw in the memory-saving technology underpinning DeepSeek’s models, marking a rare challenge to a breakthrough that has helped the Chinese artificial intelligence lab close the gap with US rivals OpenAI and Anthropic with its low-cost approach. The findings were published late last month in a paper led by researchers from ByteDance and co-authors affiliated with Princeton University, Stanford University and the University of California, Berkeley. The study, which has not been peer-reviewed, said that DeepSeek’s efforts to reduce the memory needed to process lengthy inputs may come at the expense of reliability, as the models were more likely to miss the same information when it appeared in certain positions in a document. “The phenomenon appears in DeepSeek-V4, V4.1, and all compressed models we pretrained,” the paper said. The researchers tested several DeepSeek models on their ability to retrieve information from lengthy contexts of about 128,000 tokens – equivalent to roughly tens of thousands of words. They found that the retrieval accuracy of the flagship DeepSeek-V4 family of models can differ by up to 40 percentage points. They attributed the fluctuations to how DeepSeek groups and compresses information before storing it for later retrieval. While the technique saves memory, it can also make certain details harder to recover depending on where they fall within the compressed segments. In one example, adding irrelevant text before the same piece of computer code impaired DeepSeek’s ability to choose the correct answer, illustrating how seemingly minor changes to an input could affect its performance. The researchers cautioned that their findings were based on controlled experiments and a limited number of coding examples, rather than a comprehensive assessment of how frequently the problem affects real-world applications. DeepSeek and ByteDance did not respond to requests for comment. DeepSeek has made cost-efficiency a central selling point of its technology, an approach that helped propel the Hangzhou-based company to international prominence as it challenged assumptions about the enormous computing resources required to develop competitive AI systems. Such improvements are particularly significant for Chinese AI developers, which face restrictions on access to advanced US semiconductors and growing pressure to deliver competitive performance with more limited computing resources. But as developers compete to make their models both more capable and less expensive to operate, the findings highlight a challenge in balancing efficiency with reliability, particularly for applications requiring accurate retrieval of information from lengthy documents or software code. Additional reporting by Ben Jiang