A recent study from researchers at Google Research and Technion uncovers a significant insight about frontier models like GPT-5 and Gemini-3: these large language models (LLMs) can recover up to 65% of facts they initially struggle to recall, simply by optimizing their inference time.
Traditionally, when LLMs produce hallucinations—incorrect or nonsensical outputs—developers assume the model lacks the necessary facts. This leads to common engineering responses such as increasing model size or enhancing training data. However, the latest findings suggest that the models often have the information encoded but fail to retrieve it effectively during generation.
What It Is
The study demonstrates that frontier models encode between 95-98% of tested facts. The research shifts the focus from question-level accuracy to a more nuanced approach called fact-level profiling, which evaluates whether a fact is truly stored in the model's parameters and how successfully it can be retrieved across various conditions.
Why It Matters
This breakthrough is crucial for startup founders working with AI applications. By understanding that recall, rather than encoding, is frequently the bottleneck for factual accuracy, engineering teams can improve existing models without the substantial costs associated with scaling up model size or integrating complex retrieval systems. It allows for more efficient use of AI resources and leads to better performance in applications relying on accurate information retrieval.
Key Features
- Fact-Level Profiling: A new evaluation method that tests how well facts can be retrieved under different conditions, rather than just scoring isolated prompts.
- Enhanced Recall: Frontier models can recover a significant number of facts they initially cannot recall, improving the reliability of AI applications.
- Cost-Efficiency: This approach reduces the need for larger models or complex data architectures, making AI more accessible for startups.
Founder Takeaway
For founders looking to leverage AI, focus on optimizing your model's inference processes to enhance recall capabilities. This can lead to more reliable outputs without incurring the costs of scaling your AI infrastructure.