From Newton to Neural Networks: Why Hallucinations Remain Unsolvable
Last Updated on April 29, 2025 by Editorial Team
Author(s): MKWriteshere
Originally published on Towards AI.
The Mathematical Paradox at the Heart of AI’s Greatest Challenge
When you marvel at a large language model’s capability, remember it rides on centuries-old mathematics.
Newton’s discovery of the derivative laid the groundwork for backpropagation; the same principle guides every weight adjustment in a neural network today.
By pairing his candlelit study with a glowing AI “brain,” we reveal that hallucinations aren’t a modern bug but an echo of this foundational technique.
These AI systems, designed to process and generate human-like text with remarkable fluency, are increasingly “hoist with their own petard” — undone by the very mechanisms that make them powerful.
As OpenAI finds itself puzzled by rising hallucination rates in newer models (o3 and o4 ), we’re witnessing what many AI skeptics have long predicted: a fundamental limitation that may be inherent to transformer architecture itself.
The connection between Newton’s calculus and modern AI is more than just historical trivia — it’s the key to understanding why hallucinations persist as an unsolvable problem.
Neural networks fundamentally rely on optimization techniques that trace back to Newton’s work on derivatives. Backpropagation, the algorithm that powers learning in these systems, is essentially the application of the chain rule of calculus to adjust weights and minimize error.
This mathematical lineage reveals something profound: the… Read the full blog for free on Medium.
Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.
Published via Towards AI
Towards AI Academy
We Build Enterprise-Grade AI. We'll Teach You to Master It Too.
15 engineers. 100,000+ students. Towards AI Academy teaches what actually survives production.
Start free — no commitment:
→ 6-Day Agentic AI Engineering Email Guide — one practical lesson per day
→ Agents Architecture Cheatsheet — 3 years of architecture decisions in 6 pages
Our courses:
→ AI Engineering Certification — 90+ lessons from project selection to deployed product. The most comprehensive practical LLM course out there.
→ Agent Engineering Course — Hands on with production agent architectures, memory, routing, and eval frameworks — built from real enterprise engagements.
→ AI for Work — Understand, evaluate, and apply AI for complex work tasks.
Note: Article content contains the views of the contributing authors and not Towards AI.
Related posts
Recent Posts
