Foundation Models for Scientific Computing

Foundation Models for Scientific Computing: Architectures, Training, Neural Surrogates, and Trustworthy Deployment at Scale (First Edition, 2026) by Nanda Kishore Kande is a graduate textbook about the meeting of two traditions that long developed apart. Numerical analysis starts from governing equations, discretises them, and proves convergence. Machine learning starts from data, fits flexible functions, and judges success on held-out samples. The book argues that large pretrained "scientific foundation models" have ended this separation and changed the economics of what is worth computing.

Its central thesis is that foundation models are most useful not when they replace numerical methods, but when they are embedded inside verified workflows. The author describes four modes of use (surrogate, prior, proposal, and agent), each with a different verification burden. The learned component supplies speed or breadth, and the classical component supplies the guarantee. The book makes the point that a surrogate that is right 95% of the time with a reliable signal of which cases to check beats one that is right 99% of the time but cannot be verified.

Chapters 1 and 2 establish the problem and the mathematics. They cover function spaces and operators, discretisation cost, neural networks as approximators, optimisation, spectral bias, the neural tangent kernel, and automatic differentiation. A key idea is that the object worth learning is the solution operator itself, which enables resolution-independent models.

Chapters 3 to 5 cover architectures, data, and training at scale. Architecture is framed as converting prior knowledge into constraints, with comparisons of convolutions, attention, spectral layers, graph networks, and equivariant designs. The data chapter warns that scientific corpora are narrow, that their effective sample size is often far smaller than it appears, and that every source carries its own bias. The training chapter covers scaling laws, memory arithmetic, parallelism, instabilities, and rollout-aware training for time-dependent systems.

Chapters 6 and 7 form the scientific core. Chapter 6 explains neural operators such as the Fourier Neural Operator and DeepONet. It also covers the strengths and limits of physics-informed learning, and hybrid schemes like learned closures and solver-in-the-loop training, which the author considers the most robust designs. Chapter 7 surveys results in weather, fluid dynamics, molecules and materials, structural biology, astronomy, and plasma control. It concludes that success comes from a curated corpus, identifiable physical structure, cheap verification, and a costly or absent classical alternative.

Chapters 8 and 9 address trust. Chapter 8 examines language models, retrieval, tool use, agentic loops, and autonomous laboratories. It insists that generation be separated from verification and that a model never check its own work. Chapter 9 covers aleatoric and epistemic uncertainty, deep ensembles, conformal prediction, calibration, shift detection, and validation protocols that would survive peer review.

Chapter 10 turns to systems: inference arithmetic, quantisation and distillation, serving, energy and carbon accounting, governance, and six open problems. Appendices add notation, a software stack, and a reproducibility checklist.

The book is written for graduate students, computational scientists, and engineers who are comfortable with linear algebra, calculus, probability, and Python. It builds deep learning from first principles and includes exercises that ask readers to break methods and find where they fail. Its lasting message is composition: learned speed inside classical guarantees.