The term "machine learning for science" encompasses at least two fundamentally distinct approaches that are often conflated in both academic literature and industrial practice. The first — conventional deep learning applied to scientific data — imports the architectural and optimisation innovations of data-driven AI into scientific domains, treating physical systems as high-dimensional pattern-matching problems. The second — Scientific Machine Learning (SciML) — begins from a different premise: that the laws governing physical systems carry structural constraints that should be respected, not merely approximated.
Understanding this distinction has direct consequences for model selection, validation, and the interpretability of results.
Every learning system embeds inductive biases — assumptions about the structure of the function being approximated. In deep learning, these biases are primarily architectural: locality via convolutional layers, sequential dependencies via recurrent or attention mechanisms, hierarchical composition via depth. These biases are well-suited to the statistical structure of natural language and image data but do not reflect the symmetries, conservation laws, and differential constraints that characterise physical systems.
SciML methods incorporate domain knowledge explicitly, either by constraining the hypothesis space to physically admissible functions (as in Physics-Informed Neural Networks) or by searching directly for symbolic expressions consistent with observed data (as in symbolic regression).
A critical difference between the two paradigms emerges under distributional shift — when the deployment distribution differs from the training distribution. Deep learning models, particularly those trained on large datasets, exhibit strong interpolation within the training distribution but can fail catastrophically when evaluated on inputs that lie outside that distribution.
Physical models, by contrast, generalise by construction to any configuration consistent with their governing equations. A symbolic expression recovered by SR encodes not a statistical correlation but a structural relationship, and structural relationships hold whenever the underlying physics holds — regardless of whether that regime was sampled during training.
In many scientific applications — prediction of novel drug candidates, analysis of previously unexplored parameter regimes, forecasting under novel market conditions — the target population lies outside the training distribution by definition. For these applications, statistical interpolation is insufficient; structural generalisation is required.
Physics-Informed Neural Networks (PINNs) occupy an interesting intermediate position. They retain the parametric architecture of deep networks but augment the training objective with residual terms derived from known governing equations. For a system described by a partial differential equation:
a PINN minimises a compound loss that penalises both data misfit and violation of the PDE residual. This constrains the learned function to lie in the solution manifold of the governing equations, improving generalisation and reducing the data requirement relative to unconstrained networks.
However, PINNs require the governing equations to be known a priori. When the governing equations are themselves the object of discovery, symbolic regression methods such as SINDy or PySR are more appropriate.
The system exhibits complex, high-dimensional patterns for which no compact physical model is known; large labelled datasets are available; and the primary objective is prediction within a well-defined distribution.
Interpretability is a scientific or regulatory requirement; the system obeys known physical constraints; data are scarce or expensive; generalisation to novel regimes is required; or the objective is equation discovery rather than pure prediction.
Deep learning and Scientific Machine Learning are not competitors but tools suited to different classes of problems. The choice between them is not a matter of novelty or computational fashion, but of epistemic alignment — matching the structure of the method to the structure of the problem. For scientific domains where interpretability, physical consistency, and extrapolation are essential, SciML methods provide capabilities that standard deep learning architectures fundamentally cannot.