Ever wondered if a Transformer model could successively refine a PDE solution? 🤔 Mario’s latest work as a post-doc at TUM shows a way forward: a single model auto-regressively infers and refines flow solutions over finer and finer sets of sample points. Preprint of this “SAR” approach and code are already online https://arxiv.org/pdf/2604.11403 and https://github.com/tum-pbs/SAR
In more detail: Understanding time-dependent fluid flows often requires knowing not just a single solution, but the full range of possible states over time. However, traditional PDE solvers are very expensive to run, and learned surrogate models that predict the next time step tend to accumulate errors when applied repeatedly over long time horizons.
Generative models offer an alternative by directly sampling flow states without relying on sequential time stepping, thereby avoiding error accumulation. However, existing approaches such as diffusion or flow-matching models remain computationally costly, as they require many evaluations across the entire spatial domain.
Scale-Autoregressive Modeling (SAR) addresses this limitation by generating flow fields in a hierarchical, coarse-to-fine manner. It first predicts a low-resolution version of the flow and then progressively refines it to higher resolutions, conditioning each step on the previous, coarser prediction. This strategy improves efficiency by focusing most of the computational effort on coarse scales—where uncertainty is highest. Importantly, it requires fewer denoising steps at finer scales.
Across unsteady-flow benchmarks of varying complexity, SAR attains substantially lower distributional error and higher per-sample accuracy than state-of-the-art diffusion models based on multi-scale GNNs, while matching or surpassing a flow-matching transformer, yet running 2-7x faster than it depending on the task. Overall, SAR provides a practical tool for fast and accurate estimation of statistical flow quantities (e.g., turbulent kinetic energy and two-point correlations) in real-world settings.
