The current instance of our computer game laboratory, a practical course for TUM games engineering students, is ongoing at the moment, you can check out the latest development of the four game projects on our wiki main page.
Look out for the final presentations and playable demos during the SS’18 demo day…
Computer simulations are a powerful method to study physical and engineering systems such as fluids, collections of molecules, or social agents. Traditionally, differential equations such as the Navier-Stokes equations form the basics of performing simulations as they dictate the time evolution. A recent, promising development is to use machine learning for model-free prediction of a systems’ behavior and to identify the appearance of spatio-temporal patterns. Deep learning with neural networks is a particularly interesting and powerful machine learning method that can be employed for this task.
Nils recently gave a talk titled “Deep Learning Beyond Cats and Dogs” at a TEDx event organized at the Technical University of Munich.
Abstract: Deep learning, which is seemingly everywhere these days, is well-known for its capability to recognize cats and dogs in internet images, but it can and should be used for other things too. It can be used to figure out the complicated physics that dictate fluid behavior. Actually, simulating turbulence is not only a million dollar problem (really, google it!) but it can help us create more realistic virtual worlds. It can even help us understand medical and physiological behaviors like blood flowing through our body. Nils performs cutting-edge research and explains how neural networks are well on their way to becoming the fourth pillar of science.
Biography: Nils Thuerey’s work is in the field of computer graphics: he models physical behaviors of fluids such as water and smoke to enable computer created virtual effects to look like the real thing. These phenomena are very expensive to simulate computationally, so Nils’ research explores the use of deep learning methods to generate the effects more quickly and more realistically. Before assuming his assistant professor position at TUM, Nils studied in Erlangen, held a post-doc position in Zurich, and worked in the visual effects industry. He was awarded a technical Oscar for the development of an algorithm which aids in editing explosion and smoke effects for film.
In the following we give an overview of our recent publications on physics-based deep learning methods. In particular, we focus on solving various aspects of fluid problems modeled with the Navier-Stokes (NS) equations. These topics are a central theme of the research work in our group. While, naturally, the long term goal would be to simply give the initial conditions of a problem as input to a neural network, and then rely on the network to infer the solution with appropriate accuracy, the complexity of the NS equations makes this an extremely challenging problem.
Thus, we typically consider constrained solution spaces for sub-problems of the NS equations, that we believe are nonetheless very interesting, and useful for their respective domains. In this way, we are also working towards improving the state of the art in order to tackle more and more general problems in the future.
Our publications have targeted different aspects of a typical simulation pipeline, and differ in terms of how deeply integrated they are into the Navier-Stokes solve. The following list is order from loose to tight couplings. E.g., the last entry completely replaces a regular solver.
Sebastian Barschkis, CS-student at TUM, has just presented his latest progress regarding the integration of our mantaflow solver into Blender. You can check out his full presentation including insights about code structure as well as using the solver here:
There are admittedly still some rough edges, but mantaflow should give Blender users a significant step forward in terms of visual quality and performance.
The code uses our mantaflow framework for the Navier-Stokes simulation part, and Google’s tensorflow framework for the deep learning portion. You can find a short introduction / how-to on the github page above. If you give it a try, let us know how it works!
We just completed the Ferienakademie 2017 course on Neural Networks for Fluid Simulations. In total, 17 participants prepared short presentations about research highlights, and then worked hard on an own implementation of various generative neural networks for two dimensional fluid flow. This course was jointly organized by Nils Thuerey (TUM), Michael Engel (FAU), and Miriam Mehl (Universitaet Stuttgart).
As part of the course, the participants were able to gain first-hand experience with deep learning algorithms, and explore connecting these algorithms with problems from the area of physical simulations. We used the tensorflow framework (https://www.tensorflow.org) for the deep learning part, and our own mantaflow solver (http://www.mantaflow.com) for the Navier-Stokes simulations.
The Ferienakademie (https://www.ferienakademie.de) is a long-established institution at TUM. It takes place every year in the Sarntal in “Alto Adige”, i.e., South Tyrol, and this year actually was its 34st instance. Highly recommended for motivated students that are interested in going beyond the standard curricula of a university, and who have a certain affinity for hiking, of course 🙂
Here’s a photo of this year’s participants – after all the hard work & hiking were done…
You can now find a second mantaflow-tensorflow tutorial on our webpage. It explains the example0 code of mantaflow, which represents an example that’s as simple as possible: a very simple mantaflow scene that generates some flow data, and a simple tensorflow setup that trains a simple neural network with this data.
Despite its simplicity, it contains all the important parts: data wrangling, network training, and result generation. Also, it’ll demonstrate how much you can get out of a fifty-dimensional latent space from a simple NN auto encoder.
Sebastian Barschkis, a TUM computer science student, just successfully finished his Google summer-of-code project, pushing the integration of our fluid solver mantaflow into the open-source animation package blender (https://www.blender.org) a step further. In his project, the main goals were a secondary particle extension (for splash & foam particles of liquids) and the integration of our primal-dual guiding optimization (see the full paper here). Hopefully, that moves us yet another step closer to releasing mantaflow as part of an official blender release, 2.8 hopefully!
More detailed documentation and info can be found on Sebastian’s wiki page:
We’ve just posted a first introduction in a series on how to couple mantaflow and fluid sims with tensorflow and deep learning algorithms.
The latest release (v0.11) of mantaflow comes with a set of data-transfer functions to exchange data between the two frameworks, and provides three examples with varying complexity. The page below gives an introduction to mantaflow-tensorflow coupling, and an overview of the data transfer functions. A more in-depth discussions of the three coupling examples will follow in the next weeks.