There is an interesting new field developing at the intersection of the physical sciences and deep learning, sometimes called AI4Science. In one direction, tools developed in the AI community are used to solve problems in science, such as protein folding, molecular simulation, and so on. But also in the other direction, deep insights from mathematics and physics are inspiring new DL architectures, such as Neural ODE solvers, and equivariance. In this talk, I will start with mapping out some of the opportunities at this intersection and subsequently dive a little deeper into PDE solving. Also, in this subfield, cross fertilization has already happened both ways: people have used DL tools to successfully solve PDE equations much faster than with traditional solvers. But reversely, there are also efforts to use PDEs as “infinite width”, functional representations of layers in a deep NN architecture. The latter is helpful for instance to become independent of gridding choices. In the second half of this talk, I will explain our most recent efforts to solve PDEs faster and more accurately using DL, and reversely, new ways to use PDEs as an approximate equivariance prior.