Border bases are a generalization of Gröbner bases for polynomial rings. In this article, we introduce border bases for a non-commutative ring of linear differential operators, namely the rational Weyl algebra. We elaborate on their properties and present algorithms to compute with them. We apply this theory to represent integrable connections as cyclic D-modules explicitly. As an application, we visit differential equations behind a stringy, a Feynman as well as a cosmological integral. We also...