In one of the most striking (and funny) passages in his last, unfinished novel, the American writer David Foster Wallace shows us how mathematics (or, at least, "Advanced Tax") can evoke a mystical, religious experience in certain people. Hidden symmetries lurk just beneath the surface of everyday life, and the discovery and exploration of this beautiful, parallel universe is an experience unlike no other in human thought. A new exhibition which has just opened at the Fondation Cartier in Paris – "Mathematics, a beautiful elsewhere" – offers a glimpse of this world and those who labour there.

It cannot have been an easy exhibition to plan. Mathematics, after all, essentially concerns the manipulation of symbols. How to make such a thing resonant and exciting? Rather than avoiding this fact, the exhibition embraces this process in all its mystery and strangeness. The ground level of the Fondation Cartier is divided into two areas designed by filmmaker and artist David Lynch in collaboration with the mathematician Micha Gromov: "The Library of Mysteries" and "The Room of Four Mysteries".

In the latter room, against a sonic background of Lynch's trademark sinister rumblings, we are presented with (amongst other wonders) the ghostly hand of a physicist tracing glowing lines on the wall: "Feynman diagrams", which are a symbolic representation of how the universe works on the smallest scales. These images are intercut with live video feeds from the CMS and ATLAS experiments at the Large Hadron Collider in CERN, Geneva (and in the months to come, images from the Planck Satellite, launched by the European Space Agency in 2009). Here, in these experiments, is the mathematical word made flesh.

In the second room, a library in the shape of a zero airs film made by Lynch. This features a succession of texts drawn from books selected by Gromov, each of which falls from the sky with a resounding and satisfying crash. These words attempt to show the entire evolution of human mathematical thought and its delicate and uncertain relationship with the physical world. Above, in the ceiling in the heart of the zero, the heavens turn, and a succession of animations show us the scale and size of the universe from the subatomic froth deep inside quarks to the largest structures observed by astronomers. Periodically this image fades, and the singer Patti Smith reminds us through texts from Gromov set to song that the wonders of the universe can be interpreted on levels other than the purely mathematical.

In the basement, renowned French documentarists Raymond Depardon and Claudine Nougaret offer us a series of short films in which mathematicians attempt to explain their art and motivation. In the process, they touch on some of the profound truths at the heart of the enterprise that is mathematics; once the scaffolding of human thought is removed, these beautiful edifices seem to stand by themselves, and exist in and of themselves, without us. Were they always there? Often we hear how strange it is that the physical world can be described by mathematics (and indeed this is a strange fact) but it turns out there are also vast uncharted mathematical territories entirely apart from the world we inhabit. Some part of this may possibly be of use to particle physicists and astronomers attempting to understand the nature of the physical world – but then again, much of it may not. We are reminded of Hermann Weyl's remark that if forced, he would choose beauty over truth. Of course, a scientist concerned with interpreting events from particle collisions at CERN or photons arriving from the Planck satellite might add a further criteria: calculability. It is not always possible to construct a bridge between the world of mathematics and the physical world around us.

In the last room of the exhibition, a gleaming shape stands alone in an empty space, a "pseudo-sphere" which tapers to an infinitely fine point. This is one of Hiroshi Sugimoto's "Mathematical shapes"; an equation brought into this world under the control of a computer-controlled milling machine. With a precision of a fraction of a millimetre or so, this is as close to perfection as our technology currently allows – a representation of the ideal shapes dreamt of for millennia by philosophers. Beauty may not always be the best signpost to follow in the search for an understanding of the physical world; as an astronomer, I am reminded of Kepler and his vain attempt to understand the motion of the planets in terms of perfect solids. But in traversing the uncharted territories of mathematics, it is probably the best guide there is.

Related information:

The exhibition "Mathematics, a beautiful elsewhere" opened 21 October 2011 and will run until 18 March 2012 at the Fondation Cartier, Paris.