In one sense, of course, theres less mystery in math than there is in any other human endeavor. In math we can really understand things, in a deeper way than we ever understand anything else. (When I was younger, I used to reassure myself during suspense movies by silently reciting the proof of some theorem: here, at least, was a certainty that the movie couldnt touch.) So how is it that many people, including mathematicians, do feel theres something mysterious about this least mysterious of subjects? What do they mean?
There are certainly mysteries within math. For starters, there are the thousands of unsolved problems, assertions that no one can prove or disprove, sometimes despite decades or centuries of effort. While many of these problems are deep and important, a small example will do for now: no one has proved that, as you go further out in the decimal digits of =3.141592653589, the digits 0 through 9 occur with equal frequency.
Yet, for reasons that apply to many other unsolved problems, its debatable whether to call this a mystery. What would really be mysterious, one wants to say, would be if the digits didnt occur with equal frequency! The whole challenge is to give an airtight proof that what does happen is what anyone with any common sense, after thinking the matter over a bit, would conclude almost certainly must happen. As Jordan Ellenberg wrote, a dirty secret is that many unsolved math problems have a similar flavor: theyre less about mysterious coincidences than about the lack of them.
Take, for example, the Twin Primes Conjecture, that there are infinitely many pairs of prime numbers separated by 2 (like 3 and 5 or 11 and 13). For reasons Ellenberg explains, for this to be true, there doesnt need to be any mysterious force pulling primes together: there just needs not to be a mysterious force pushing them apart. Or take the Riemann Hypothesis, which says that the infinitely many nontrivial roots of a certain complex function all lie on a single line. When its stated that way, it does sound like a mystery: why should infinitely many numbers all happen to line up like that? But the mystery lessens once you realize that each zero of this function encodes global information about the distribution of prime numbersand a single zero off the line would represent infinitely many prime numbers clumping together in astronomically improbable-seeming ways. So, if you like, one mysterious pattern has to be there to prevent a second pattern that would be even more mysterious.
Granted, not all mathematical mysteries have the character of rigorously proving what common sense would predict. In 1978, John McKay noticed that the number 196,883 showed up in two completely unrelated-seeming parts of math. Surely it was just a coincidence? In 1998, Richard Borcherds won the Fields Medal largely for proving that it wasnt (following an inspired guess by John Conway and Simon Norton, which they called the monstrous moonshine conjecture).
Math, you might say, is a conspiracy theorists dream: its the one part of life where, when you see things match up, the odds are excellent that its not just a coincidence, that there is a deep explanation waiting to be unearthed. On the other hand, precisely because the entire subject is shot through with non-coincidental patterns, once youve spent enough time doing math you might stop being so surprised by them; you might come to see them as just part of the terrain.
So maybe the right question is: after a mathematical pattern has been explainednot only proved, but, lets say, proved in 20 different ways, really exhaustively understood, like the Pythagorean Theoremis there still a residual mystery about it? I would say that there is, but it takes some effort to put our finger on it.
Last year, the string theorist and notorious conservative blogger Lubo Motl accused theoretical computer scientists like me of believing the PNP conjecturea central unproved hypothesis about the limits of efficient computationas a matter of groupthink and ideology, of having no rational grounds for our prejudice. By itself, that isnt so remarkable; Lubo certainly isnt alone in that opinion. But Lubo went further: while he conceded that, in continuous math close to physics, there can be reasons why statements are true, he claimed that, as you get further away from physics, math becomes just a disorganized mess of propositions. There are statements that happen to have been proved and that we can therefore agree are true, as we might agree with our accountant that 532+193=725. But if a statement hasnt been proved or disproved yet, then theres no way even to guess, better than chance, which way it will turn out. There are no valid analogies to any previously-proven statements, no broad patterns, just one damn lemma after another.
To put it mildly, this hasnt been my experience, or the experience of anyone else I know who works in any part of math. Yes, sometimes people are surprised; surprises are part of the thrill. But the surprises only are surprises because of their rarity, because of all the other times when things worked out pretty much the way the experts expected them to.
And yet the rarity of surprises is itself a surprise. A priori, math could have been like Lubo Motl said it was, with the statements we care about lacking any humanly-comprehensible reasons for being true or false. But by and large, it isnt that way. Why not?
We can put the point even more sharply. Kurt Gdel famously taught us that, given any mathematical question that hasnt been answered yetexcept for questions that boil down to a finite calculation, like whether White has a win in chessits possible that the answer is just unprovable from the usual axioms of mathematics. And yet, 85 years after Gdel uncovered this gremlin in the center of mathematics, the fact is that its remained mostly dormantrearing its head only for questions about axiom systems that you would only run into if you were looking for unprovable truths; or questions in transfinite set theory that one could argue never needed to have definite answers anyway; or questions that ask whether a particular string of 0s and 1s is patternless (but that, for that very reason, have no general interest, unless we care for some reason about this patternless string); or questions that involve super-rapidly-growing functions.
Again, it didnt need to be that way (well, maybe it did, but its not obvious why). A priori, Fermats Last Theorem, the Poincar Conjecture, and pretty much every other statement of mathematical interest could have been neither provable nor disprovable: if true, then totally disconnected from all the other interesting truths, an island onto itself, with the only question (a question of taste!) being whether we should add it on as a new axiom. But it didnt turn out like that. Instead of millions of islands, mathematicians discovered a supercontinent, with just a few islands here and there off the coastand many of the islands, when explored further, ended up being connected to the mainland after all.
Why? One possible answer is a selection effect: sure, there are plenty of patternless parts of math, but for precisely that reason, those parts arent interesting to humans. The parts that we teach students, put in textbooks, pontificate about in essays like this one, etc. are the parts that ended up being interconnected and elegant. (Likewise, no one wonders why the subjects of biopics so often turn out to have lived riveting lives: if they didnt, there wouldnt be biopics about them.) This strikes me as clearly part of the answer. But it cant be the whole answer, because it doesnt account for something all mathematicians have experienced: namely, the frequency with which there turn out to be striking patterns and connections between seemingly-unrelated concepts, even when no one had thought to expect them beforehand, even when no one had charted out the territory and assured the latecomers that such patterns were there to be found.
A second possible answer is that even the parts of math that look far removed from physics, are indirectly inspired by our experience with the physical worldand that theyre coherent because the physical world is. This answer would push the mystery of maths comprehensibility and elegance back to a different mystery, what Eugene Wigner called the unreasonable effectiveness of math in the physical sciences. A third sort of answer might focus on the peculiarities of the human brain, on its ability (but why?) to zero in on mathematical questions that will turn out to be answerable and concepts that will turn out to be interestingly interrelated, even when it has no idea that its doing so.
I dont know which of these answers is closest to the truth, or whether its something else entirely. But I feel confident in saying that, yes, there is something mysterious about math, and the main thing thats mysterious is why there isnt even more mystery than there is.