On the intractability of free will

[Author's note: the thoughts here originated as a Letter to the Editors at New Scientist, in response to a somewhat throwaway remark, in an article about randomness, that chance may be essential to the existence of free will. I felt that the point deserved delving into in more depth. New Scientist did publish the letter, but as is typical edited it down for publication -- in particular, many of my adverbs did not survive. Consequently, I wanted to share the full text here.]

Randomness may be necessary to "admit free will" in an otherwise-mechanical universe ("Chance", New Scientist 14 March 2015, p.28ff) but by itself it is not sufficient. It's hard to argue scientifically about the existence of free will in the absence of a rigorous definition of what it is, but we can say something about what it does. And at a minimum, it's existence requires that the outputs of my brain -- my actions -- are not completely determined by the inputs plus initial state. This is, of course, an astonishing proposition that is contrary to any other known physical system or law. Even if we introduce randomness, we merely allow a range of outcomes distributed probabilistically, but we still have no element of intentionality or purpose, the other essential ingredient in free will. Randomness alone would make us no more free than tumbling dice.

One intriguing possibility, however, is that whatever free will actually is, randomness provides a means for it to influence the brain without apparently violating known physical law; a curtain behind which it can hide. Imagine that free will is able to influence apparently-random outcomes deep in the brain, to achieve a desired output, but is also constrained by the need to appear random over the long term. The brain would be like a rigged casino where the roulette wheel comes up red or black at the casino's own choosing, but it must still ensure that the two come up equally over the long term if it is not to be caught breaking the rules. Correspondingly, we might speculate that free will rigs the brain game by influencing individual apparently-random quantum outcomes, which chaotic systems in turn amplify to macroscopic scale, but is limited in the long run because the overall outcomes must match our probabilistic quantum expectations.

Intriguingly, existing psychological experiments are consistent with this model of free will. For example, we know that behaviors that are usually considered exercises of free will such as "paying attention" or "resistance to temptation" are limited and can be exhausted, requiring time to recharge, even though they don't seem to be associated with anything as obvious as depletion of specific neurotransmitters or saturation of synapses. Yet this is exactly what we expect if free will can only influence a limited number of outcomes while staying hidden within known physical laws.

Whatever free will turns out to be -- assuming it exists at all -- understanding it will take at least as great a conceptual leap as that from classical mechanics to quantum theory. And perhaps it is only the reality of chance that connects these three views of reality into a consistent, scientifically explicable universe.