The Chronicle Review

How Art Reveals the Limits of Neuroscience

Mark Shaver for The Chronicle Review

September 08, 2015

You go to a gallery. The work is strange. You don’t know the artist. You aren’t familiar with the style. The pictures all look the same. Flat and dull. They fail to capture your attention. You move on to the next room. You give your energy to your date.

But what if your friend knows this work and she invites you to stop and look again? She calls your attention to features. She suggests questions for you to ask about patterns and relationship, color and content, as well as about the artist’s intent, influences, milieu, and preoccupations.

Sometimes, if you’re lucky, something remarkable happens. Where all the pieces looked alike, now they start to stand out individually. Where they were flat, now you see depth. Where they were dull, now they fascinate.

What a transformation. The paintings have not changed. But neither is the change simply a subjective one, as if you merely have new beliefs or feelings about what you see. No, thanks to your exchange with the work — your queries, your probes, against the background of what you know, how you feel, what you desire — you now see what is there differently. The change opens up a world to you.

This shift — from not seeing to seeing, from seeing to seeing differently, from not getting it to getting it — is actually very common. We live and learn, look and ask, bring what’s around us into focus continuously. At least part of what makes art different, or special, is that it yields the opportunity not only to "get" something, perhaps something new, but also to catch ourselves in the very act. In this way, art illuminates us to ourselves.

These days neural approaches to art — so-called neuroaesthetics — are all the rage. We find it somehow compelling to think that the brain holds the answers to the questions about, well, everything that matters to us, including art. It’s hard not to be impressed by the excitement scientists feel as they try to hunt down aesthetic experience in the brain using the advanced methods and technologies of cognitive science.

But art is an elusive quarry, and it leaves its clumsy predator flailing in the dust. In vain will you find art, or our experience of art, illuminated in these empirical investigations. This points out not just the limits of the neural approach to the arts, but also the limits of neural approaches to human experience in general.

The basic problem with the brain theory of art is that neuroscience continues to be straitjacketed by an ideology about what we are. Each of us, according to this ideology, is a brain in a vat of flesh and bone, or, to change the image, we are like submariners in a windowless craft (the body) afloat in a dark ocean of energy (the world). We know nothing of what there is around us except what shows up on our internal screens.

The brain is necessary for human life and consciousness. But it can’t be the whole story.

Crucially, this picture — you are your brain; the body is the brain’s vessel; the world, including other people, are unknowable stimuli, sources of irradiation of the nervous system — is not one of neuroscience’s findings. It is rather something that has been taken for granted by neuroscience from the start: Descartes’s conception with a materialist makeover.

Careful work on the conceptual foundations of cognitive neuroscience has questioned the plausibility of straightforward mind-brain reduction. But many neuroscientists, even those not working on such grand issues as the nature of consciousness, art, and love, are committed to a single proposition that is, in fact, tantamount to a Cartesian idea they might be embarrassed to endorse outright. The momentous proposition is this: Every thought, feeling, experience, impression, value, argument, emotion, attitude, inclination, belief, desire, and ambition is in your brain. We may not know how the brain manages this feat, but, so it is said, we are beginning to understand. And this new knowledge — of how the organization of bits of matter inside your head can be your personality, thoughts, understanding, wonderings, religious or sexual impulses — is surely among the most exciting and important in all of science, or so it is claimed.

Some scientists attempt to escape this Cartesian vertigo by trying to have it both ways. They grant that we can’t understand the value of money, for example, or the attachment between a parent and a child, without taking up economics and history, on the one hand, or love and caring, on the other. But this, it turns out, is simply a fact about us, about the kind of explanations we, owing to our cognitive limitations, find satisfactory. Love is a neural condition. The value we attach to money is a neurological fact about us and nothing more. Even if we find it hard to describe a mother’s relation to her child without using the folk-psychological category of love, it would be possible to do so, at least in principle. If not to our satisfaction, than to that of a better scientist than we can ever manage to be. That is the argument.

It is remarkable that many people are so quick to be persuaded that we are just packets of neurons, and that the world we think it is science’s very aim to explain is our brain’s confabulation. It isn’t surprising, really, that art gets lost in the shuffle.

Could it be that art, far from getting explained away with the rest of creation, might give us resources for rethinking nature, our nature?

John Dewey offered the somewhat paradoxical thought that it is the very existence of art objects that stands in the way of our framing a plausible aesthetic theory. It isn’t about the objects, he argued. It’s about the experience. And experience, crucially, is not something you get for the price of museum admission. It’s more hard-won than that. It is something we achieve through thoughtful and active engagement.

The fundamental problem with the brain view is that it fails to comprehend that last point. For, according to the neural view, experiences — episodes of awareness, encounters with works of art as well as with other people — are nothing more than bodily effects triggered in us by this stimulus or that. They are not our experiences, in Dewey’s richer sense; they belong to the nervous system. We get them free.

But a work of art, like the meaningful world around us, is not a mere stimulus. And we work hard — we look, we ask, we think, we collaborate — to bring art and world into focus for consciousness.

This is not to deny that the world acts on us, triggering events in the nervous system. Of course it does! But the thing is, we act right back. Every movement of the eye, head, and body changes the character of our sensory coupling to the world around us. Objects are not triggers for internal events in the nervous system; they are opportunities or affordances for our continuing transactions with them. The world shows up, in experience, not like a diagram in a brain chart but as the playing field for our activity. Not the brain’s activity. Our activity. Not activity inside our head. But activity in the world around us.

The concern of science, humanities, and art, is, or ought to be, the active life of the whole, embodied, environmentally and socially situated animal. The brain is necessary for human life and consciousness. But it can’t be the whole story. Our lives do not unfold in our brains. Instead of thinking of the Creator Brain that builds up the virtual world in which we find ourselves in our heads, think of the brain’s job as enabling us to achieve access to the places where we find ourselves and the stuff we share those places with.

Neuroaesthetics, we have observed, is chic. And not only in neuroscience. Humanists — art historians, students of literature, artists themselves, after decades of suspicion of any attempts to bring our cultural lives and our biology into fruitful dialogue — now seem to be inclined gleefully to embrace the new brain-centered conception of everything.

If you are at all sympathetic to the skepticism about neuroscience that I am advancing, then you will perhaps find it plausible that neuroaesthetics is another instance of neuroscience’s intellectual imperialism, another chapter in its attempt to come up with a brain theory of everything. But neuroscience’s recent preoccupation with art, however, reveals something deeper and more fundamental about the neuroscience project itself.

Neuroaesthetics seems unable even to bring its own subject matter, art, under observation.

This comes out clearly when we turn to visual neuroscience and the primacy accorded there to "neural representations." Visual consciousness — what we really see — is said to be given by a picture in the brain. The neuroscientist David Marr called this the two-and-a-half-D sketch, a kind of line drawing in which boundaries between objects are explicit but objects show up as all surface with no hidden parts.

For instance, seeing a cube, according to this sort of approach, is having a line drawing of a cube in your brain. And that, in turn, is supposed to explain why we consciously experience a cube when we look at a mere picture of one. For the seeing brain, a line drawing is a cube’s equivalent. This follows from the starting assumption that the cube itself, in its three-dimensionality, is screened off by the optics of projection. The best we can have is the cube’s 2-D (or two-and-a-half-D) projection.

A raft of ideas about pictures, pictoriality, depiction — neuroaesthetics, for short — is part of visual neuroscience’s very DNA. It shouldn’t be surprising, then, that the Harvard vision scientist Patrick Cavanagh, writing in the journal Nature, advances the thesis that artists, insofar as they are in the business of making effective pictures, "act as research neuroscientists," their pictures brain experiments. But this contention — that art is a kind of neuroscience or that artists act as neuroscientists — is really a statement of first principles and marks no discovery at all. It is, after all, axiomatic for visual neuroscience that we see a cube when presented with a line drawing of one, that for the visual brain at least a line drawing of a cube is a cube’s equivalent. So when we don’t immediately notice mistakes in a painter’s rendering of shadows — a key human foible from Cavanagh’s standpoint — that’s taken as proof that the brain is not governed by a knowledge of physics in constructing its visual world.

So the would-be discovery that artists are in effect carrying out experiments on the brain when they build pictures is not a discovery at all. It follows from the familiar starting idea that we are confined, in our experience, not to things themselves but to neural pictures of them. Cavanagh reads his findings right off his philosophical starting points.

One noteworthy upshot of this putative neural equivalence of representations with that which they represent is the fact that objects themselves, with all their specific, tangible substantiality, play almost no role in neuroscience (let alone in the neuroscience of art). If you want to study object perception, it suffices to work with photographs of objects. The power or meaning of a picture of an object, according to the equivalency thesis, is exhausted by the power and meaning of that which is depicted; and, vice versa, there is no more to an object’s significance (at least to vision) than is carried by a suitable pictorial rendition of it.

The equivalency of a picture and what it depicts, however, is deeply implausible. It is one thing to see a thing and another to see its depiction. That is, whatever we say about why pictures have representational powers, pictures surely don’t give us the experience of encountering in the flesh that which they depict. When you see a picture of something, you have a sense of its presence despite the manifest and perceptually salient fact of its nonpresence. You may see a woman in the statuette. But you are not in the same state you would be in if you were in the presence of an actual woman. The fact that you don’t notice that shadows in pictures look and behave differently than real shadows doesn’t imply that we make no distinction between the two.

A clever neuroscientist will object. If there is a difference between seeing a thing and seeing it in a picture — indeed, if the picture shows you something while at the time showing you that it isn’t there — then this distinctive structure in the experience of pictures will itself be neurally realized. Neuroscience is not committed to ignoring this experientially important difference; it is committed only to the idea that there is no difference that matters that is not, itself, ultimately a neurological difference.

But as we’ve seen, the assertion that mental states of whatever kind — love, monetary value, aesthetic bliss — are identical to brain events is not one that has yet been made good on. And anyway, the neural-equivalency thesis — that seeing something and seeing its picture are states of the same kind — is taken for granted from the start. If you were to give it up, then you’d have to give up Cavanagh’s claim that painters are neuroscientists.

In any case, there is a more pressing issue: It isn’t clear how the work of art can ever make an appearance in neuroaesthetics. After all, for neuroscience, objects themselves are only triggers that are functionally equivalent to their neural representations. So the same is true of art objects. At best they are triggers for events in the nervous system; the object screens itself off thanks to its effect on the nervous system. And so it shouldn’t be surprising that researchers looking at our aesthetic response to sculptures conduct their research in the absence of sculptures, confining themselves to pictures of sculptures, or that scholars like New York University’s Gabrielle Starr, looking into the neural correlates of aesthetic experience, identify such experiences as those triggered by digital photographs of familiar paintings and measured in the test subject by functional magnetic resonance imaging.

If the equivalency thesis is right — if a picture is the neurological equivalent of that which it depicts — then the disappearance of the artwork itself from the study of art can be chalked up to good scientific method. But if we grant that it is one thing to see a picture of something and another to encounter it in the flesh, and that when it comes to art, we are interested, in part, in creations that capture our attention directly and in their own material uniqueness, and not merely as transparencies through which to encounter whatever it is that they represent, then we’ve got a problem on our hands. Neuroaesthetics seems unable even to bring its own subject matter, art, under observation.

Actually, this difficulty shows up already in the programmatic claims, advanced by the University College London neuroscientist Semir Zeki, that it is brains that see art, and that the laws of the brain constrain art. To support this view, Zeki points, for instance, to the way Mondrian paintings exploit intricate details of human color perception. No artist makes art from ultraviolet light, Zeki notes, precisely because such radiation is, after all, invisible.

The problem here is that the brain constrains our experience of art, not because of anything distinctive to do with art, or with the neural representation of art, or with the experience of art, but rather because the brain constrains our visual experience of everything. We can engage with art only to the extent that we can see it. What we have here isn’t a false account of the neural underpinnings of artistic appreciation but one that leaves art and its distinctive neural underpinnings, if there are any, out of serious consideration. It proves the obvious — that we can’t appreciate what we can’t see — and never even gets as far as an analysis of art and our response to it.

If I have at least made you wary of the idea that perceptual experiences take place primarily in your brain, it’s not a leap to suggest that one category of perceptual experiences, aesthetic ones, is also not confined to your head.

My worries here are broader than my own "externalist" starting point. A striking feature of aesthetic responses — this comes out in the gallery example with which I began — is that they are cognitive achievements, comparable to getting a joke. "Getting it" requires wit, insight, understanding. With jokes the relevant scope of understanding concerns what is communicated and expected and in particular what is expected about ways of communicating. The same is true with art.

Here is another feature of aesthetic responses. We have them not only, as it were, in isolated encounters with works of art (as we might when we are in the brain scanner). We frequently learn to have them, and our responses are informed by what teachers, critics, friends, and family say and think, by what work we have seen before, and by what we do or are interested in doing in relation to the art.

Aesthetic responses are not fixed data points but are rather like positions staked out in a conversation, continuing in our day, our lives, and in the historical time of our culture. Aesthetic responses are cultivated and nourished, and they are also challenged. Encountering a work of art is more like an evening with a friend, or an afternoon at work, than it is like the transient relishing of a flavor. Aesthetic responses are also sometimes questions that the art poses to us rather than definitive answers; they are beginnings, opportunities, not data points.

This brings me to a final feature. Aesthetic responses frequently take the form of judgments. We take stands on works of art. We don’t just like them, we react to them, as Kant said, in the "universal voice." That is, we expect others to react to them in a like manner, and if they don’t, then we expect that this is something worth arguing about. Aesthetic disagreements matter. Aesthetic judgments come at the beginning of conversations and not at their conclusion. Art is experienced in the setting of argument, criticism, and persuasion. This is all compatible, Kant realized, with the fact that there is no way of adjudicating disputes in this area, that there are no decision procedures, no rules, no way of proving who’s right and wrong. And yet art, historically, consistently, insistently compels us to agree or disagree.

Aesthetic responses, then, are not symptoms or reactions or stable quantities. They are actions. They are modes of participation. They are moments of conversation. There is nothing about which we can even ask: What are its neural correlates? And moreover, to look for neural correlates is already to have turned away from what deserves to be called aesthetic experience, for it is to have turned away from art and toward mere synaptic goings-on inside us.

Neuroscience is too individual, too internal, too representational, too idealistic, and too antirealistic to be a suitable technique for studying art. Art isn’t really a phenomenon at all, not in the sense that photosynthesis or eyesight are phenomena that stand in need of explanation. Art is, rather, a mode of investigation, a style of research, into what we are. Art also gives us an opportunity to observe ourselves in the act of knowing the world.

We can’t take neuroscience for granted as a kind of intellectual ready-made and apply it to the problem of art. I say this not because I think that science has no business trying to gain an understanding of something so meaningful, so historical, and so cultural as art. The point rather is that neuroscience has failed to frame a plausible conception of human nature and experience. That work still needs to be done.

This suggests an intriguing possibility. Far from being able to explain art from the standpoint of neuroscience, it may be that the order of explanation goes in the other direction. Perhaps it is art that will allow us to forge a more plausible conception of ourselves, one suitable, finally, for grounding a better neuroscience.

Alva Noë is a professor of philosophy at the University of California at Berkeley. This essay is adapted from his new book, Strange Tools: Art and Human Nature (Farrar, Straus and Giroux).