Perception as prediction error minimization

Perception is not just a kind of prediction but “prediction error minimization is all the brain ever does” says Jakob Hohwy in the book “The predictive mind”. The predictive processing paradigm inverts the traditional sandwich perspective of perception-cognition-action by bringing prediction to the center stage. And Hohwy’s book gives an excellent introduction to various dimensions of the predictive processing paradigm. In this article, we first look at perception as Bayesian inference. And then explore the question – what is prediction error minimization? We will use Hohwy’s favorite example – binocular rivalry to illustrate the concept.

Perception as Bayesian inference: How do we perceive the world? According to this framework, we keep a model of the world that predicts the generation of observable data. Based on this model, we have prior hypotheses about the causes of our sensory input. As we receive new sensory input, the brain computes the posterior using the Bayes rule – posterior is proportional to likelihood times prior. The winner, the hypothesis with the maximum posterior, is declared as the cause of the sensory input. Thus perceptual inference involves updating the internal model of the world consisting of a set of prior beliefs based on incoming sensory information. See 3Blue1Brown and Veritasium videos for more details on Bayes theorem. Now, let’s see what happens when we try to trick the brain.

Perceptual inference in binocular rivalry: Binocular rivalry is a phenomenon that has been documented for over four hundred years. If you show two different images to the left and the right eye – say a face and a house – then you don’t see a mixture of the two images say a face-house. Instead, you either see a face or a house. Moreover, what you see keeps alternating even when the images are held constant. The following picture from the book gives a possible explanation of this phenomenon using perception as predictive error minimization.

The likelihood of face+house (F+H) sending such sensory input is higher than just face or just house hypothesis. However, the prior probability of F+H i.e. we ever seeing such a thing as a mixture of a face and a house is very low. Hence, when you multiply the two – i.e. prior and likelihood – the face-only or house-only hypothesis wins over the face-house hypothesis. And that’s what you see.

Now, how do you explain the alternating images in the binocular rivalry? As you see one of the two images, say the face, the prediction error resulting from the sensory input from the house is explained-away. Over time, the prediction error builds up and the brain is not able to explain it away. And the brain chooses the competing hypothesis which is the house-only image. This is a hand-wavy explanation. It is heavy-duty mathematics at play. If you are curious, check out Hohwy’s paper “Predictive coding explains binocular rivalry”.

What if you are able to bias the prior in favor of one of the images? Hohwy talks about a variant of the binocular rivalry experiment in his book where one eye is shown text markers and the other eye is shown roses. Then olfactory evidence was added and participants smell roses. As predicted by the Bayesian rule, the participants consequently spent more time perceiving the role image. 

Isn't perception just one of the processes at play in the brain? What about action, attention, learning? And what about various biases? Well, the book shows that the predictive error minimization framework is quite ambitious in its goal and does a good job of attempting to explain various phenomena using the framework. 

To understand the free-energy principle, the core principle behind prediction error minimization, in its full depth, one would need to go into statistical mechanics, self-organization, dynamical systems theory, information theory. However, the book lays a good ground for the curious. And the theory is in its infancy and an active area of research right now.    

Image source: 

book image is from amazon.com, the binocular rivalry image is from Hohwy’s paper “Predictive coding explains binocular rivalry” and it is very similar to the image in the book.

Resources:

• Jakob Hohwy, “The predictive mind”, Oxford University Press, 2013. 
• Manuell Brenner, “The Bayesian brain hypothesis”, Aug 15, 2019 (blog) 
• Jared Tumiel, “Spinning up in active inference and the free energy principle: A syllabus for the curious”, Oct 14, 2020. Extensive list of references to dive deep into this area.
• Kai Uletzhoffer, "The free-energy principle and active inference", June 23, 2017 (blog)
• Sergio Graziosi, “The predictive brain (part one): what is this about?” January 17, 2015, blog  
• Sergio Graziosi, “The predictive brain (part two): is the idea too generic?” January 24, 2015, blog
• Predictive coding, wikipedia page