Blink

I have just started reading "Blink, the power of thinking without thinking" by Malcolm Gladwell. Similarly to Freakonomics, this book has sold very well in the US. I was thus curious about it.
Overall, it is fun to read, although a bit unorganized. But what is especially striking for me is the main claim that humans can reason unconsciously.
More precisely, there are many situations (and the book gives a large number of surprising yet convincing examples) where humans are able to perform difficult "classification" tasks unexpectedly fast. For example, some art experts are able to tell apart genuine sculptures from fake ones virtually in a blink. Even more surprising: they are completely unable to explain what makes them think a specific sculpture is fake!

It thus seems (and there are plenty of psychological studies about this), that, with enough training, humans are able to learn very difficult classification (I take it in the classical Machine Learning sense) tasks, including tasks that are not natural.

Let me try to explain what is new here.
We know that humans are very good at learning certain classification tasks: young children can classify objects from images very easily and get a much better performance than any computer to date.
Also we know that once this has been learned, the actual classification of any new image can be done in a few milliseconds.
Hence, with enough training, the brain is able to perform this complicated task very easily, without requiring any conscious reasoning to take place.

However, I used to think that the tasks we can learn easily are those for which we have a sufficiently strong prior encoded into our genes. In other words, I thought that the ability to learn visual classification tasks was the result of a long natural evolution (which provides us with the appropriate pre-wiring, or the prior in Bayesian terms) combined with a short period of adaptation (similar to computing the posterior in Bayesian terms again).
What is new to me in this book is the following: we can be trained to perform tasks that have nothing to do with evolutionary constraints, and this training can be performed unconsciously (without any explicit or conscious reasoning). An example of this phenomenon is given in the book: a tennis coach once realized that he could predict whether a tennis player would miss his service right before he would hit the ball. However he would not be able to explain why and how he could do so!

This may show that our brain hosts a powerful learning engine (with a powerful feature extractor to isolate the relevant information) that does not even require our attention to be triggered and that can deal with many different learning tasks.
Of course this raises the question of the prior: we know that there is no better learning algorithm, but only algorithms better adapted to learning problems. In other words, we can only learn the problems that have a large enough weight under the prior, which means it is hard to be good simultaneously for many different tasks.
Why is it that the prior encode into our brain allows us to learn such useless tasks as being able to tell whether a tennis-man will fail his service? and why is it that this prior is not more "peaked" around the tasks that are really useful for our survival?
I guess this book is related to a lot of interesting cognitive science problems but it also revived my interest in human learning and its relationship to Machine Learning...

Comments

[misha b]

Have you played lightning chess? It is possible to play a decent approximation of the game, taking no more than 1-2 secs per move.

[Michael]

I'd appreciate your thoughts on this article: http://vcmike.blogspot.com/2006/05/right-cognitive-testbed-for-ai-babies.html

[G. Blanchard]

Hi Olivier. I have not read this book, but I think it does not necessarily contradict your intuition about evolution. I would look at it this way: humans have been successful in evolution because they are able to adapt (or learn) quite broadly to new situations. It would not be a very successful evolutionary strategy (I think) to put all the "prior" on tasks-immediately-related-to-survival only. First, because then it would probably be more efficient to hardcode them rather than have a strong prior and learn them anyway. Second and foremost, because the definition of such a "survival task" can vary a lot depending on surroundings, time, etc.

My vision is this: evolution has devised for us humans a very good learning algorithm along with a very good "prior", which, is turns out, is both adapted to the world we live in yet flat enough to cover a very wide variety of possible tasks, potentially related to survival. But as a side effect of this broad range, it extends beyond that.

How do you encode efficiently such a prior? I think we as machine learning guys also can follow our intuition: good features. We all know that the quality of a learning algorithm means nothing if you do not operate on good features to begin with (think visual, auditory signal processing). I think "good features" are a very powerful way to encode such a broad-yet-adapted prior.

Coming back to the examples you mentioned, I am amazed by the versatility and speed of human classification, yet somehow I can at least put some reason behind it. In the tennisman example: it sems clear that it is critical for survival to be able to decode (and predict) very precisely animal (and human) behavior. Will it attack? Is it looking in my direction? Did it see me? The tennisman example is a very different task of course, but I don't think it is unreasonable to assume that it can be based on the same kind of "features".

In a sense, I am much more impressed by people being able to solve a Rubik's cube in 15s. Or simply playing the piano. I mean, you have to wonder why evolution has made such a degree of hand-eye-abstract thinking coordination learning even possible. How is that possibility really going to help me in the jungle?

[Patrick Chanezon]

You seem to assume that evolution has stopped: we don't live in the jungle anymore and this skill that the tennis coach in the book developed is how he "makes his living", litterally.

The really interesting question for me is more: what's the next step required in human evolution if we want to be able to survive and maybe exapnd beyond earth:-)

[Olivier Bousquet]

Thanks to all of you for your interest. Let me try to answer all of your comments.
I will start with Misha's comment about lightning chess.

Taking again the point of view I was advocating in the post, the fact that one can perform complex reasoning tasks very fast is not contradictory with the fact that humans cannot learn easily tasks that are not "natural". Let me try to explain: you could think that chess is not a very natural task and it would be surprising if humans had a built-in ability to play chess. However, their ability to make high level reasoning allows them to learn this game and to be relatively good at it. But this learning is completely "explicit", that is, it occurs consciously (for the most part of it). So, at least initially, beginners can explain quite easily the reasoning they followed when choosing a move. Then, one can assume that the more you train, the more you can transfer this reasoning into some kind of "reflex". So in some sense, the learning process transfers the ability from the conscious to the unconscious level. Once this stage is reached, it is much more difficult to explain one's choices.

But this whole process (if indeed it works as described above) would not be contradictory with my previous belief that tasks that are not natural cannot be easily (and I mean here unconsciously) learnt by humans.
So a possible conclusion would be the following:
- either a task is "natural" and humans are likely to have the appropriate hard-wiring (prior), so that they can learn it without explicit training (ie unconsciously)
- or the task is "unnatural" and humans need first a phase of conscious or explicit training (where the reasoning is explained in a high-level way), before the process can be, with sufficient practice, 'automated', or transformed into some kind of reflex.

So, if you accept this theory, there is no contradiction between my post and your comment, as far as I can see.
Of course, you may discuss the theory itself...

[Olivier Bousquet]

Now, let me move to Michael's comment.
The link is very interesting, and, although the suggested approach, ie to try and build machines that can imitate basic newborn behaviour rather than complex adult reasoning, is not new, I quite like it.
Two important components of babies intelligence, as described by Mike, are some sort of novelty detection engine, and some sort of pattern recognition engine.

It is clear that humans (but also animals, and very basic ones -- so this is not a human trait -- but who said intelligence is only human?) have the ability to perform both tasks pretty easily. But the question is how general are these general components?
Can we learn to classify (in the sense of supervised binary classification) anything?
Probably not (the human brain is surely not 'universally consistent' due to its physical limitations). Even further, we are probably able to learn a very restricted set of tasks.
In Bayesian terms, one would say that the prior encoded (or hard-wired) in human brains is very concentrated.
The point I am trying to make here is that the impressive ability of babies to learn to distinguish objects or faces, that we are still not able to match despite decades of research, may not come from a "powerful" pattern recognition engine or from "powerful" sensors, but simply from a very concentrated prior, which itself has been designed over thousands of years of evolution.

So a possible conclusion (although I would not fight too hard for it) is that our algorithms are perfectly fine and up to human's performance (probably even much better), but what we are lacking is the right prior and the right preprocessing (which also is part of the hard-wiring in the brain).

[Olivier Bousquet]

Now let me move to Gilles' comment.
This is a very reasonable way of putting things: it is indeed very likely that our "prior" is flat enough to adapt to many different environments and learn many different tasks that can be useful for survival (even though the effect on survival is indirect).
In a way, all of our abilities could be related one way or another to survival: if you are good at something (even rubik's cube or piano), this may help you have more children (eg you can make more money, or attract women,...), although this effect would be very deem in general (but probably strong enough to orient the evolutionary process).
So, one could imagine that many of the skills we have are indeed due to the evolutionary process favoring (sometimes randomly) them.
I agree with your point of view on the features: good features are crucial for good learning performance and the tennis coach ability is very likely connected to the innate ability to identify ennemies' actions (ie these may use the same kind of feature extractors). So, a combination of a flat prior with a specialised feature extraction engine can explain the human learning ability in many situations.

[Olivier Bousquet]

Regarding Patrick's comment: I agree that we are still evolving, but I suspect that given the timescale at which the evolutionary process operates, we are not far from still living in the jungle.
On the other hand, there are known examples of evolution happening of very short timescales (this has been observed for some butterflies adapting to the pollution for example -- I do not remember the reference though).
It seems that evolution happens faster when there is some kind of immediate threat to the survival of the specie. So maybe if we happen to go to another planet, the significant change of environmental conditions would induce a significant evolution and we would be able to adapt in a few generations.
The sad thing is that we will never be able to see such things in our lifespans...

[assman]

I think the tennis think is not hard to understand. It conveys a direct survival advantage to have a good understanding and anticipation of physical movements. When I have played soccer or other sports I have consistently observed that the best players have an extremely good sense of where the ball is going to be, are able to anticipate other players etc. This can be explained by innate ability.

However, why do humans have to have an innate ability (concentrated prior) in order to be good at something? It is possible that there is no prior info for something like solving a rubics cube but that humans compensate by having tonnes of training. Many experts spend years training. In other words you are taking a massive neural network, then running it for years on a specific data set. Its not suprising that the neural network is good at doing something given that you have expended an enormous amount of resources training it.

To make a machine good at something you do not necessarily have to have a good learning algorithm. You could have a really crappy algorithm but devote enormous resources to training it.

[Stephen Harris]

I experienced "blink" recently. There is an economic aspect of rational decision theory which tries to explain why religion is robust in the US, even though the US has separation of church and state. Paradoxically, the claim is that this separation encourages competition with a free market which will produce the sought after religion of seekers (demand)with a fluid supply. My subconscious said Nope! I pondered why and the answer of cause and effect reversal flashed into my consciousness. Religions spread usually starting as a cult and then proselytize, not as a response to religious consumer demand. The theory needs an abstruse mechanism like a collective (un)conscious so that the desire for the new brand of religion could be seeded so that some individual could conjure up a new revelation. Something, which I think is similar is the channeling of a metaphysical entity which usually happens after people have immersed themselves by reading in the topic. Steve Bruce provides a critique of the supply-and-demand religious theory which has gathered some adherents in sociological circles. Also there is something like "blink" with savants and high-end autism.

[Cheap Hermes Handbags]

I recently came across your blog and have been reading along. I thought I would leave my first comment. I don't know what to say except that I have enjoyed reading. Nice blog. I will keep visiting this blog very often.