« Extracting Information from People | Main | Learning abilities of computers and humans »

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d8342dcba453ef00d83496a23d53ef

Listed below are links to weblogs that reference Blink:

Comments

misha b

Have you played lightning chess? It is possible to play a decent approximation of the game, taking no more than 1-2 secs per move.

G. Blanchard

Hi Olivier. I have not read this book, but I think it does not necessarily contradict your intuition about evolution. I would look at it this way: humans have been successful in evolution because they are able to adapt (or learn) quite broadly to new situations. It would not be a very successful evolutionary strategy (I think) to put all the "prior" on tasks-immediately-related-to-survival only. First, because then it would probably be more efficient to hardcode them rather than have a strong prior and learn them anyway. Second and foremost, because the definition of such a "survival task" can vary a lot depending on surroundings, time, etc.

My vision is this: evolution has devised for us humans a very good learning algorithm along with a very good "prior", which, is turns out, is both adapted to the world we live in yet flat enough to cover a very wide variety of possible tasks, potentially related to survival. But as a side effect of this broad range, it extends beyond that.

How do you encode efficiently such a prior? I think we as machine learning guys also can follow our intuition: good features. We all know that the quality of a learning algorithm means nothing if you do not operate on good features to begin with (think visual, auditory signal processing). I think "good features" are a very powerful way to encode such a broad-yet-adapted prior.

Coming back to the examples you mentioned, I am amazed by the versatility and speed of human classification, yet somehow I can at least put some reason behind it. In the tennisman example: it sems clear that it is critical for survival to be able to decode (and predict) very precisely animal (and human) behavior. Will it attack? Is it looking in my direction? Did it see me? The tennisman example is a very different task of course, but I don't think it is unreasonable to assume that it can be based on the same kind of "features".

In a sense, I am much more impressed by people being able to solve a Rubik's cube in 15s. Or simply playing the piano. I mean, you have to wonder why evolution has made such a degree of hand-eye-abstract thinking coordination learning even possible. How is that possibility really going to help me in the jungle?

Patrick Chanezon

You seem to assume that evolution has stopped: we don't live in the jungle anymore and this skill that the tennis coach in the book developed is how he "makes his living", litterally.

The really interesting question for me is more: what's the next step required in human evolution if we want to be able to survive and maybe exapnd beyond earth:-)

Olivier Bousquet

Thanks to all of you for your interest. Let me try to answer all of your comments.
I will start with Misha's comment about lightning chess.

Taking again the point of view I was advocating in the post, the fact that one can perform complex reasoning tasks very fast is not contradictory with the fact that humans cannot learn easily tasks that are not "natural". Let me try to explain: you could think that chess is not a very natural task and it would be surprising if humans had a built-in ability to play chess. However, their ability to make high level reasoning allows them to learn this game and to be relatively good at it. But this learning is completely "explicit", that is, it occurs consciously (for the most part of it). So, at least initially, beginners can explain quite easily the reasoning they followed when choosing a move. Then, one can assume that the more you train, the more you can transfer this reasoning into some kind of "reflex". So in some sense, the learning process transfers the ability from the conscious to the unconscious level. Once this stage is reached, it is much more difficult to explain one's choices.

But this whole process (if indeed it works as described above) would not be contradictory with my previous belief that tasks that are not natural cannot be easily (and I mean here unconsciously) learnt by humans.
So a possible conclusion would be the following:
- either a task is "natural" and humans are likely to have the appropriate hard-wiring (prior), so that they can learn it without explicit training (ie unconsciously)
- or the task is "unnatural" and humans need first a phase of conscious or explicit training (where the reasoning is explained in a high-level way), before the process can be, with sufficient practice, 'automated', or transformed into some kind of reflex.

So, if you accept this theory, there is no contradiction between my post and your comment, as far as I can see.
Of course, you may discuss the theory itself...

Olivier Bousquet

Now, let me move to Michael's comment.
The link is very interesting, and, although the suggested approach, ie to try and build machines that can imitate basic newborn behaviour rather than complex adult reasoning, is not new, I quite like it.
Two important components of babies intelligence, as described by Mike, are some sort of novelty detection engine, and some sort of pattern recognition engine.

It is clear that humans (but also animals, and very basic ones -- so this is not a human trait -- but who said intelligence is only human?) have the ability to perform both tasks pretty easily. But the question is how general are these general components?
Can we learn to classify (in the sense of supervised binary classification) anything?
Probably not (the human brain is surely not 'universally consistent' due to its physical limitations). Even further, we are probably able to learn a very restricted set of tasks.
In Bayesian terms, one would say that the prior encoded (or hard-wired) in human brains is very concentrated.
The point I am trying to make here is that the impressive ability of babies to learn to distinguish objects or faces, that we are still not able to match despite decades of research, may not come from a "powerful" pattern recognition engine or from "powerful" sensors, but simply from a very concentrated prior, which itself has been designed over thousands of years of evolution.

So a possible conclusion (although I would not fight too hard for it) is that our algorithms are perfectly fine and up to human's performance (probably even much better), but what we are lacking is the right prior and the right preprocessing (which also is part of the hard-wiring in the brain).

Olivier Bousquet

Now let me move to Gilles' comment.
This is a very reasonable way of putting things: it is indeed very likely that our "prior" is flat enough to adapt to many different environments and learn many different tasks that can be useful for survival (even though the effect on survival is indirect).
In a way, all of our abilities could be related one way or another to survival: if you are good at something (even rubik's cube or piano), this may help you have more children (eg you can make more money, or attract women,...), although this effect would be very deem in general (but probably strong enough to orient the evolutionary process).
So, one could imagine that many of the skills we have are indeed due to the evolutionary process favoring (sometimes randomly) them.
I agree with your point of view on the features: good features are crucial for good learning performance and the tennis coach ability is very likely connected to the innate ability to identify ennemies' actions (ie these may use the same kind of feature extractors). So, a combination of a flat prior with a specialised feature extraction engine can explain the human learning ability in many situations.

Olivier Bousquet

Regarding Patrick's comment: I agree that we are still evolving, but I suspect that given the timescale at which the evolutionary process operates, we are not far from still living in the jungle.
On the other hand, there are known examples of evolution happening of very short timescales (this has been observed for some butterflies adapting to the pollution for example -- I do not remember the reference though).
It seems that evolution happens faster when there is some kind of immediate threat to the survival of the specie. So maybe if we happen to go to another planet, the significant change of environmental conditions would induce a significant evolution and we would be able to adapt in a few generations.
The sad thing is that we will never be able to see such things in our lifespans...

assman

I think the tennis think is not hard to understand. It conveys a direct survival advantage to have a good understanding and anticipation of physical movements. When I have played soccer or other sports I have consistently observed that the best players have an extremely good sense of where the ball is going to be, are able to anticipate other players etc. This can be explained by innate ability.

However, why do humans have to have an innate ability (concentrated prior) in order to be good at something? It is possible that there is no prior info for something like solving a rubics cube but that humans compensate by having tonnes of training. Many experts spend years training. In other words you are taking a massive neural network, then running it for years on a specific data set. Its not suprising that the neural network is good at doing something given that you have expended an enormous amount of resources training it.

To make a machine good at something you do not necessarily have to have a good learning algorithm. You could have a really crappy algorithm but devote enormous resources to training it.

Stephen Harris

I experienced "blink" recently. There is an economic aspect of rational decision theory which tries to explain why religion is robust in the US, even though the US has separation of church and state. Paradoxically, the claim is that this separation encourages competition with a free market which will produce the sought after religion of seekers (demand)with a fluid supply. My subconscious said Nope! I pondered why and the answer of cause and effect reversal flashed into my consciousness. Religions spread usually starting as a cult and then proselytize, not as a response to religious consumer demand. The theory needs an abstruse mechanism like a collective (un)conscious so that the desire for the new brand of religion could be seeded so that some individual could conjure up a new revelation. Something, which I think is similar is the channeling of a metaphysical entity which usually happens after people have immersed themselves by reading in the topic. Steve Bruce provides a critique of the supply-and-demand religious theory which has gathered some adherents in sociological circles. Also there is something like "blink" with savants and high-end autism.

Cheap Hermes Handbags

I recently came across your blog and have been reading along. I thought I would leave my first comment. I don't know what to say except that I have enjoyed reading. Nice blog. I will keep visiting this blog very often.

The comments to this entry are closed.