Can the study of human learning help?

The question is whether, in order to make progress toward building learning machines, it is necessary to study the only available examples of such machines we have so far: animal brains (and more specifically human ones).
People who would answer no to this question often cite the example of planes: the first successes for building flying machines were obtained when people stopped trying to imitate birds. So it seems that understanding how Nature solved the problem may not always help. One reason is that animal brains were not designed only to solve the learning problem, just as birds were not designed to solve the flying problem. It is only a byproduct of an evolution that was mainly geared towards survival and adaptation to certain environmental conditions.

Despite these considerations, there has been many attempts to make bridges between the study of natural and artificial learning. For example, people working on artificial neural networks, or on genetic algorithms were never ashamed of using biological findings as a source of inspiration.

My feeling is that it is fine to be interested both in artificial and natural learning provided the following is accepted:

• First of all it is not necessary to start from natural learning to develop a theory of artificial learning, and there is no need for this theory to explain the specifics of natural learning.
• Second, it is important to abstract away from natural learning in order to formulate precisely what learning means.
• However, any source of inspiration is good, especially when one gets stuck, so why not looking for inspiration in natural learning.

I used to think that the work being done in cognitive psychology was too specifically human to be of any interest for people working on learning theory. Also I was very suspicious about  whatever was said to be "biologically inspired" or "cognitively inspired". However, the remarkable efforts for abstracting concepts that has been done by some cognitive psychologists suggest that might have been too critical.

In a recent interview, Tom Mitchell, a famous ML researcher expressed similar views:

[The interviewer] - Learning the brain’s algorithms for doing things is very difficult, and is not very well understood as yet. Do you ever find it frustrating trying to get computers to learn things that we ourselves don’t know the inner workings of?
[Tom Mitchell] - That’s actually a very interesting observation—I actually don’t get frustrated by that—why? I don’t know!
Maybe it’s odd, but it’s true that much of the work in machine learning—how to get computers to learn—has been kind of unguided by anything we know about human learning. It just grew up on its own—“ok, how would we engineer this system to look at a lot of data and discover regularities?”—so people engineered those instead of looking at how humans do it and then trying to duplicate it. But recently, because I’ve been looking at the brain, I’ve been starting to learn more about what people know about human learning—and it’s very different. For example, when we humans learn, a big part of what determines whether we succeed or not is all about motivation. And there’s nothing in machine learning algorithms that even remotely corresponds to motivation. So it’s just a very different phenomenon…maybe in 10 years we’ll understand it better, but right now, the two are very different.

Also, I recently heard about the work of Alison Gopnik who studied the way children learn causes and she draws some interesting connections between the causal structure that is inferred by them and graphical models such as Bayesian networks. Also she explains that the way children learn is very "multivariate" in the sense that they try many things at a time and extract causal relationships easily from multidimensional observations. In other words, it is not necessary for them to act on one knob at a time to understand how a machine works.

Comments

[Csai]

Hi,
I am a first-time visitor and from the looks of it, I will be a regular. I second Tom Mitchell's thoughts on the inspiration you derive from the brain. The motivational aspect of learning is surprisingly absent from ML, while it is taken to be critical and essential by people working on learning in the brain. As the amount of data to be analyzed inreases exponentially, a subsystem specifically desgined to weed out irrelevant data is important. However, the operating word 'irrelevant' is a tricky and complex one as it is not a static concept. It depends on current task and the like. The important thing to realize here is that the brain is not limited to solving a narrow domain of problems, and has not evolved specific modules for performing tasks in each sensory domain. As to your observation about the absence of any learning theories inspired by cognitive processes, there is Adaptive Resonance Theory, that has led to the development of classifiers like ARTMAP and its many flavors.

[Jefferson Provost]

"...the first successes for building flying machines were obtained when people stopped trying to imitate birds." I think this isn't strictly true. The concept of the airfoil, so essential to airplanes and helicopters, came from the study of bird's wings. I think the important message here is that in studying the brain, we should be looking for important computational principles of learning, rather than trying to imitate structure. When these principals are discovered they can be abstracted away from the structure and the resulting machine learning systems may look as different from brains as helicopters do from birds.

But you're also right, just as there are flying machines that don't use airfoils (blimps, rockets), there will be successful ML systems that don't use any computational principles from the brain.

[akex]

Check out the work of thom grifiths, tenebaum, yale niv, nathaniel daw.......the list goes on. There is LOADS of interesting machine learning research going on at the intersection of ML and cognitive psychology, you just haven't looked hard enough.
Reinforcement Learning -> invented by psychologists

[Kamagra]

Hey, this is cool stuff, something worth my time. Good attempt. Quiet interesting way of narrating interesting stuff. I think you have spent a lot of time in researching as some of the facts were just so shocking and never heard of. I am speechless…

[hollywood bistro]

It always does. It's the root of learning.

[buy viagra]

I liked this blog, i think is very interesting, most of all for the new ideas that this blog talk.

[christian louboutin knockoffs]

I did a little research on the topic and found that most people agree with your blog. Thanks.