How do these two domains relate?
Actually there are many interconnections between them. Here are some:
- Most learning algorithm start by defining a criterion that they optimize. Hence, once the criterion is chosen, learning is just optimizing this criterion.
- In optimization problems, one may have situations where part of the objective function which is not known exactly, but with uncertainty (via a sampling mechanism for example). This is the context of stochastic optimization. The goal being to optimize a function which you know only partially (via its value at random points for example).
- In the process of optimizing a function, you may gather information about it. For example if you only know a function implicitly (you can get its value at any point, but you don't know its mathematical representation), you may want to build a model of (i.e. learn) this function in order to speed up the optimization process.
In most real-world applications of Data Mining or Machine Learning, there is an optimization component which may be ignored.
People tend to apply the techniques they know best and since the communities of optimization and learning are quite distinct, this leads to sub-optimal solutions.
Most businesses have as a primary goal to optimize return under constraints. They may have collected data on how the market, the customers or their production lines behave, but the goal is not just to model this behavior, it is to use this model in order to optimize the return.
So it seems that for most practical problems it is important to have in mind both aspects.