Tuesday, 14 July 2009

Interactions Between Learning and Evolution

David Ackley and Michael Littman (1991). "Interactions Between Learning and Evolution", In: Proceedings of the Second Conference on Artificial Life.

New paper added to my reading list.

This is a very interesting paper about some agents that can learn and also evolve. The learning happens over a short time span, the lifetime of an agent, the evolution happens as agents reproduce and populate the simulation. The agents use two neural networks as controllers, one of them is an 'action network' which controls how the agent reacts to a situation. The other is an 'evaluation network' which is how the agent decides what is a good situation and what is bad. The evaluation network and the initialisaiton for the action network are inherited from parent agents. So natural selection controls what the agents think is good or bad, and how they will initially react to stimuli in the environment, and over time the agent will adapt its action network to learn behaviours that result in situations it believes are good. The agents' aim is to live for as long as possible and to reproduce as often as possible.
The simulation they used was apparently very difficult to survive in, which skewed their results a bit (it made evolution a bit crappy for instance because agents, and most of the population, died before they could pass along genes). The results they presented of the one population that lasted over 9 million time steps were interestin though. They showed how having the population start out knowing that food is good eventually led to the agents instinctively getting food, it also showed that when the agents start off instinctively avoiding predators their evaluation of whether predators are good or bad had little effect. These results are somewhat contradictory but the experiment was successful in that the agents lasted a long time before going extinct.
Overall what they present is a very interesting control structure, and I think how they implement instinctive actions is quite clever (but smart people might think it was obvious) the paper leaves me wondering what would happen if it was easier to survive in the simulation, (probably there would be less adaption and the experiment would be much less interesting).

No comments:

Post a Comment