What's Wrong with Genetic Algorithms
Marvin Minsky pointed out that genetic algorithms, and evolution in general, have a serious flaw. In almost all respects, they are worse than the standard artificial intelligence heuristic search.
Genetic algorithms attempt to find the solution to a problem by creating thousands of randomly mutated spawns of a somewhat-good solution. Then, most of the bad ones are slaughtered and the survivors interbreed and make more mutated spawns for the next generation. If you repeat this process for a looooong time, after many generations you'll end up with a good approximation to the correct answer, or maybe even the right answer.
However, this brute-force search through solution space is definitely not guaranteed to find the correct answer, and when it does, it takes an incredibly long time to do it. The reason is that it's only remembering what went right, not what went wrong -- if there are 10,000 mutations in every generation, and it only keeps 5, it's throwing away useful information about why 9,995 of them died. Over time, it should slowly build up the most common mistakes.
Perhaps that's why humans are so stupid! If we could look at the mistakes we (and other people) make, and save this information in our genetic code so that our offspring would be born knowing what we learned in our lifetimes, our species would evolve much faster.
The idea that genetic algorithms are a great way to solve optimization problems is an almost superstitious belief.
A good way to illustrate the difference between heuristic searches and genetic algorithms is with a simple problem: finding your way through a maze that has only one correct path to its exit. The problem is to find the correct path (or set of variables) that gives you a solution. This is an identical problem to searching for a solution in a search space.
The genetic algorithm's approach to solving the maze would be to spawn thousands of little maze-solving entities, each of which has a specific set of maze-moves stored in its genetic code. The genetic code is generated randomly for the first generation, and then tested out -- all of the entities scurry through the maze, and every entity that hits a wall dies, thus eliminating it from the gene pool. The entities that make it the furthest get picked as the cream of the crop to randomly cross-breed with each other to produce the next generation of maze-bots. Wave after wave of these entities pour like a river through the entire maze, casting about blindly and killing themselves, until that one lucky entity makes it to the exit carrying the answer in its DNA.
Now, a heuristic search's approach starts off the same way -- casting about randomly. However, instead of killing itself and forgetting everything it knows, it records all the moves it makes that result in death. Every time it makes a mistake, it learns not to repeat it, thus narrowing the search space with every wrong move. Eventually it'll find the correct solution in a fraction of the time of a genetic search.
Now, genetic searches are great when you're given a situation like in nature -- you have enough resources to build as many of these little entities as you want, and you have no prior knowledge of the world you exist in (i.e. no memory or previous genetic code). Since there's energy and material everywhere, this is a fine solution! You can exploit the massive parallelism of the universe to test out billions of possibile solutions simultaneously. It's a genius process because it allows you to create intelligence when there is no intelligence.
The problem is that we don't run our genetic algorithms on an infinitely parallel computational resource like nature -- we use puny little single-processor computers with limited memory. The genetic algorithms are hugely wasteful and very slow, especially if you're trying to solve a real, meaty problem. The complexity explodes exponentially. Remembering the mistakes is the best way to solve a problem quickly. If evolution could remember its mistakes, we could've evolved to where we are today in 4 million years instead of 400 million -- and with less glitches!
Evolution in nature, however, is a genius system because it explains how you can create intelligence without having any intelligence to start with. Evolution ended up creating lifeforms with brains that allow them to remember their own mistakes and pass this knowledge to their offspring via language. Evolution has discovered that evolution itself is not the most efficient way to solve a problem, so why would you use it? Do you not trust evolution? 
What's also fascinating is that what we store in our brains actually shapes the evolution of our genes. You can read about it in this paper: How Learning Guides Evoluton (by Geoffrey Hinton and Steven Nowlan).
Questions
- Many people use a combination of neural networks and genetic algorithms to solve problems. The genetic algorithm picks the size and structure of the neural net, and the neural net does the actual learning/solution-finding. Is this actually an efficient way to find solutions, or would it be better to use a heuristic search or unsupervised learning algorithm to design the neural net's structure?