After listening to Roger Penrose (someone I learned to admire) in this interview, I became really interested in something he alluded to, about the implications of Gödel’s incompleteness theorems to the pursuit of human-like AI (a.k.a “Artificial General Intelligence”). I had studied those theorems in university and after twenty years could still remember the feeling of amazement I got from them at the time, but the particular details required to form an opinion on what I had just heard were long gone (even if we did go through the proofs in the class back then) .
Fortunately we have Internet (clearly this topic does interest my lazy self, but not enough to buy a book about it… and I gave most of my academic books away)!
I found this link to be exceedingly helpful and clear about the subject, and to be perfectly honest, much simpler than the way I learned it (that much I could remember…).
So, hypothetical claims that the incompleteness theorems by Gödel may imply some insurmountable limitations on the pursuit of AGI would be in my humble opinion after what I read, not totally substantiated. Either the mind can be seen as an effective, consistent and rich formal system, in which case it is also incomplete as the theorems state, as much as any other formal system trying to emulate it (e.g. a computer program), or it does not have such characteristics, in which case nothing is stated in the theorems about its completeness (or purported superiority).
There may be truths that are equally hidden from mind and machine, even if inconsistency may hold some promise for completeness : our minds are possibly more rooted in quantum reality and contradiction than in the formal models of logic reasoning, a notion that is very hard to entertain. If we are to construct machines that transcend the human mind as regards the ability to prove everything, Gödel’s theorems are just stating what doesn’t work, not what does.
Update: this wonderful BBC documentary goes much deeper into the history of the topic.
Origami Tree 