Welcome to the forum, Goaty. I'm pretty new here myself.
One tip about your thinking: Don't assume that AGI must be powered by a "computer." It might be powered by a neural network, cellular automaton, or some other exotic type of machine.
(p. 12)
It was like running into a brick wall. MIT was the mother-
ship of artificial intelligence. At the time I applied to MIT, it was
home to dozens of bright young people who were enthralled with the
idea of programming computers to produce intelligent behav-
ior. To these scientists, vision, language, robotics, and mathe-
matics were just programming problems. Computers could do
anything a brain could do, and more, so why constrain your
thinking to biological messiness of nature's computer?
Studying brains would limit your thinking. They believed it
was better to study the ultimate limits of computation as best
expressed by digital computers. Their holy grail was to write
computer programs that would first match and then surpass
human abilities. They took an ends-justify-the-means approach;
they were not interested in how real brains worked. Some took
pride in ignoring neurobiology.
This struck me as precisely the wrong way to tackle the
problem. Intuitively I felt that the artificial intelligence approach
would not only fail to create programs that do what humans can
do, it would not teach us what intelligence is. Computers and
brains are built on completely different principles. One is pro-
grammed, one is self-learning. One has to be perfect to work at
all, one is naturally flexible and tolerant of failures. One has
a central processor, one has no centralized control. The list of dif-
ferences goes on and on. The biggest reason I thought comput-
ers would not be intelligent is that I understood how computers
worked, down to the level of the transistor physics, and this
knowledge gave me a strong intuitive sense that brains and
computers were fundamentally different. I couldn't prove it,
but I knew it as much as one can intuitively know anything.
(p. 13)
Ultimately, I reasoned, AI might lead to useful products, but it
wasn't going to build truly intelligent machines.
Hawkins, Jeff. 2004. On Intelligence. New York: Times Books.