Not only do humans create tools, we create tools with control loops. A thermostat has a goal. We give it the goal. It senses its environment and performs actions to change the environment. When it senses, it compares the environment to the goal.
A manufacturing line that assembles thermostats is a control loop that assembles a control loop. It has sensors and actuators which achieve the goal of putting thermostats together. The manufacturing line is at least two levels of abstraction from the basic problem of room temperature.
An axe is a tool but not a control loop. A sharpening stone sharpens axes which cut trees which are used to build structures. Adding levels of abstraction to our technology stack is something innate. We’ve been doing it since the Stone Age. But adding levels of abstraction to tools with control loops is something different.
An artificial intelligence revolution would look different from an industrial revolution because AI that writes software isn’t just a tool that creates a tool; it is a control loop that creates a control loop.
If an AI is able to write programs and redesign the lower levels of its own operation, we would be looking at a daunting possibility: an AI system capable of recursive self improvement. That would be the threshold of an intelligence explosion. There would be no guarantee that humans would remain in the loop.
Two things that stand out to me about humans compared to all other species are that we create tools with control loops and our own cognitive loops stretch far into the future. Our ability to look forward and backward in time is fundamental to our ability to cooperate with other people.
State of the art artificial intelligence could be gaining something like the ability to create tools with control loops. I don’t think anyone is seriously attributing to them an ability for mental time travel. But is that in the cards?
