The Imagination Stack: the missing skills for an age of AI
Existential Hope in Practice
In the Existential Hope in Practice essay series, our fellows write about concrete ideas and practical examples of how to build a great future.
The Imagination Stack: the missing skills for an age of AI
By Keith Patarroyo
In brief:
Everyone agrees we need better ideas about the future. Almost no one treats imagination, the muscle that generates them, as the thing to train.
Imagination can be broken into trainable skills. Here are five to start with: creativity, mathematical intuition, cognitive artifacts, taste formation, and serendipity.
Einstein chose to put marginal ideas at the center of his theories. That ability to select what matters is taste, and it is what the internet age failed to distribute.
AI trained on human content is, at best, a very sophisticated communication technology. It cannot self-reflect, continually learn, or predict the consequences of its actions.
A civilisation that puts imagination at its foundation is the natural operating system of an age of strong AI, and the only one where humans remain irreplaceable.
For many years, I’ve been fascinated by the power of culture, especially of cinema, to transmit important ideas and, perhaps in a more mysterious way, to convey dreams of possible worlds. For that same reason, I’ve been concerned by the prevalence of either utopian or dystopian visions of the future in most of our media. Stories that portray a genuine middle way remain very much the exception. The lack of genuinely hopeful stories has led me to conclude that society needs to radically improve its capacity to imagine. This is easy to diagnose, but difficult to make practical. However, I’ve realised recently a narrow path forward, a path that puts imagination at the roots. This program holds the promise of helping society solve the most pressing problems and become the new operating-system in the upcoming AI age.
The architecture of imagination
The vocabulary of imagination is extremely polluted; we have words like create, invent, fantasize, daydream, dream, self-help … Moreover, they are extremely poorly defined, and depending on the context, they may be understood as child development, or as scientific breakthroughs, or as day-to-day planning, or conspiracy theorizing. There are genuine efforts to generate a taxonomy of imagination and to properly study and define exactly this process. However, here I’ll attempt to convince you not of the study of imagination but rather of embracing it in your personal journey; imagination must be understood fundamentally by practicing it. Next, I want to sketch some abilities that I think reflect the power of imagination, and then in the second part of the text, we will imagine a future where they are at the roots. Here are five practical skills: creativity, mathematical intuition, cognitive artifacts, taste formation, and serendipity.
Creativity
The secret to creativity is simple: make a lot. Whatever field you want to be creative in, the key is to be constantly making new things in that field. Constancy is your friend, and perfectionism is your enemy. Once you have become used to generating a lot, you will learn to feel when one of the things you have generated is the correct way to go. Therefore, if you want to become a creative writer, write about anything every day, no stakes, just having fun, and eventually you will have very creative texts.
Five years ago, I set myself to become a great creator. I had always admired friends, colleagues, and idols who were so productive, generative, and creative. Despite all of my efforts before that period, something was always missing. I was never able to focus on something and generate things with it for a long time. However, one day five years ago, I found the solution in the most unlikely of places. The words of Rick Ruben, a music producer, unlocked in me the secret of creativity. In the months and years that followed, I became a lot more productive, a lot more disciplined, and most importantly, I learned how to feel when I was experiencing the creative process and use it to help me solve problems that I’m currently facing.
Mathematical intuition
Recall the beautiful piece of the Little Prince where a kid draws a contour that to all adults looks like a hat, but he sees a frightening boa that has just eaten an elephant. Imagination in mathematics is just that, it is the return to a child-like state of freedom, where one is free to make one’s own whatever mathematical concept one is facing. More specifically, it is a constant back and forth between logic and intuition. It is the constant process of refining a cognitive model of a mathematical object. This is fundamentally a very slow process, taking days, weeks, or even years, but its compound effect leads to the most powerful tool in these skills.
The power of the mind is unparalleled; historical figures like Ramanujan, von Neumann, or even contemporary figures like Perelman or Terence Tao show us that the mind has orders of magnitude of depth to explore. But what if I tell you that you can obtain that power? And one of the keys is to train your imagination. This is the thesis of mathematician David Bessis. He claims that most math barriers are psychological and that mastering several techniques to overcome your inhibitions is the fundamental step in a mathematician’s journey. This is a very provocative claim, and fundamentally subjective. However, the power of creation seems to have that very peculiar property. Therefore, the only way I can attempt to prove this to you is again by trying to become a great imaginator myself.
Cognitive artifacts
Master a musical, astronomical, or calculation instrument. This could be a string, brass, or electronic musical device, or an astrolabe, a sextant, a slide-rule, an abacus, etc. The first step to make a cognitive structure is to take a real-world object or an action and abstract it to generate a first- order representation. In this way, you will increment your repertoire of cognitive models in order to be used for imagination purposes.
Tools have been instrumental in the development of civilization, not only because they provide a specific use, but rather because they promote the creation of novel cognitive structures. These structures are part of the vast catalog of external representations that may be used as a correct mental representation for imagination. One instance of this invention is the esoteric programming language Orca, a language that started with the aim of being a live-coding language for musical composition. It has developed into several applications, but fundamentally, it is a way to conceive of complex timing operations. This is just one of multiple technologies that promise to enhance human cognitive capabilities.
Taste formation
You need to get taste. In whatever your creative endeavor, you need to choose what to pay attention to and what not to. A good hack is to find a mentor and work on a project together. Because new important ideas are always lying out there, but they are usually the marginal ideas of very important thinkers. The starting point for a new system of knowledge is often found within an existing conceptual network when it is being picked up at a different point.
Scientific taste is a skill that seems to be tacit knowledge; in fact, it has been proposed as the reason why the internet age didn’t generate a bunch of Einsteins. It is, in fact, illustrative to recall Einstein’s discovery of special relativity; he was very much part of a community of practice, and he was aware of the former work of Poincaré and Lorentz. He took the marginal ideas of these thinkers and put them as central in his theory. So Einstein’s greatest asset was not his supply of knowledge, but rather his imagination. Scientific taste, imagination, and creativity are precisely what Einstein practiced at an extraordinary level. So, in a future where imagination is put at the foundation of knowledge, we will have a bunch of Einsteins.
Serendipity
Constantly do new things. Serendipity is not an accident; it is a feature of search in very complicated spaces. Aiming for novelty alone is sufficient for solving very complex problems. You should also abandon objectives or benchmarks. Only brute force problems are solved by optimizing an objective. These are the low-hanging fruit that AI is going to storm. However, pursuing novelty has the secondary effect of generating complexity and thereby has an accumulative effect. The skill of knowing when to try something new is also non-trivial, and it is related to a strong sense of personal criticism or the ability to doubt and ask questions.
For the past five years, I investigated how different fields look at creation, and very surprisingly, I came across the field of Open-Endedness. This marginal subfield of AI has captured great insights into the power of creation. Algorithms like novelty search, quality diversity, or NeuroEvolution capture deep mechanisms of novelty generation. This, however, is only a partial solution to the creation of truly creative algorithms. Other topics remain very much open questions, like the nature of creativity, compositionality, continual learning, etc. To my knowledge, there has been no AI system that has generated any true transformational invention. I’ll mention more on this in the interaction of AI and imagination.
Imagining the future
The understanding of imagination promises a unification of knowledge that has been promised for decades. The union of science and the humanities. The practice of imagination is crucial in both fields, and understanding how a novel idea comes into being without mysticism is much needed today. However, reorganizing the structures of society for this revolution will take decades. I presume that it will take at least two generations(~40 years) to have a community of practice that is robust enough to transmit its tacit knowledge to absolute novices.
In a future of constant restructuring of the mind through imagination, the diversity of thought has exploded. People are able to enter states of thinking on command, or with the aid of hallucinogenic drugs, which might, for example, be able to induce neurodivergent ways of thinking for specialized purposes. These ways of thinking would not only solve some of the pressing problems of the day, like climate catastrophes, human disease, and population collapse, but they would open the doors to the atom, the cell, and the cosmos.
However, imagination can also lead us astray; current dangers include conspiracy theories and paranoia. In such a future, imagination at scale could also be weaponized, just like today we have memetic warfare. In the future, there might be weapons of mass psychosis that would rewire the brain to believe in delusions. Therefore, responsible development should be advocated, as well as proper education and defensive mechanisms. Keeping this in mind, we can steer the development of this paradigm to be a broadly beneficial technology.
Imagination in the AI age
We are on the cusp of a new age, and with the technologies available right now, we are disrupting some of the foundations of society. However, the paradigm upon which AI is built today can be better seen as artificial communication. The digital content produced by humans is only but a slice of its cognitive state. AI tools provide mostly a better and more efficient way of communication, but they are fundamentally not thinking as much as a piece of paper with text on it is not thinking. Therefore, current AI is fundamentally incapable of imagination as we conceived before. However, this doesn’t mean that AI cannot generate useful objects or that AI in some future can become imaginative.
Many people are working on an imaginative AI, including some of my own efforts. As we mentioned before, the field of Open-Endedness offers several clues on possible paths to build a truly creative AI. Promising approaches consist of World Models, Self-Play, Robotic Intelligence, AI self-driving labs, Artificial Living systems, and more. Moreover, a deeper understanding of human imagination might even jump-start these systems into becoming truly thinking machines. However, even if this is the case, these AI systems cannot be you. They cannot perfectly encode your evolutionary and cognitive history. Your novelty, your unique perspective, and your imagination are the only things that are left to you to exploit. Nurturing this is the only way to never be replaced, so I call upon your future and your kids’ future to exploit your imagination.
We began with the observation that our culture struggles to imagine hopeful futures. Perhaps that is because the people making culture have not yet had their imaginations fully liberated. I have given you five fundamental skills to help solve this problem. Moreover, if technology is meant to fulfill its emancipatory purpose and liberate us humans from thinking about work and repetitive tasks, and focus us on the truly creative endeavors. Then, a civilization based on imagination as a core virtue is the natural operating system of an AI future. In this way, both Humans and AIs will contribute to the growth of knowledge and the spread of Intelligence through the universe.
About the author
Keith Patarroyo is a research fellow in chemical intelligence at the University of Glasgow. His research interests center around the idea of Open-Endedness. What is the true nature of open-ended systems? How can we instantiate them in chemistry, in materials, or in silico? and the development of algorithms for the exploration of large combinatorial systems and automated scientific discovery. He was also a Research Affiliate of the Wolfram Physics Project, where he worked on discrete computational models of physics. He also has a deep fascination for cellular automata, and led a project to map the creativity of the enthusiasts that create all sorts of complicated patterns online.
 | A guest post by
|