Natural Selection for AI

Author(s): Valeria Fonseca Diaz

Originally published on Towards AI.

Now that AI is officially born and being raised, it is almost impossible to stop ourselves from having philosophical discussions about its meaning and its impact. We, humans, need to define our relationship with AI, we cannot ignore it. The path is still long for most of us. And yet, while I dump my words and thoughts here, there’s a machine out there that does this and much more in a fraction of the time. Hopefully, also, in a fraction of its creative and moral value.

What gave AI birth?

The answer to that question is not different from the evolution process of other entities. AI came to be what it is today after years of research, experimentation, and joining the forces of Statistics, Mathematics, Optimization, and computer power. It was first only one neuron making binary predictions. Then it became several neurons making predictions of several classes. Then it was a bunch of layers of neurons figuring out classes that they didn’t even see before. And now, AI is many times more productive than a human brain, capable of telling humans what to do.

We created AI. We, humans, gave birth to this new way of intelligence. In our excitement for how fun and interesting it seemed to be to create a new type of intelligence, AI grew to be far more real than just fun.

But maybe we didn’t just create the whole thing. Did we really create it, did we discover it, or did it evolve naturally? The answer might just not be so trivial or simple. Just as we don’t know if we invented or created math, the process of obtaining AI can be just as well a complex mechanism combining elements of our creation and elements of our discovery.

Regardless, AI evolved. It grew from simple math elements to complex algorithms. Two elements are the fundamental pieces in the evolution process of artificial intelligence.

Let’s recall for a moment the history of Statistics as one of the initial states of artificial intelligence. Linear regressions emerged some centuries ago, joining observations registered as data, a regression function, and an optimization problem to obtain the regression function. Very few data points and simple computational capacity were needed at the time to make linear regression become a staple mechanism for understanding a phenomenon. Not even a computer was necessary to obtain the regression function parameters given a set of data points. The origins of AI were very well handled with pencil and paper and a calculator at most.

Regardless of its simplicity, linear regression emerged from data, from a regression function, and from the possibility of calculating it (solution and computation of the optimization problem).

As of 2024, AI does not look at all as simple as linear regression, but their evolution process is comparable: data and computation of the optimization problem. While, evidently, they are not the only elements that played a role in the evolution of AI, it is to argue that they are the fundamental pieces selected for the development of AI. They are the elements that define the level of capacity of AI. Without them, AI would cease to exist just like living things would do without food. The concept of data might be easier to make sense of, but when it comes to “computation of the optimization problem”, things got very interesting during this evolution time.

Data

From registered observations in pieces of paper, to Microsoft Excel, to databases, to the whole world wide web, data is nowadays an ocean containing the registry of experience. We started registering data to uncover patterns of different mechanisms through the different sciences. Whether in physics, biology, or psychology, we used registered data since the origins of early Statistics to understand connections among variables and causality patterns.

Thanks to these recorded observations, we have unveiled thousands of secrets of the atom and the universe. Stephen Hawking did not live to see the image of a black hole deducted from billions of data registries of the light and energy activity by an international network of radio telescopes called the “Event Horizon Telescope” (EHT). After so many years of his dedicated and thoughtful research about black holes, the first real image of one of these objects was probably a deserved experience for him. Thankfully, we did get to see such an object. But for what matters in our current conversation, without data, and so much of it, and its complexity of registration, the image of a real black hole would not have been possible. Once again, it has been from recorded observations that we have unveiled thousands of secrets, from the atom to the entire universe.

Data is also to AI what food is to humans. It’s in-taken, processed, and finally used for something. With that said, there’s one possible way of defining AI, and that is “the capacity to digest billions of data to emit one decision in a small fraction of time”. AI’s life consists of making constant decisions: a prediction, creating statements, creating images, finding the hidden pattern, etc. If we compare the level of capacity of the human brain to emit one similar decision given billions of possibilities, we might be able to achieve it, but unfortunately the processing time would be just a bit longer than a fraction of a second or a minute. Regardless of our differences, we do have many things in common and one of them is our need for some input material. Data is to AI what food is to humans: it would cease to live without it.

AI is the capacity to digest billions of data to emit one decision in a small fraction of time

ChatGPT was the major democratized breakthrough of artificial intelligence. Before it, other AI solutions existed but were not in the hands of everyone. The average human being with access to a computer finally experienced the meaning of AI with the launch of the interface for textual processing of the GPT model launched in November 2022. What data was used to train this model? A very clear list of data domains is disclosed in the GPT-2 GitHub repo (See here). In a nutshell, the whole WWW was scrapped grabbing our actions, opinions, knowledge, reactions, and so much more.

Before we realize it, the data has become so diverse and big that AI will derive all the secrets of the physical world.

In the first versions of ChatGPT, when asking for recent facts or results that have emerged after the registered data used for its training, it very politely and robotically explains that those facts are not available at the time of its training. If ChatGPT is not fed with recent data, the claims it creates become outdated and likely invalid in time. This is how data acts as the food source of this type of AI.

But as we said, data is not just the energy source of AI, it is selection for AI. As more data becomes available, in time data also becomes more diverse than it was before. The processes of the universe are transformed in time and this information is hidden in the data that we register of our phenomena. Uncovering those hidden patterns is what demarks the evolution of artificial intelligence entities. Today, ChatGPT can answer questions, explain facts, and extract summaries of long documents. Tomorrow it can receive a research hypothesis and deliver a full thesis proving or disproving the hypothesis, or a thesis of a reformulated hypothesis because the initial hypothesis the human formulated did not make much sense. Before we realize it, the data has become so diverse and big that AI will derive all the secrets of the physical world.

But, as far as data was concerned, it did not act alone in the evolution of AI.

Software

If you are not part of the AI community in general, have you wondered how something like ChatGPT actually comes up with so much sensible, almost accurate, textual content? The confidence with which this machine can provide information to answer our requests is something a human needs to build with time following a long path of hard and deep work.

The second responsible element for the evolution of AI is the refinement of the software. I mentioned before that aside from data, there was something called “the computation of the optimization problem”. A model such as a Generative Pre-trained Transformer (GPT) is a mathematical mechanism that processes an input to create an output concept. In the case of the model behind ChatGPT, it receives a query as input (“write an essay about topic x”) and it processes this query deeply to create an entire textual output answering the request. The way this machine processes this query is something that needs to be trained first. Just like when certain living entities are born, they have brains and they need training to learn things. Training a computer so it learns how to process future queries is far from a trivial task.

Richard Stallman was the creator of the so-called free software. The slogan to define the essence of this type of software since its origin has been “free as in freedom, not as in free beer”. With the growth of personal computer technology in the 70’s and 80’s, a key business opportunity came about selling the software running the machines separately from its hardware. With that, one single physical machine would represent income from every piece of software that it contained. Running a Windows machine required buying a license for the operating system. After that, writing a formatted document would require a user to purchase another license for Microsoft Word. This business model was the same for other types of software to run other processes, like printing, making calculations, drawing, etc.

The license between the user and the software has always been a barrier. Whether it is a positive or a negative barrier is another topic. However, the existence of this barrier did not allow the user to make any adaptation of a piece of software for a new computational feature. This meant that innovation in the capacity of software was very limited and subject only to the availability of the software owner.

Stallman established the concept of free software as software that can be used, copied, modified, and re-distributed, without liability to the original developer. Free software did not mean gratis. It meant to have the freedom to transform it. Now we see where this is going.

Training a model for a complex AI task requires software features emerging from different disciplines. Complex mathematical formulations, numerical solutions, fast optimization algorithms, programming languages of fast compilation, and scripting environments, among many more. When joining the efforts of all these disciplines, the necessary software to train these complex models was not a linear evolution that could come from a single private corporation. It emerged from the invisible force of the transformation of free software. Who transformed it? Communities, experts, and enthusiasts who contributed to the contributions of others. No wonder why a few years ago Microsoft bought GitHub after decades of refusing the concept of free software.

GPT models have a foundation in the dominant and advanced Python libraries of deep learning TensorFlow and PyTorch. Both of these software solutions are open source and have been in evolution since their release between 2015 and 2016. The parent of the model running behind ChatGPT OpenAI, a pioneer in popularizing the use of AI technology, developed its first versions of the GPT models and image generator models using these established open-source frameworks which already gave a solid landscape. So to this point, it is still amusing to imagine where we would be right now with AI, had open-source software not existed.

At this point, it is worth having another thought bubble to acknowledge and differentiate the contribution of Richard Stallman. While I have been using the concepts “free software” and “open source” interchangeably, they are by no means carrying the same fundamental meaning. The concept of free software, as originally defined in the General Public License (GNU GPL) series, had the spirit of freedom for the use, copy, modification, and redistribution of software guaranteeing its longevity as free software. This means that free software under GPL licenses shall remain free upon modification or redistribution. This is what has been known as copyleft licenses.

So to this point, it is still amusing to imagine where we would be right now with AI, had open-source software not existed.

OpenAI originally used and intended to develop this generative AI technology with a free software approach. However, the licenses that regulate software such as TensorFlow and PyTorch are of permissive nature, which was the perfect combo for OpenAI to achieve their current potential and closing the software right after crossing the peak moment.

Under a proprietary software paradigm, training an AI machine like the ones we are welcoming now would have been impossible. The changes these models and software needed to support more complex tasks would have required waiting for the proprietaries to release more versions. Under a free software paradigm, big changes in software capacity may become available in a few days. Nowadays, the dominant software that supports deep learning is open-source software. Just as in the case of data, the life of AI depends on and evolves with the evolution and availability of free or open-source software.

Data and software only?

We can ask now, how are data and free/open source software selecting for the evolution of AI more than other features that also play a crucial role in it? Naturally, these two features are not the only ones that AI needed to become what it is today. Powerful hardware is one of them. While fast algorithms and efficient programming languages are one necessary condition, they would play a null role in practice without the existence of powerful hardware. Graphical processing units, exponential increase of RAM, high-performance computing, etc., are all necessary elements to develop and run these complex models. So, where is the difference?

It’s all about the invisible forces. To develop powerful hardware, big funding and sufficient tangible material is needed. These resources are assets that big private corporations can buy. This is not the case for diverse data and powerful software. The diversity and complexity of data is a quality that money alone cannot buy. Data is a registry of human and natural experiences. The diversity of natural experience is created by all the invisible forces that act around us. The same is true for powerful software. The contributions of so many experts and enthusiasts make the software become invisibly more solid and advanced. Here again, this diversity and complexity is something that money alone cannot buy.

What will happen next with AI?

Until now, we have been using artificial intelligence solutions in a rather predictive, static way. Now, those entities that we trained in the past are learning from their own mistakes because we reinforce their behavior based on the predictions they have made. Now those entities are coming up with ideas and solutions that were hidden from the human mind before. AI has evolved to a level that it constitutes a dynamic entity. While it still goes on with human guidance, it surpasses humans in its ability to generate knowledge that is hidden from us.

AI is incorporated into human daily life. It will continue coexisting with us and will start guiding our actions and interactions. The more hidden patterns of the universe are to us, the more power artificial intelligence will gain because we will feed more experience into a type of intelligence that has proved capable of unveiling what is far from obvious. The more hidden the patterns, the more AI has an opportunity to learn something else. The moment this opportunity meets diverse enough data and software, selection for new capabilities of AI will happen.

As these words are written, generative AI and other types of artificial intelligence continue to improve and grow their capabilities and find their way into our daily lives. Our previous generations had to compete with physical force natural to other species who have physical abilities that humans don’t. The biggest uncertainty now comes about the question of whether our current and future generations will need to compete with AI systems that can think faster than us. Ideally, AI would be a tool for humans that increases our efficiency and accuracy. With the fast evolution of AI, we might be making an independent entity out of it that can take control easily away from humans. Yet there, it will do so for as long as diverse-enough data and software exist.

Great sources that inspire ideas

• Madhumita Murgia, Code Dependent

• Richard Stallman, Free Software, Free Society: https://www.gnu.org/philosophy/fsfs/rms-essays.pdf
• https://www.deeplearning.ai/the-batch/issue-229/
• https://www.theredhandfiles.com/chat-gpt-what-do-you-think/

Interested in following these discussions? Looking forward to your comments!

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI