Content
Introduction and Classic AI
Current AI solutions and architectures based on classical approaches, in our opinion, are approaching their limit.
At the current level of development and efficiency of AI models, creating and training strong AI will require enormous computing power. After all, to increase efficiency, you need to increase the number of neurons and build connections between them. If human neurons can be in several states, and activation can occur “in different ways” (may biologists forgive us for such simplifications), then machine AI cannot do this. That is, conditionally, a machine’s 80-100 billion neurons are not equal to a person’s 80-100 billion. The same GPT4 is estimated at 100 trillion parameters (relatively neurons), and it is still inferior to a person.
This all comes down to several factors.
The first is that AI models will need to become increasingly more complex. And an increase in complexity always leads to reliability problems; the number of points of failure increases. That is, models are becoming more and more difficult to create and to maintain from degradation over time and during operation. They need to be constantly “maintained”, otherwise she will become like an old man with dementia, and her “life span” will be very short.
Let's give a couple of simple examples from life. The first is based on human muscles. When we just start working out in the gym, progress comes quickly, but the further we go, the lower the efficiency and the increase in results. Plus, the increase in strength comes from its thickness (cm^2), but the mass grows from the volume of the muscle (cm^3). As a result, at a certain point the muscle will become so heavy that it will not be able to move itself, and may even damage it.
A similar example can be given in the engineering field. For example, Formula 1 racing. So, a lag of 1 second can be eliminated if you invest 1 million and 1 year. But to win back the decisive 0.2 seconds it may take 10 million and 2 years of work. And the fundamental limitations of the car's design may force us to reconsider the entire concept of a racing car.
If we return to the issue of degradation, then in addition to technology there will also be the influence of people. Any strong AI will self-learn based on feedback, and this, at least at an early stage, will be shaped by people (their satisfaction, initial requests and tasks) and their behavior. An example here is the same ChatGPT4. It uses user requests to improve its answers. As a result, by the end of 2024, users began to notice that he had become “lazy.” They claim that recently, when using the GPT-4 or ChatGPT API, the chatbot either refuses to answer questions, or interrupts the conversation, or simply responds with excerpts from search engines and other sites.
In general, here you can give an example of an anecdote: “I type at a speed of 300 characters per minute! - What, really? - Yeah, it’s just complete bullshit.”
We can easily scale up the current models by XXXX times. But there is a problem - the problem of training large networks and maintaining them. Moreover, ChatGPT5 no longer has enough training data.
The second factor is that the AI model will be tied to its “base”. It will require huge and complex data centers to operate, with powerful energy sources and high-quality cooling. Thus, according to some estimates, 5 - 50 requests for ChatGPT 4 require up to 0.5 liters of water for cooling. And no matter what the Internet bandwidth is, the “brain” will still be in these centers, there will be no distributed networks. Firstly, distributed computing still loses performance and efficiency. Secondly, it is vulnerable to attacks on communication channels.
And if everything comes to the point that strong AI will need a body to interact with the world, then it will be even more difficult to implement this. Then the AI model will be limited, otherwise how will it all be powered and cooled? Where to get data processing power? That is, it will be a limited AI with a constant connection to the main center via wireless communication. And this is vulnerability again. Modern communication channels provide higher speeds, but this affects the range and penetration. In addition, such channels are easier to suppress using electronic warfare. That is, we get an increase in the load on the communication infrastructure and an increase in risks.
In general, strong AI now is not about ESG, ecology or commercial success. Its creation can only be within the framework of strategic and national projects that will be financed by the state.
Here you can, of course, object. For example, the fact that you can take a pre-trained model and make it local. Much the same way as we propose to deploy local AI models with “additional training” to the subject area (we wrote about this in the article about digital advisors). Yes, in this form all this can work on one server. But such AI will be very limited, it will not be a strong AI, and it will be “dumb” in conditions of uncertainty, it will still need energy. That is, this is not a story about the creation of humanoid superbeings. This will be a large number of strong AI, but expensive and with limited capabilities, which is not very interesting to the market.
As a result, we are now learning to effectively and efficiently apply what scientists have come up with and developed over the past 30-40 years. We are now reaching a productivity plateau and mastering current technology.
But is there really no progress? Naturally not. Our view is that the next leap will be in quantum and neuromorphic computing. And perhaps with a combination of them. However, such a hybrid is a prospect for the very distant future.
But what is neuromorphic and quantum computing and why do I think so? Let's look at these areas.
Neuromorphic systems
What are neuromorphic systems
All current IT solutions are based on a simple architecture - there is a processor and there is memory. The processor performs logical calculations and accesses the memory module as needed. This is called von Neumann architecture.
Neuromorphic systems are different in that they try to apply technologies from the field of brain biology. Scientists are working on combining calculations and memory within one block, and the blocks themselves are assembled into complex networks that do not work constantly, but in impulses.
To make it easier to understand, here is a visualization.
Why is this necessary? To solve the problem that we talked about in the block about strong AI - current solutions are complex and consume a huge amount of energy.
For example, for the most powerful computer to recognize a cat or a dog, it needs a thousand times more energy than our brain.
During operation, a typical PC constantly exchanges data between the processor and memory. For example, you need to add two numbers. It takes one number from memory, places it in a processor register, then takes another number and also places it in a processor register. Performs an operation, and from the 3rd processor register the result goes back to memory. Or you can embed a processor for simple calculations directly into memory and give the command: “add two numbers and make a third in memory.” And then we remove the process of data exchange via the bus. And this significantly reduces the cost of time and energy.
As a result, neuromorphic systems make it possible to solve problems where you need to continuously collect data (see, hear, smell), analyze it, and do it quickly and without much energy expenditure. Three key areas can be identified here.
The first is sensors that can operate autonomously, without being tied to the cloud or network.
For example, a detector on clothing that analyzes air toxicity and signals about it. If you make one according to the classical model, then it will either have a data transmission module (then without a network it will be stupid), or it will need a large battery, which will limit the operating time or make it inconvenient to carry and use.
That is, the key advantage of neuromorphism in the first direction is autonomy and the absence of the need for large power sources. This is one of the fundamental problems of smart devices.
The second direction is reducing the time for processing data and making decisions (all current AI models think for a long time).
For example, Boston Dynamics robot dogs. Previously, when pushed, they fell. The problem is that recalculating data after a shock requires processing a large amount of data: you need to collect data from sensors, carry out calculations, and issue commands to drives. As a result, he did not have time to do all this and fell.
Another example is vibration diagnostics based on video analytics. You also need to collect a large amount of data, analyze the video stream, and carry out calculations.
The third direction is an increase in the number of neurons. In the chapter about strong AI, we discussed the problem that current solutions are already difficult to scale. And neuromorphic computing helps solve this problem.
Case Study - Bicycle Control System
Researchers using one Chinese Tianjic chip, without other computing power, made a bicycle control system.
One neural network allows you to perceive voice commands and see. The second controls balance. In general, a whole orchestra of different neural networks based on different architectures, controlled by a certain orchestrator, a master AI, who analyzes tasks into different AI models, collects data and makes complex and quick decisions.
As a result, instead of a bunch of equipment with video cards and processors, a huge battery, there is a small solution. At the same time, the bike moves, avoids obstacles and responds to commands. Fast decision making and autonomous operation based on a single processor.
Flaws
A beautiful prospect, isn't it? But the problem here is the early stage of technology development, and therefore the lack of competencies, resources, and so on.
That is, it is necessary to solve fundamental problems with mathematical algorithms, languages, architectures, and so on. Huge resources of mathematicians and engineers are required. But then there will be an even more difficult task - the organization of pilot and ultimately industrial production. Starting to produce millions of chips, which are currently only being made for experiments, is not an easy task and will not take 1-2 years. And let's remember that now even “classic” equipment is produced worldwide by only a few factories. So this technology will not become widespread very soon.
For those who are interested in materials on this topic, we recommend several interesting and accessible articles and videos using the links below.
Links
Video
Quantum systems
In classic IT systems there are 2 states - on and off. This includes the state of the processor, writing/reading data, and the operation of neurons in the AI model.
In quantum computing, a “neuron” can be both on and off, and in two states at the same time (superposition). It seems like a small difference and doesn’t mean much to the uninitiated. But these are radical changes. So large-scale that scientists are still at the very initial stage of mastering this technology. This technology is even younger than neuromorphic computing.
If we return to the conversation about quantum processors, they can work with larger and more complex AI models, as well as use new AI architectures (where neurons can be in superposition) that are not available to current technologies. And research in this area inspires positive expectations. For example, researchers from Freie Universität Berlin discovered that quantum neural networks are capable of not only learning, but also remembering random data. “It’s like discovering that a six-year-old can memorize random strings of numbers and the multiplication tables at the same time. The experiments show that quantum neural networks are incredibly adept at picking up random data and labels, challenging the foundations of how we understand learning and generalization,” said Elies Gil-Fuster, lead author of the study.
A detailed article is available on the Nature website link
If we aggregate all the advantages into a small list, we get:
higher memory capacity (per volume) and small size (1011 neurons/mm³), which also allows you to increase data processing speed and reduce power consumption;
to solve similar problems, fewer neurons are needed, which again reduces energy consumption and the complexity of the IT infrastructure;
faster learning from less data;
it is possible to solve problems that are currently inaccessible.
If you look at it from a different angle, it comes out as:
absence of catastrophic forgetting of context;
solving linearly inseparable problems with a single-layer network;
higher stability and reliability of the AI model;
the ability to create “multidimensional” AI models that process multiple data streams.
In general, here we begin to go into the world of fantasy. And if we combine neuromorphic and quantum technologies, then I’m afraid we won’t have enough imagination.
Flaws
Well, now let's return from space to earth and understand why technology has not yet caused a revolution in AI.
Firstly, the first research in this area appeared only in 1995. By the standards of science, technology is still in its infant stage. As we wrote above, the technology is even younger than neuromorphic computing. This means that there is not the required number of specialists for the rapid development of technology.
Secondly, quantum computers themselves are still at a very early stage. These are solutions that you cannot buy for your home or organization. Quantum computers, although they are developing quickly, are still about the same now as we saw with PCs in the 1970-1980s. That is, now, as with neuromorphic systems, the most basic problems related to mathematics and physics are being solved. Now researchers are asking questions like:
what hardware resources (hardware, number of physical and logical qubits) are required to solve a particular problem;
how long will it take for calculations;
how to write basic algorithms;
how to ensure processor stability and remove interference that affects qubits.
At the moment, this is the lot of scientists and the military, and entry into large business can be expected in 5-20 years, large-scale distribution in 30-50 years.
Below is an interesting visualization of one of the forecasts for the development of quantum technologies.
Thirdly, quantum computing is not very well suited for solving everyday problems. It’s like plowing potatoes using a spaceship: the preparation will be too difficult, and the speed of the solution will not change much. In the end, it doesn't make sense. A quantum computer is not a supercomputer that does everything faster. And one of the key tasks at this stage is to understand what problems can be solved by a quantum computer faster than a classical PC, and how much more useful it can be.
And at the moment, quantum computers are performing well in solving problems that require calculating a large number of possible combinations. These types of problems occur, for example, in quantum simulation, encryption, and search problems. For example, D-Wave scientist Zhengbing Bian used one of the company's computers to solve a resource-intensive computing problem - constructing a two-color graph of Ramsey numbers. A typical average computer would take more than 10,000 years to solve this problem. But the D-Wave quantum computer needed only 270 milliseconds.
As a result, for those who are interested in this area, we recommend starting your dive with the articles and videos from the links below.
Links
Video
Yes, quantum and neuromorphic networks will be the next revolutions in AI. And the development of strong AI can be associated with the development of precisely these technologies.
But these technologies are now at such an early stage of development that it is impossible to make predictions here. It is the cutting edge of science, research for the future and a destination for promising investment. Quantum and neuromorphic technologies themselves will not be able to solve applied problems entirely in the coming years. But, given their capabilities, we should expect the development of some quantum-neuromorphic-classical solutions.
The main advantage from the development of these technologies is that AI models will become less demanding in terms of the amount of computing power and energy, and will be more “smart.” As we previously looked at, the same ChatGPT 4 already has many more neurons than a person, it is a huge model, but it consumes a huge amount of energy and is maintained by a whole staff of very expensive specialists. However, it cannot be called more intelligent than a person. And this despite the fact that he was trained on such volumes of data that no one has ever received at his disposal.