Neural networks in AI less connected than expected

Artificial neural networks continue to pose puzzles. It is difficult to decipher exactly how the information is compiled, the texts are created, why major errors creep in and why some chatbots simply break down.

One reason for this is that the basic structure of these networks is similar to that of a brain. It is not the data points and individual pieces of information that characterize the way they work, but the almost infinite number of possible links between them.

However, the neural networks of artificial intelligence often remain at a much simpler level, as a recent study at https://arxiv.org/pdf/2308.09124.pdf has found out.

AI deals with basic information in a much less complex way than previously expected. For example, instead of collecting all possible answers to a query and responding with the most likely, because most frequently linked, fact, a linear link is used.

The researchers were able to identify a total of 47 different linear links in the artificial intelligence under investigation. This was achieved by means of a corresponding query and the subsequent tracing of the signal path through the neural network.

And this was surprisingly short for many facts. For example, questions about the instrument played by well-known artists, the type of sport played by professional athletes or even the capitals of individual countries could be answered with a single link. In contrast, it is more difficult to find answers to less obvious questions in the network.

Nevertheless, it could be shown that a lot of information is directly linked to each other and is hardly organized as a network. Understanding how such a chatbot outputs its knowledge and how the signal can be reliably traced in the network should in any case help to understand the sometimes curious, but sometimes difficult to recognize errors of an AI.

The insight into how it works with its surprisingly linear structure alone is worth a lot.