Brain and artificial neural networks ​

People have long been interested - how is it all arranged in our heads? Are there words written in the brain? Or are pictures drawn there?

Neurobiology studied these questions, but its methods were limited, research on the brain down to individual neurons is extremely complex, practically impossible. A breakthrough in this area occurred only in the last decade, with the development of artificial neural networks, which have already earned the name "artificial intelligence." Although computer neural networks are built on a completely different computing hardware, unlike the human brain, at a fundamental level they perform calculations on the same principles, so research on computer models has confirmed the conclusions of neuroscience and greatly advanced it.

In a neural network (artificial or natural), there are no analogues of files with texts or archives of photos (this was determined by neurobiologists quite a while back).

Data about the world in the form of texts (written or spoken words), images (videos), sensations, etc., enter the neural network from the sense organs (or from a disk, or from sensors, input devices) and undergo another transformation (they are encoded) there.

Information in the neural network is stored as a huge number of changing connections between neurons and their weights (connection strengths). Information about one sign is distributed over a large area of the brain, information about the next sign is in another area, overlapping with the first, information about an image or a whole situation covers even larger areas. As a result, each new second of learning (our life) leads to changes in the entire volume of the neural network, something in it is preserved in a transformed form, something disappears, something is distorted.

The representation of ideas with this understanding of the principles of the neural network has also turned out to be quite formal. We have already discussed it when discussing the approach of computational linguistics to conceptual space and its density. The points mentioned there in a multidimensional space - that is the result of the neural network's work. Based on the processing of some information, the brain neurons (or only a part of it) come to a certain state, and a group of such similar states (a cloud of points) is perceived as "justice," while another group of states, which the brain can enter after receiving different information, is perceived as "sadness."

Although neurobiology has long understood these processes in general, it was very difficult to imagine them, and the power of our brains as a computing system remained a mystery. Could such a seemingly chaotic process eventually generate our thinking, our ability to remember, act, work with signs and ideas, be creative? Experiments with artificial neural networks have confirmed - yes, everything is indeed so. Even a computer neural network, which is tens of thousands of times less computationally powerful (in terms of the number of neurons) than the brain, is capable of demonstrating extremely complex thinking. It may not be creative, may not even be intellectual, but it will be a full-fledged operation with signs and images without the ability to store and process them as a regular computer does or as people do in the exocortex (on paper or on a computer screen).

Distributed representations go beyond the scope of our course, we will not discuss what happens inside agents (humans, animals, computers). We will simply state "we see," "we imagine," "we interpret," "we think," understanding that inside the neural network there are huge computations, built according to special rules, very different from what happens outside our brains.

Outside, external representations and communication between agents remain symbolic, so we will continue discussing sign systems.

However, the cutting edge of scientific research and modern technology today lies precisely in the field of neural networks, and the understanding of their relationship with sign systems is far from complete. Perhaps, someday direct connections between neural networks will make all our theories about signs and meanings in text (verbal) communication simply unnecessary.