Piero Scaruffi

These are excerpts and elaborations from my book "The Nature of Consciousness"

Computational Models of the Brain In 1943 the US physiologist and mathematician Warren McCulloch, in cooperation with Walter Pitts, wrote a seminal paper (“A Logical Calculus of the Ideas Immanent in Nervous Activity”) that laid down the foundations for a computational theory of the brain.  McCulloch transformed the neuron into a mathematical entity by assuming that it can only be in one of two possible states (formally equivalent to the zero and the one of computer bits). These "binary" neurons have a fixed threshold below which they never fire. They are connected to other binary neurons through connections (or "synapses") that can be either “inhibitory” or “excitatory”: the former bring signals that keep the neuron from firing, the latter bring signals that want the neuron to fire. All binary neurons integrate their input signals at discrete intervals of time, rather than continuously.  The model is therefore very elementary: if no inhibitory synapse is active and the sum of all excitatory synapses is greater than the threshold, then the neuron fires; otherwise it doesn’t. This represents a rather rough approximation of the brain, but it can do for the purpose of mathematical simulation.  Next, McCulloch and Pitts proved an important theorem: that a network of binary neurons is fully equivalent to a Universal Turing Machine, i.e., that any finite logical proposition can be realized by such a network, i.e., that every computer program can be implemented as a network of binary neurons. Two most unlikely worlds as that of Neurophysiology and of Mathematics had been linked. It took a few years for the technology to catch up with the theory. Finally, at the end of the 1950s, a few neural machines were constructed. Frank Rosenblatt's "Perceptron" (1957), Oliver Selfridge's "Pandemonium" (1958), Bernard Widrow's and Marcian Hoff's “Adaline” (1960) introduced the basic concepts for building a neural network. For simplicity purposes a neural network can be structured in layers of neurons, the neurons of each layer firing at the same time after the neurons of the previous layer have fired. The input pattern is fed to the input layer, whose neurons trigger neurons in the second layer, and so forth till neurons in the output layer are triggered at last, and a result is produced. Each neuron in a layer can be connected to many neurons in the previous and following layer. In practice, most implementations had only three layers: the input layer, an intermediary layer and the output layer. After a little while, each layer has "learned" something, but at a different level of abstraction. In general, the layering of neurons plays a specific role. For example, the wider the intermediate layer, the faster but less accurate the process of categorization, and viceversa. In many cases, learning is directed by feedback. "Supervised learning" is a way to send feedback to the neural network by changing synaptic strengths so as to reflect the error, or the difference between what the output is and what it should have been; whereas in "unsupervised" learning mode the network is able to learn categories by itself, without any external help.  Unsupervised learning became feasible after the introduction of algorithms such as Teuvo Kohonen's Self-Organized Maps (1982), Geoffrey Hinton's and Terry Sejnowski's Boltzmann Machine (1983), Stephen Grossberg's Adaptive Resonance Theory (1987) although the idea dates back to British mathematician Albert Uttley’s “Informon” (“The informon: A network for adaptive pattern recognition”, 1970). Whether supervised or not, a neural network can be said to have learned a new concept when the weights of the connections converge towards a stable configuration.   Back to the beginning of the chapter "Connectionism and Neural Machines" | Back to the index of all chapters