The network of the human nervous system is composed of nodes (for example, neurons) that are connected by links (for example, synapses). The connectivity may be viewed anatomically, functionally, or electrophysiologically. These are presented in several Wikipedia articles that include Connectionism (a.k.a. Parallel Distributed Processing (PDP)), Biological neural network, Artificial neural network (a.k.a. Neural network), Computational neuroscience, as well as in several books by Ascoli, G. A. (2002),Ascoli, G.A. (Ed). (2002). Computational Neuroanatomy: Principles and Methods. Totowa, New Jersey: Humana Press. Sterratt, D., Graham, B., Gillies, A., & Willshaw, D. (2011),Sterratt, D., Graham, B., Gillies, A., & Willshaw, D. Ch 9 (2011). Principles of Computational Modelling in Neuroscience, Chapter 9. Cambridge, U.K.: Cambridge University Press. Gerstner, W., & Kistler, W. (2002),Gerstner, W. and Kistler, W. (2002). Spiking Neuron Models, Chapter 9. Cambridge, U. K.: Cambridge University Press. and David Rumelhart, McClelland, J. L., and PDP Research Group (1986)Rumelrhart, D., McClelland, J. L., & the PDP Research Group (1986). Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume 1: Foundations. Cambridge: The MIT Press. among others. The focus of this article is a comprehensive view of modeling a neural network (technically neuronal network based on neuron model). Once an approach based on the perspective and connectivity is chosen, the models are developed at microscopic (ion and neuron), mesoscopic (functional or population), or macroscopic (system) levels. Computational modeling refers to models that are developed using computing tools.
Introduction The nervous system consists of networks made up of neurons and synapses connected to and controlling tissues as well as impacting human thoughts and behavior. In modeling neural networks of the nervous system one has to consider many factors. The brain and the neural network should be considered as an integrated and self-contained firmware system that includes hardware (organs), software (programs), memory (short term and long term), database (centralized and distributed), and a complex network of active elements (such as neurons, synapses, and tissues) and passive elements (such as parts of visual and auditory system) that carry information within and in-and-out of the body.
Although highly sophisticated computer systems have been developed and used in all walks of life, they are nowhere close to the human system in hardware and software capabilities. So, scientists have been at work to understand the human operation system and try to simulate its functionalities. In order to accomplish this, one needs to model its components and functions and validate its performance with real life. Computational models of a well simulated nervous system enable learning the nervous system and apply it to real life problem solutions.
It is hypothesized that the elementary biological unit is an active cell, called neuron, and the human machine is run by a vast network that connects these neurons, called neural (or neuronal) network.Ramon y Cajol, S. (1894). Textura del Sistema Nervioso del Hombre u los Vertebrados. English translation by Swanson, N. & L. W. (1994). New York: Oxford University Press. The neural network is integrated with the human organs to form the human machine comprising the nervous system.
Network characteristics
The basic structural unit of the neural network is connectivity of one neuron to another via an active junction, called synapse. Neurons of widely divergent characteristics are connected to each other via synapses, whose characteristics are also of diverse chemical and electrical properties. In presenting a comprehensive view of all possible modeling of the brain and neural network, an approach is to organize the material based on the characteristics of the networks and the goals that need to be accomplished. The latter could be either for understanding the brain and the nervous system better or to apply the knowledge gained from the total or partial nervous system to real world applications such as artificial intelligence, Neuroethics or improvements in medical science for society.
Network connectivity and models
On a high level representation, the neurons can be viewed as connected to other neurons to form a neural network in one of three ways. A specific network can be represented as a physiologically (or anatomically) connected network and modeled that way. There are several approaches to this (see Ascoli, G.A. (2002) Sporns, O. (2007), Connectionism, Rumelhart, J. L., McClelland, J. L., and PDP Research Group (1986), Arbib, M. A. (2007)). Or, it can form a functional network that serves a certain function and modeled accordingly (Honey, C. J., Kotter, R., Breakspear, R., & Sporns, O. (2007), Arbib, M. A. (2007)). A third way is to hypothesize a theory of the functioning of the biological components of the neural system by a mathematical model, in the form of a set of mathematical equations. The variables of the equation are some or all of the neurobiological properties of the entity being modeled, such as the dimensions of the dendrite or the stimulation rate of action potential along the axon in a neuron. The mathematical equations are solved using computational techniques and the results are validated with either simulation or experimental processes. This approach to modeling is called computational neuroscience. This methodology is used to model components from the ionic level to system level of the brain. This method is applicable for modeling integrated system of biological components that carry information signal from one neuron to another via intermediate active neurons that can pass the signal through or create new or additional signals. The computational neuroscience approach is extensively used and is based on two generic models, one of cell membrane potential Goldman (1943) and Hodgkin and Katz (1949), and the other based on Hodgkin-Huxley model of action potential (information signal).
Modeling levels
Sterratt, D., Graham, B., Gillies, A., & Willshaw, D. (2011) classify the biological model of neuroscience into nine levels from ion channels to nervous system level based on size and function. Table 1 is based on this for neuronal networks.
Level Size Description and Functions Nervous system > 1 m Total system controlling thought, behavior, and sensory & motor functions Subsystem 10 cm Subsystem associated with one or more functions Neural network 1 cm Neural networks for system, subsystem, and functions Microcircuit 1 mm Networks of multilevel neurons, e.g., visual subsystem Neuron 100 μm Elementary biological unit of neuronal network Dendric subunit 10 μm Arbor of receptors in neuron Synapse 1 μm Junction or connectivity between neurons Signalling pathway 1 nm Link between connecting neurons Ion channels 1 pm Channels that act as gateway causing voltage change
Sporns, O. (2007) presents in his article on brain connectivity, modeling based on structural and functional types. A network that connects at neuron and synaptic level falls into the microscale. If the neurons are grouped into population of columns and minicolumns, the level is defined as mesoscale. The macroscale representation considers the network as regions of the brain connected by inter-regional pathways.
Arbib, M. A. (2007) considers in the modular model, a hierarchical formulation of the system into modules and submodules.
Signaling modes
The neuronal signal comprises a stream of short electrical pulses of about 100 millivolt amplitude and about 1 to 2 millisecond duration (Gerstner, W., & Kistler, W. (2002) Chapter 1). The individual pulses are action potentials or spikes and the chain of pulses is called spike train. The action potential does not contain any information. A combination of the timing of the start of the spike train, the rate or frequency of the spikes, and the number and pattern of spikes in the spike train determine the coding of the information content or the signal message.
The neuron cell has three components – dendrites, soma, and axon as shown in Figure 1. Dendrites, which have the shape of a tree with branches, called arbor, receive the message from other neurons with which the neuron is connected via synapses. The action potential received by each dendrite from the synapse is called the postsynaptic potential. The cumulative sum of the postsynaptic potentials is fed to the soma. The ionic components of the fluid inside and outside maintain the cell membrane at a resting potential of about 65 millivolts. When the cumulative postsynaptic potential exceeds the resting potential, an action potential is generated by the cell body or soma and propagated along the axon. The axon may have one or more terminals and these terminals transmit neurotransmitters to the synapses with which the neuron is connected. Depending on the stimulus received by the dendrites, soma may generate one or more well-separated action potentials or spike train. If the stimulus drives the membrane to a positive potential, it is an excitatory neuron; and if it drives the resting potential further in the negative direction, it is an inhibitory neuron.
The generation of the action potential is called the "firing". The firing neuron described above is called a spiking neuron. We will model the electrical circuit of the neuron in Section 3.6. There are two types of spiking neurons. If the stimulus remains above the threshold level and the output is a spike train, it is called the Integrate-and-Fire (IF) neuron model. If output is modeled as dependent on the impulse response of the circuit, then it is called the Spike Response Model (SRM) (Gestner, W. (1995)).
The spiking neuron model assumes that frequency (inverse of the rate at which spikes are generated) of spiking train starts at 0 and increases with the stimulus current. There is another hypothetical model that formulates the firing to happen at the threshold, but there is a quantum jump in frequency in contrast to smooth rise in frequency as in the spiking neuron model. This model is called the rate model. Gerstner, W., & Kistler, W. (2002), and Sterratt, D., Graham, B., Gillies, A., & Willshaw, D. (2011) are good sources for a detailed treatment of spiking neuron models and rate neuron models.
Biological vs. artificial neural network The concept of artificial neural network (ANN) was introduced by McCulloch, W. S. & Pitts, W. (1943) for models based on behavior of biological neurons. Norbert Wiener (1961) gave this new field the popular name of cybernetics, whose principle is the interdisciplinary relationship among engineering, biology, control systems, brain functions, and computer science. With the computer science field advancing, the von Neumann-type computer was introduced early in the neuroscience study. But it was not suitable for symbolic processing, nondeterministic computations, dynamic executions, parallel distributed processing, and management of extensive knowledge bases, which are needed for biological neural network applications; and the direction of mind-like machine development changed to a learning machine. Computing technology has since advanced extensively and computational neuroscience is now able to handle mathematical models developed for biological neural network. Research and development are progressing in both artificial and biological neural networks including efforts to merge the two.
Nervous system models Evolutionary brain model