Biologically plausible credit assignment
The credit assignment problem is likely one of the biggest open questions, both in neuroscience and AI. Stated dramatically, suppose you are playing tennis and you saw you hit the ball incorrectly. Which one of your 100 trillion synapses are to blame? And how does the brain specifically find and correct the right set of synapses in your motor system, especially when the error is delivered through the visual system hundreds of milliseconds after the error occurred? In AI, this credit assignment problem is solved in many cases through backpropagation of error through multiple layers of computation. However, it is unclear how the brain solves this problem. What is true is that the brain solves it using a local learning rule: that is every synapse adjusts its strength using only information that is physically available to it, for example the electrical activity of the two neurons connected by the synapses, the strength of other synapses nearby, and any neuromodulatory inputs reflecting rewards and errors. Elucidating what such local synaptic rules are and how they work could have a dramatic impact in AI, leading to embarrassingly parallel implementations of learning on neuromorphic chips that avoid the communication overheads of backpropagation. But more generally, the very identification of a common unsolved problem plaguing both neuroscience and AI should motivate progress by bringing together synaptic physiologists, computational neuroscientists and AI practitioners to collectively crack the problem of biologically plausible credit assignment. Such a combination of experimental knowledge, theory, and engineering know-how is likely needed to successfully address this grand challenge.
Incorporating synaptic complexity
A major divergence between biological and artificial neural models lies in the very way we model synapses connecting neurons. In artificial networks, synapses are modelled by a single scalar value reflecting a multiplicative gain factor transforming how the presynaptic neuron’s input affects the postsynaptic neuron’s output. In contrast, every biological synapse has hiding within it immensely complicated molecular signaling pathways [1]. For example hippocampal synapses underlying our memory of recent events each contain a chemical reaction network of hundreds of different types of molecules capable of implementing an entire dynamical system with sophisticated temporal processing capabilities [2].
Upon seeing such complexity, a theorist or engineer may be tempted to simply ignore it as biological messiness arising as an accident of evolution. However, theoretical studies have shown that such synaptic complexity may indeed be essential to learning and memory [3]. In fact network models of memory in which synapses have finite dynamic range, require such synapses be dynamical systems in their own right with complex temporal filtering properties to achieve reasonable network memory capacities [4]. Moreover, more intelligent synapses have recently been explored [5] in AI as a way to solve the catastrophic forgetting problem, in which a network trained to learn two tasks in sequence can only learn the second task, because learning the second task changes synaptic weights in such a way as to erase knowledge gained from learning the first task.
|
|
|
More generally, it is likely that our current AI systems are leaving major performance gains on the table by ignoring the dynamical complexity of biological synapses. Just as we have added spatial depth to our networks to achieve complex hierarchical representations, we may also need to add dynamical depth to our synapses to achieve complex temporal learning capabilities.
Complex molecular states within single synapses can aid learning and memory. (see e.g. A memory frontier for complex synapses.)
Taking cues from systems-level modular brain architecture
Often, current commercial AI systems involve training networks with relatively homogenous layered or recurrent architectures starting from a tabula rasa of random weights. However, this may be too hard of a problem to solve for more complex tasks. Indeed biological evolution has taken a very different path. The last common ancestor of all vertebrates lived 500 million years ago. Its rudimentary brain has been evolving ever since, leading to the mammalian brain about 100 million years ago, and the human brain a few million years ago. This unbroken chain of evolution has lead to an intricate brain structure with highly conserved computational elements, and immense system level modularity. In fact we currently lack any engineering design principles that can explain how a complex sensing, communication, control and memory network like the brain can continuously scale in size and complexity over 500 million years while never losing the ability to adaptively function in dynamic environments. Thus it may be very interesting for AI to take cues from the systems-level structure of the brain.
|
|
|
One key systems property is modularity both at a functional and anatomical level. The brain is not homogenous like our current AI architectures, but has different modules, like the hippocampus (subserving episodic memory and navigation), the basal ganglia (underlying reinforcement learning and action selection), and the cerebellum (thought to automatize skilled motor control and higher level cognition through supervised learning). Moreover, memory systems (habitual memories, motor skills, short term memory, long-term memory, episodic memory, semantic memory) in the human brain are also functionally modular; different patients can have deficits in one type of memory without deficits in the others. Also, in the motor system nested feedback loop architectures predominate [6,7], with simple fast loops implementing automatic motor corrections in 20 ms through the spinal cord, slightly slower smarter loops implementing more sophisticated motor corrections over 50 ms through the motor cortex, and finally visual feedback flowing through the entire brain implementing conscious corrections of motor errors. Finally, a major feature of all mammalian brains is a neocortex consisting of a large number of relatively similar 6-layered cortical columns, all thought to implement variations on a single canonical computational module [8].
Overall, the remarkable modularity of the modern mammalian brain, conserved across species separated by 100 million years of independent evolution, suggests that this systems-level modularity might be beneficial to implement in AI systems (in functional principle, if not in biological detail) and that the current approach of training neural networks from a tabula rasa is likely an infeasible path towards more general human-like intelligence. Indeed, as an example, a combination of systems-level modularity (both anatomical and functional), nested loops which segregate different types of error correction, and more dynamically sophisticated synapses may all be critical ingredients in solving the grand challenge of biologically plausible credit assignment raised above.
Дата добавления: 2019-02-26; просмотров: 169; Мы поможем в написании вашей работы! |
Мы поможем в написании ваших работ!