What are the functional components of the brain responsible for the creation of intelligent motion?
The Puppet Master
The brain of a gymnast faces a seemingly impossible computational problem of control. First, the gymnast must learn a routine from a demonstrator either from scratch, as with a complete novice, or by bootstrapping off of previous experience as with an amateur or professional. The brain must then coordinate the approximately 200 joints and over 600 muscles to create whole body, goal directed movement and needs to achieve reasonable performance within a relatively few number of trails. As the body tires, the gymnast must adapt to changing dynamics along with external changes in equipment and setting. Additionally, a good forward model is needed in order to predict the outcome of current and future movements and adapt accordingly. Finally, in order to get home, the gymnast may need to get into a vehicle and operate a system with wildly different kinematics with just as much proficiency.
This kind of intelligent motion control is the holy grail of robotics.
In the previous post, Basis of Biological Motion pt. 1, we examined the most elementary mechanisms by which biology achieves motion. Motor neurons and CPGs, while robust against local perturbations, lack the computational complexity to achieve the kind of intelligent motion control present in the previous example of the gymnast. In this second half, we will look at the functional components located in the brain that play key roles in vertebrate movement.
This review followed closely to the structure of (Schmuelof et al., 2011) which is an excellent place to look for more information.
As early as 1870, neuroscientists had identified a map of the body's motor system that appeared to exist on the surface of the cortex (Fritsch and Hitzig, 1870). Researchers charted this map by applying electrical stimulation to the surface of the brain and observing the corresponding bodily movement. For example, current applied to the region of the map related to the thumb would cause corresponding twitches in the muscles of the thumb. However, the map did not contain clear distinctions between motor areas but instead presented with overlapping (read "blurry") regions where stimulation resulted in muscle movement in a number of different areas. Over time the model of a simple map became more complex, splitting into separate premotor and primary motor regions based on lesion studies which found the regions had similar but separate functions and operated in parallel (Fulton, 1935). The premotor cortex appeared to control gross, wide sweeping movements while the primary motor cortex handled fine motor control.
Regions of the motor cortex became increasing subdivided as individual functional components were identified with more sophisticated techniques (Graziano et al., 2007). A high resolution fMRI scan of human patients performing various movements involving hands, arms, eyes, and lips demonstrated that the cortex map is activated during voluntary movement and has the same anatomical segmentation found by electrical stimulation (Meier et al., 2008). A study involving neonatal kittens discovered that the cortex motor map originally begins with a separated representation of joints and only develops into an overlapping map depending on learning (Martin et al., 2005). Over a long period of time, the learning dependent shifts in the motor map disappeared, suggesting the map is being continually rewritten to optimize for behaviors that have a high likelihood of occurring and will overwrite outdated behaviors.
Most stimulation studies of the motor cortex in the 1900's had used short burst (50 ms) and low current stimulation which evoked only brief muscle twitches. Twitches; however, reveal little insight into the computational mechanism of control. In 2002, Michael Graziano at Princeton University applied a much longer stimulation (500 ms) onto the motor cortex of a monkey and found something remarkable. When stimulation was held on complex, behavioral movements such as hand-to-mouth movements, defensive movements, and reach and grasp movements were evoked in the primate (Graziano et al., 2002). At the same time, an organized spatial map of hand positions was discovered. For example, stimulation applied to the part of the map encoding for the lower right location would cause smooth movement of the hand to that position regardless of its initial condition. While stimulated movement had much of the same kinematic properties of voluntary movement, if an obstacle was placed between the hand and a final destination the hand would run into and press against the obstacle without avoiding it (Graziano et al., 2002).
Stimulation of the premotor cortex and the SMA evoked even more complex, whole body movements such as climbing and leaping movements often from a single stimulation site (Graziano, 2010). In order to explain the diversity of behaviors embedded in the motor map, Graziano postulated that the map serves as a dimensionality reduction technique that stores high dimensional actions on a two dimensional surface using a Kohonen self-organizing map (Graziano et al. 2007). This model helps to explain some of the overlap between regions observed and suggests how the brain might learn over a large action space with few trials by updating nearby regions with a proximally depreciated signal.
The primary motor cortex seems to play an important part in the storage and retrieval of motor stereotype behaviors that are experience dependent and cannot be effectively coded for at the spinal level because of the required use of large numbers of joints / muscle groups. This theory is further supported by the use of transcranial direct current stimulation (tDCS) applied to the motor cortex in humans which found increased retention in learned motor skills (Galea et al., 2010). Likewise, transcranial magnetic stimulation (TMS), which prevents neuronal activity in the applied region, produced an inability to retain learned skills. Yet the motor cortex should not be thought of as the single action module in the brain, indeed, the motor cortex only accounts for about 40% of the connections in the brainstem (Scott, 2012). Thus, the motor cortex must operate in parallel with other modules to share control of the body.
The primary motor cortex region, M1, is one of the most promising locations for brain computer interface devices. In particular, work out of the Motorlab at the University of Pittsburgh with a prosthetic arm and two micro-electrode arrays implanted into the M1 region has been able to give a quadriplegic patient a surprising amount of skilled movement including simple self feeding (Collinger et al., 2012). Videos can be found here.
The cerebellum is an ancient sub-cortical structure found in every vertebrate from humans to birds to sharks with a highly preserved organization across species. The cerebellum has a number of anatomical curiosities such as an inner cortex of dense gray matter with about 3.6 times as many neurons as the neo-cortex. This ratio of 3.6 is preserved roughly across all species (Herculano-Houzel, 2010). Thus the cerebellum has more neurons than the rest of the brain combined and yet accounts for only around 10% of the volume. Trained pianists were shown to have a larger amount of gray matter in their cerebellum suggesting the possibility of neurogenesis and increased levels of gray matter were also shown to improve learning speeds (Steele et al., 2012). It has direct connections with the brainstem and despite its clear anatomical separation from the cerebral cortex, and it has many connections to the premotor, parietal, and frontal cortex especially in humans (Penhune et al., 2011).
The main function of the cerebellum is still in dispute but the literature converges on a few theories involving internal state estimation, feed forward models, skill learning, or some combination thereof. Of course, considering the neuronal density of the cerebellum, the answer could be all of these and many more. The state estimation theory is supported by many studies looking at patients and animals with damage or decreased activity in their cerebellum and observing changes in motor tasks. For example, a study by Miall et al. showed transcranial magnetic stimulation applied to the cerebellum resulted in decreased ability to adapt to visual changes in reaching tasks (Miall et al., 2007). Likewise, tDCS in the same area improved visuomotor adaptations (Galea et al., 2010). Arguments for the cerebellum serving as a forward model integrated in sensory feedback can be found in Tanaka et al., 2009, based on the existence of targeted connections from the cerebellum to the parietal cortex that appear to compute a prediction error.
The cerebellum, like the motor cortex, is a highly plastic structure that changes with adaptation (Penhune et al., 2011). Interestingly, kittens who have had their cerebral cortex removed (decorticated) after birth could survive for a more than a year and still exhibited many complex behaviors such as sleeping, eating, and goal directed walking (Bjursten et al., 1976). Thus the cerebellum may be capable of providing some direct control signals for very simple, primitive behaviors associated with the bare essential needs of survival whilst relying on the primary motor cortex to provide optimized, more diverse behavioral control (this is speculation).
The basal ganglia (BG) is a group term for the sub-cortical structures near the center of the brain that share a high degree of inter-connectivity and appear to be functionally related. The BG sources input connections from many areas of the brain but has particularly high numbers of output connections with the motor cortex. Of the neural components reviewed so far, the BG is least understood from a functional perspective both because its difficult to reach location makes traditional experimental methods challenging and because the high degree of connectivity makes causation difficult to separate from association. As with the cerebellum, the basal ganglia has a highly preserved structure across vertebrates. The complexity of the BG region correlates with the complexity of movement as amphibians have stereotyped movements and a BG with fewer connections compared to reptiles and mammals with complex movements and highly connected BG (Shmuelof et al., 2011).
One function of the BG with strong experimental evidence appears to be that of supplying variability to motor actions. Inhibiting the LMAN region in juvenile songbirds, a similar region to the BG in mammals, showed dramatic reduction in the variability of the song and revealed a stereotyped pattern (Ölveczky et al., 2005). Producing variability is a critical component of reinforcement learning which requires pairing motor changes with associated feedback in order to converge to a desired solution. In this case, variability in the juvenile songbird allows the child to learn the song of a parent through exploration.
Other experiments have shown that the BG is an important component in learning a sequence of movements. BG impairment in monkeys has shown an inability to learn sequences of button presses and in rodents an inability to learn sequences of nose pokes (Shmuelof et al., 2011). The BG also appears to play a role in the cognitive control of eye movements (Hikosaka et al., 2000). However, lesions to of the BG produce a wide range of movement disorders including an inability to suppress involuntary movements. This corresponds to other work pointing to the BG as having a role not only in varying actions but in deciding which action to take (Balleine et al., 2007).
What can be learned from the brain about motion from a robotics perspective?
Complexity isn't the problem, complexity is the answer
In physics community, there is a widely held belief that the fundamental laws of nature should take simple forms. While the underlying rules may be simple, incredible complexity arises from the interaction between them. And although we have made progress in understanding the fundamental rules behind in intelligence, such as RL, Bayesian inference, and dimensionality reduction, most work studies these components in isolation with no real consideration for how a particular algorithm will interact with other components. A robot can not create a new value function for every state space encountered nor more than it can maintain a prior for every object in the universe. Additionally, most designs follow a pipeline approach where information is processed in a serial chain before a final high level representation is distributed with no feedback information and no parallel processing connections.
Designing robots that can handle the variability in the natural world requires methods that are both scalably complex and dynamically reconfigurable which is simply not achievable with finite state machines and graph based search. The complexity of the brain is not the result of a terrible design choice by evolution but is rather telling us something important about the answer to problem of intelligent motion. I do not believe the biggest problems in robotics come from a lack of understanding surrounding the fundamental rules. The fundamental rules are just the tip of the iceberg.
The cerebellum question
The fact that the cerebellum has more neurons than the rest of the brain combined is a very troubling issue. Roughly speaking, it means that whatever problem(s) the cerebellum is designed to solve it requires more computational power than every other problem solved by the cerebral cortex. It is more difficult than language. It is more difficult than vision. It is more difficult than giving you a personality and storing all your memories. Although it undoubtedly does more than feed-forward modeling and state estimation alone, the fact that these appear to require so much computational resources is troubling. However, one optimistic explanation might be that efficient feed-forward modeling is simply difficult to compute on a neuronal architecture and might have a simpler representation that can be solved more easily by a traditional silicon processor.
Balleine, B. W., Delgado, M. R., & Hikosaka, O. (2007). The role of the dorsal striatum in reward and decision-making. The Journal of Neuroscience, 27(31), 8161-8165.
Beevor, C., and Horsley, V. (1890). An experimental investigation into the arrangement of excitable fibers of the internal capsule of the bonnet monkey (Macacus sinicus). Philos. Trans. R. Soc. Lond. B Biol. Sci. 181, 49–88.
Bjursten L-M, Norrsell K, Norrsell U (1976) Behavioral repertory of cats without cerebral cortex from infancy. Exp Brain Res 25:115-130
Collinger JL, Wodlinger B, Downey JE, Wang W, Tyler-Kabara EC, Weber DJ, McMorland AJC, Velliste M, Boninger ML, Schwartz AB High-performance neuroprosthetic control by an individual with tetraplegia. Lancet 6736:61816-61819 (2012)
Fritsch, G., and Hitzig, E. (1870). Ueber die elektrishe Erregbarkeit des Grosshirns. Translated by G. von Bonin, In The Cerebral Cortex, W.W. Nowinski, ed. (Springfield, IL: Thomas, 1960), p. 73-96
Fulton, J. (1935). A note on the definition of the "motor" and "premotor" areas. Brain 58, 311-316
Galea JM, Vazquez A, Pasricha N, Orban de Xivry JJ, Celnik P. (2010) Dissociating the roles of the cerebellum and motor cortex during adaptive learning: the motor cortex retains what the cerebellum learns. Cereb Cortex.
Graziano MSA, Taylor CSR, Moore T (2002) Complex movements evoked by microstimulation of precentral cortex. Neuron 34, 841-851.
Graziano MS, Aflalo TN. (2007). Mapping behavioral repertoire onto the cortex. Neuron 56(2):239–51
Graziano MSA (2010) Ethologically relevant movements mapped onto the motor cortex. In: Primate Neuroethology, Ghazanfar and Platt (Eds.), Oxford University Press, pp. 454-470.
Herculano-Houzel S (2010). "Coordinated scaling of cortical and cerebellar numbers of neurons". Front. Neuroanat. 4:12. doi:10.3389/fnana.2010.00012.
Hikosaka, O., Takikawa, Y., & Kawagoe, R. (2000). Role of the basal ganglia in the control of purposive saccadic eye movements. Physiological reviews, 80(3), 953-978.
Martin, J.H., Engber, D., and Meng, Z. (2005). Effect of forelimb use on postnatal development of the forelimb motor representation in primary motor cortex of the cat. J. Neurophysiol. 93, 2822–2831.
Meier J., Aflalo T., Kastner S., Graziano MS (2008). Complex Organization of Human Primary Motor Cortex: A High-Resolution fMRI Study Journal of NeurophysiologyPublished 1 October 2008Vol. 100no. 1800-1812DOI: 10.1152/jn.90531.2008
Miall RC, Christensen L, Cain O, Stanley J. (2007) Disruption ofstate estimation in the human lateral cerebellum. PLoS Biol; 5:e316
Ölveczky BP, Andalman AS, Fee MS (2005) Vocal Experimentation in the Juvenile Songbird Requires a Basal Ganglia Circuit. PLoS Biol 3(5): e153. doi: 10.1371/journal.pbio.0030153
Penfield, W., and Welch, K. (1951). The supplementary motor area of the cerebral cortex: A clinical and experimental study. Am. Med. Assn. Arch. Neurol. Psychiat. 66, 289–317.
Penhune, V. B. and C. J. Steele (2012). "Parallel contributions of cerebellar, striatal and M1 mechanisms to motor sequence learning." Behav Brain Res 226(2): 579-591.
Scott SH (2012) The computational and neural basis of voluntary motor control and planning. Trends in cognitive sciences 16: 541–9. doi: 10.1016/j.tics.2012.09.008
Shadmehr R., Brandt J., Corkin S. (1998) Time-dependent motor memory processes in amnesic subjects. J Neurophysiol;80:1590
Shmeulof, L. and Krakauer, J.W. (2011) Are we ready for a natural history of motor learning? Neuron 72, 469–476
Steele, C, Scholz J, Douaud G, Johansen-Berg H, and Penhune V. (2012)Structural correlates of skilled performance on a motor sequence learning task. Front Hum Neurosci.; 6:289