RICERCA ITALIANA     

Computational and neural mechanisms of sensorimotor learning and control

Università degli Studi di Genova

Abstract

Because the brain is fundamentally a learning system, understanding the systems architecture of how it learns motor control is a central theme in motor neurophysiology and neurorehabilitation.
A promising approach that has emerged in the last few years is based on the use of virtual environments (VE), as experimental tools for designing and monitoring the sensorimotor interactions that underlay the learning process. Two basic types of virtual environments have been defined, related to the fields of computer graphics and computer haptics. For example, by using immersive virtual reality systems it is possible to induce distortions of the visual feedback or a visuo-proprioceptive mismatch that need to be compensated by a learned internal model of the novel visuo-motor transformation. Another example is to program the mechanical impedance of a robotic haptic device in such a way to induce distortions of the movements that depend on their dynamics. Again, the ability to achieve the goal, after a suitable learning process, requires to establish an internal model of the dynamical environment to be integrated in the motor control process. However, it is a fact that the two types of virtual reality approaches (based, respectively, on Visual Virtual Environments - VVE – and Haptic Virtual Reality – VHE) have been developed to a large extent in an independent way. One reason is the complexity of the experimental setup and the other perhaps has to do with the historical/academic separation between those who study perception and those who study motor control. We shall attempt to overcome both barriers and to achieve new interdisciplinary results in which perceptual plasticity and motor plasticity are viewed as two faces of the same coin. We shall take advantage of the availability in the consortium of expertise and experimental capabilities in the two types of environments. We shall further develop them, aiming at some form of integration both at the conceptual and experimental levels. Moreover, we shall enrich the experiments with measurements that are able to evidentiate some of the neural correlates of the sensorimotor learning process, at the central and peripheral levels.
One of the major theoretical advances that was made recently was to demonstrate that adaptation is represented in a state-space framework with a generalization function. The theory links trial-by-trial variability in performance with system identification of adaptive systems. Because generalization is a fundamental property of any intelligent learning machine, the ability to measure generalization from trial-by-trial variability is an important tool to infer how the brain builds internal models of the world. However, this theory is still a work in progress and many aspects need to be clarified by suitable experimental and modelling approaches.
We are also confident that understanding the neural mechanisms of sensorimotor learning in normal subjects can be of great help in the design of new neurorehabilitation protocols that can exploit the new VE techniques in a principled way because as we make progress in discovering how a specific neurological disorder affects information processing in the sensorimotor control system, we gain insight into methods of redirecting the learning capabilities of the brain through focused rehabilitation. For this reason, in the last part of the project we intend to initiate a pilot clinical study involving a selected group of neurological patients. <<<

Principal Investigator

Pietro Giovanni MORASSO Università degli Studi di GENOVA

Research Objectives

Virtual reality systems, based on advanced techniques of computer graphics and computer haptics, are more than a decade old. Biomedical applications were attempted immediately, considering these systems as new ways to carry out well established paradigms of biofeedback. However, we think that this view of quickly pouring the new technological "wine" in an old ideological "bottle" is narrow minded and it has at least two main drawbacks: 1) it does not exploit the epistemological potential of the technology and 2) it risks being ineffective as a clinical tool if used in a raw empirical way. Our goal is to take these technologies and aim them at the general goal of the multi-disciplinary and multi-technological investigation of the fundamental mechanisms of sensorimotor learning and control. The term "sensorimotor" is used on purpose because we should like to stress that the two elements of plasticity of human behaviour, plasticity in the visuomotor transformations and plasticity in neuromotor dynamical mechanisms, are so intimately fused in our brain that it makes little sense to isolate them into two separate chapters of our neurophysiological knowledge of skilled behaviour unless this is only an intermediate step towards a more global attack on sensorimotor plasticity.

In the limited time-span of the project we shall exploit the background knowledge and experimental capabilities of the partners by rapidly converging to have two operational platforms, one mainly focused to Visual Virtual Environments and the other to Visual Haptic Environments. The two platforms will employ state of the art techniques and will be developed (in the first phase of the project) enhancing computational compatibility. This will facilitate, in the second phase of the project, at least a partial integration of the two platforms. A complete integration would go beyond the time span of the project and would require far larger resources.

The epistemological power of the experiments will be boosted by integrating, in the computational architecture, neurophysiological measurement for investigating plastic neural changes in healthy subjects during motor learning, either in global terms (assessing the sensorimotor system as a whole) or in sectorial terms (targeting the measurements to specific neural structures). The measurements will be carried out before the start of the training (baseline condition) and immediately after each training session. In particular we shall use the following techniques: 1) tonic vibration reflex (TVR), elicited in the biceps and triceps muscles; 2) the somatosensory evoked potentials (SEPs), elicited by electrical stimulation of musculocutaneous nerves, 3) the transcranial magnetic stimulation (TMS), for estimating resting and active motor thresholds and recruitment curves; 4) repetitive TMS (rTMS), for discriminating different aspects of synaptic potentiation; 5) short and long latency stretch reflexes; 6) non-linear analysis of surface EMG signals (sEMG) that takes into account their non-stationarity during voluntary movements, in order to characterise the formation of muscle activation synergies.

This analysis will also be supported by detecting and describing the effect of muscle fatigue during learning (a topic that is frequently ignored, although it might bias the analysis of learning in a significant way) and by testing if multi-cgannel, closed-loop electrical stimulation of muscles may facilitate learning, both in the VHE platform and VVE platform.

We anticipate that with this wide-ranging approach we shall advance in a significant way the current level of understanding of the dynamics of sensorimotor behavioural plasticity, to be formalized in appropriate computational models. But this is not only meant to be a pure research goal. We believe that this kind of fundamental research is the necessary basis for investigating principled approaches to the design of advanced biomedical systems for the rehabilitation of neurological patients. In fact, in the last part of the project we shall carry out a few very limited pilot studies in this direction.

The five partners of the consortium have the methodological, technological, and professional range of capabilities that are required for this multi-disciplinary and multi-sectorial project. This implies competences in 1) robotics, 2) computer graphics, 3) computer haptics, 4) biomedical instrumentation, 5) biomedical signal processing, 6) real-time control, 7) biomechanical and connectionist modelling, 8) stochastic and optimisation modelling, 9) neurophysiological measurements, 10) clinical experience in motor-impaired patients, to name just a few. Although each of the partners excels and specialises in only a few of the previous items, they all share a long lasting interest and activity in the study of sensorimotor systems. This is important because really multidisciplinary research is not just an assembly of different expertises but requires a common understanding of the object of study, i.e. a common language. <<<

Timescale

24 months

National and international background

MOTOR LEARNING AND CONTROL
The increasing knowledge about the structure and organization of the nervous system has inspired in the years new theories of movement control, intended as conceptual tools for building a coherent framework in the analysis of empirical observations. We may consider, as an example, the following list:

Reflexologist theories, based on the notion of reflex arc, a name traditionally attributed to Descartes (1662): they are linked to the hypothesis of Sherrington (1906) that reflex arcs have an integrative function.

Hierarchical theories, linked to evolutionistic concepts and based on the observation by J.H. Jackson (1873) that the nervous system appears to be hierarchically organized. In the course of evolution, the nervous system is thought to pass from a simpler state, in which lower neural centers prevail, to a more complex state, in which higher centers become predominant and thus control patterns, formerly automatic, become voluntary.

Motor progamming theories, inspired by the discovery (Wilson 1961, Grillner 1981) that cyclical movements are controlled by central pattern generators (CPG), which only weakly depend upon central control. Later, the notion of CPG has evolved into the more general concept of motor program (Keele 1968).

Cybernetic theories (Bernstein 1967), inspired by the advent of automatic control theories: they emphasize the importance of the problem of motor redundancy and define the notion of functional synergies.

Dynamical theories, derived from the concepts and terminology of non-linear dynamical systems. In this view, a fundamental property of the motor control system is self-organization and the corresponding presence of multiple attractor dynamics.

Ecological theories, inspired by the work of Gibson (1966) on affordances that greatly simplify the sensorimotor computational processes.

One way to integrate the elements of truth/plausibility of each of these theoretical approaches is to characterize in computational terms the complexity of the basic processes that underly the apparent simplicity of the human trajectory formation process (Morasso 1981). First of, all one has to recognize that the human arm has an intrinsic dynamics that dictates a complex relationship between motion of the joints and torques. This has led to the idea that contrary to earlier hypotheses (Flash 1987), passive properties of muscles are not enough to compensate for the complex physics of our limbs. Rather, the brain must predict the specific force requirements of the task in generating the motor commands that eventually reach the muscles. To study the properties of the neural system with which the brain learns to predict forces, a paradigm has been defined about a decade ago (Shadmher & Mussa Ivaldi 1994) where arm dynamics is systematically changed through forces on the hand (force field). The subject is provided with a target and asked to reach while holding the handle of a robot. When the robot's motors are off (null field condition), movements are straight as reported in early experiments on reaching. When the field is applied, movements are perturbed but with practice hand trajectories once again become smooth and nearly straight. The brain ability to modify motor commands and predict the novel forces is revealed as sudden removal of force in catch trials and in the corresponding after-effects. Improvement in performance occurs because training results in a change in the motor commands. One possibility is that movements improve because subjects co-contract antagonist muscle groups. However, in a catch trial this kind of adaptation would not produce any after-effects. An alternate hypothesis is that the composition of motor commands by the brain relies on a neural system that for any given movement direction predicts the corresponding forces. To test this idea, Conditt et al (1997) trained subjects to a reaching task and then suddenly asked them to draw a circle in the same field. They reasoned that if what was learned was like a tape recording of the forces encountered in reaching, then the neural system that had been trained to predict forces in reaching movements should be ineffective. However, they found that performance was quite good in circular movements and, remarkably, the subjects showed after-effects when the field was off. This suggested that the brain had learned to associate the sensory states of the limb to forces. These and similar experiments suggested that with practice, participants learned a sensory to motor transformation where a velocity-like input signal was transformed into a force-like output signal. This is an internal model of the force field.


VISUOMOTOR TRANFORMATIONS
Another source of complexity in motor control, which corresponds to an equally important learning process is related the visuomotor transformations that are an essential part of task planning and execution. The term sensorimotor transformation refers to the process by which sensory stimuli are converted into motor commands; they are typically formalized in terms of coordinate transformations, combining visual information with proprioceptive signals, such as the position of the eyes in the head, the position of the head with respect to the trunk, and the starting position of the arm in a common egocentric frame of reference (Laquaniti 1997, Laquaniti & Caminiti 1998).
Several cues contribute to the localization of both target and the arm in 3D space (visual intensity, accommodation, retinal topography, binocular disparity, and ocular vergence/version). Among them, binocular disparity represents a rich source of information to gain absolute and relative depth cues about the perception-action workspace. Early studies comparing motor performance when viewing objects with one or two eyes suggested that the brain relies primarily on binocular information to control goal-directed hand movements in depth (Marotta et al 1997, Servos et al 1992). A number of more recent studies, however, have shown that the brain can accurately control some aspects of hand movements when only one eye is open. In general, visuo-motor control of object placement relies much more heavily on binocular cues than does the perceptual system in tasks requiring estimates of the same surface property (Knill 2005). This provide direct evidence bearing on the question of how binocular and monocular cues contribute to motor control when they are both available.
The plasticity of the brain structures that implement visuomotor transformations is made evident by the capability of subjects to adapt to radical distortions in visual inputs. In one study, a monkey was fitted with dove prisms, which invert images along the left–right plane (Sugita 1996). For the first few weeks of continuous prism-wearing, the monkeys could barely move, but afterwards their movements improved markedly. Each day the monkeys attempted to reached to a sequence of targets. In 1–2 months, the monkeys began to reach accurately to the targets, and similar results have been obtained by Sekiyama et al (2000) in humans. After two months, the prisms were removed and, for a day or so, the monkeys again had trouble moving because they made pointing errors in the opposite direction: a phenomenon that is know as after-effect, quite similar in principle to the after effects in force field adaptation. However, unlike the many weeks necessary for adaptation, the after-effects were washed out by the third day.


VIRTUAL REALITY: COMPUTER GRAPHICS & COMPUTER HAPTICS
Visual Virtual Reality (VVR) is a very powerful tool in the analysis of sensorimotor learning because it allows the investigator to have a complete control of the spatio-temporal parameters of the visuomotor transformation. VVR-based simulators have been used as a training tool in many settings, although very few studies examined transfer of training from simulators to a real world task, particularly for manipulation tasks. In a pick-and-place task (Tracey and Lathan 2001) the relationship between motor tasks and participants' spatial abilities has been investigated. Spatial abilities were characterized using a battery of recognition and manipulation figural tests. Subjects with lower spatial abilities demonstrated significant positive transfer from a simulator based training task to a similar real world robotic operation task. Subjects with higher spatial skills did not respond as positively from training in a simulated environment. Such classification of subjects is clearly relevant for the design of effective procedures of virtual reality training systems.
Another possibility is to use VVR to simulate prisms or more complicated distortions of the visuomotor transformation. The principle of divide-and-conquer, i.e. the decomposition of a complex task into simpler subtasks each learned by a separate module, has been proposed as a computational strategy during learning visuo-motor maps (Ghahramani & Wolpert 1997). They used a virtual reality system in which subjects were exposed to opposite prism-like visuomotor remappings for movements starting from two distinct locations. Despite this conflicting pairing between visual and motor space, subjects learned the two position-dependent mappings and the generalization of this learning to intermediate starting locations demonstrated an interpolation of the two learned maps. These results provide evidence that the brain may employ a modular decomposition strategy during learning.
Haptic interfaces are devices that enable human-machine interaction via the kinesthetic and/or tactile sense (Basdogan and Srinivasan 2002). Typically, a haptic interface is designed as a manipulandum, i.e. a robotic manipulator that is equipped with an handle, which can be grasped by the user. The manipulandum may apply forces, which can be programmed to vary with hand position, speed, or acceleration. In virtual reality or telepresence systems, this allows to mimic the dynamic behaviour of virtual or remote environments (Haptic Virtual Reality: HVR). Haptic interfaces have been successfully used in various domains, including telesurgery (Rosen et al 1999), medical and surgical virtual reality systems (Kühnapfel et al 200), micro-manipulation (Grange et al 2001), and neuromotor rehabilitation. Although the different application areas share methodological and technological issues, the design constraints can be quite different. In this project we focus on the study of motor learning with an interest in neuromotor rehabilitation, i.e. on the testing/treatment of persons with motor disabilities. The best known example in this class of systems is the MIT-MANUS (Krebs et al 1998), although other systems have been developed or are under development: for instance, MIME (Lum et al 1997), KINARM (Scott 1999), PFM (Gomi & Kawato 1997), MEMOS (Micera et al 2003), and Braccio di Ferro (Casadio et al 2005).


NEURAL CORRELATES OF SENSORIMOTOR LEARNING
Neural circuits of adults retain the ability to reorganize, changing their properties in response to afferent inputs, efferent demand, environmental and behavioural influences (Traversa et 1997, Trompetto et al, 2001). These modifications, usually defined as neural plasticity, are likely to be based on various mechanisms, which are still largely unknown (Cohen et al 1998). However, it is possible to identify plausible basic strategies (Donoghue et al 1995): the functional modulation of pre-existing synapses, obtained by mechanisms similar to those described for long-term potentiation (Bliss & Lomo 1973) and long-term depression (Marr et al 1969); changes of post-synaptic excitability (Woody et al 1991) and in axonal sprouting leading to the creation of new synapses (Ramirez 2001). Studies using imaging have demonstrated skill-associated changes in neural plasticity (Pascual-Leone et al 1995). A growing body of evidence supports the notion that the overall sensorimotor system is involved in skill-associated plastic changes (Schouenborg 2004). Several tools are available to investigate the neural plasticity during sensorimotor learning. In the following we list those used in this project.
TMS
Transcranial magnetic stimulation (TMS) is very useful to investigate the motor cortico-spinal excitability (Abbruzzese and Trompetto 2002). TMS, which is based upon the principle of electromagnetic induction, uses the rapid, time-varying magnetic field provoked by a current pulse in a coil, which is unattenuated by the scalp and the skull and induces brief intracranial electric currents. These induced currents produce action potentials in excitable cells of cerebral cortex, with a preferential activation of horizontally oriented neural elements. The first generation of stimulating coils were non-focal coils, while nowadays figure-of-eight coils are available, allowing a more focal stimulation of the brain with a spatial resolution of about 0.5 cm. TMS has been recently investigated as a technique that produces virtual and reversible "lesions" on parietal and cortical level: it was shown to determine a destructuration of kinematic patterns associated to grasp (Tunik et al 2005), or reaching (Della Maggiore et al 2004). An important new development is the repetitive transcranial magnetic stimulation (rTMS), which consists of the application of trains of magnetic stimuli over the cortical motor areas. This technique has become a useful tool for modulating cortical excitability in humans (Romeo et al 2000). Moreover, it has been proved that this technique is capable of studying the excitability of glutamate-dependent excitatory motor cortex interneurones (Kobayashi et al 2003).
TVR
Another tool for the analysis of nervous plasticity is based on the fact that vibration of human muscle or its tendon causes a tonic contraction in the vibrated muscle. This tonic vibration reflex (TVR) is due to the activation of anterior horn cells by the afferent spindle discharge induced by vibration, which reaches the spinal motoneurones through both mono and polysynaptic pathways, travelling mainly across the brain stem (Eklund and Hagbarth 1966, Marsden et al 1969). Therefore, the TVR seems to be an excellent tool for the investigation of the overall excitability of the sensory-motor system.
EMG
As a "final common pathway" of the motor control process the electrical activity of muscles (EMG) is a signal worth monitoring in order to improve the understanding of motor learning. The acquisition of motor skills is connected to the appearance of "muscle synergies", i.e. strategies adopted by the central nervous system to coordinate either motor unit groups or muscular complexes. The study of muscular synergies construction can thus provide insights on motor learning mechanisms. To this end, the literature shows several algorithms that can overcome the degree of arbitrarity inherently present in the categorization of EMG signals (Kleissen 1990), when extracting amplitude and timings parameters with high robustness and reliability, also in relation with muscle fatigue (Bonato et al 1999, D'Alessio & Conforto 2001).
FET
Although functional electrical stimulation (FES) has been widely used for more than 40 years with mixed success, there has been recently an increased interest in a new development known as Functional Electrical Therapy (FET), which is a close-loop technique (Popovic et al 2002). Several studies showed that early motor rehabilitation, applied in combination with functional stimulation, is effective in determining a more rapid recovery in subjects who underwent CNS lesions. In the context of the present project we are not specifically interested in the neurorehabilitation value of the technique but we think that a better understanding of the way in which this kind of tool can modulate motor learning in normal can be extremely valuable for designing principled approaches to the rehabilitation of patients. <<<