To understand underlying neural mechanisms of learning and memory, we take a combined approach of computational modeling, behavioral experiments, neuroimaging, and neuromodulation. We are interested in two distinct memory systems, which are procedural and declarative memory, and how these two systems interact. Building on these scientific findings, we investigate the development of efficient learning protocols applicable to the rehabilitation of patients with stroke and Parkinsonism.

Ongoing Projects

• DeepDraw Project

  The DeepDraw project investigates the Humans sensory-motor control system by combining the explainability of MRI neuroimaging and the computational power of task driven deep neural networks. The overall goal of this project is to compare brain activity and neural network model activity in a sequential single letter writing task to explore the eventual hierarchical composition of proprioception and motor control in humans.

• fMRI study using task-driven deep reinforcement learning models of human motor control

  The current study aims to apply AI to understand the neural basis of human motor control. For this, we first create an fMRI experiment in which human participants track a moving target using a joystick in the scanner. Next, we developed a deep reinforcement learning (DRL) model simulating the same motor task performed by the human participants. The model takes raw pixel inputs from images and generates continuous motor action to account for the entire cognitive process of humans, from perception to action. Specifically, it combines a convolutional neural network (CNN) and a twin-delayed deep deterministic policy gradient algorithm (TD3). We successfully trained the model simulating the human motor control and then tested whether the trained model predicts the neural responses. We used the Voxel-wise Encoding Model (VM) and demonstrated that the internal representations of the trained model better predicted the fMRI data than the untrained model. Furthermore, representational similarity analysis (RSA) revealed that the trained model showed higher similarity to the spatial pattern of fMRI data than the untrained model. In sum, we showed that the deep reinforcement learning model would be an excellent candidate for understanding the neural basis of human motor control.

• The relationship between motor learning and hippocampus

  Hippocampus is known to contribute learning, but the traditional view of motor learning area says that hippocampus is not involved in motor learning. Against the traditional view of motor learning, we are studying the relation between interleaved-state brain activity in hippocampus and motor learning.

• The effects of TMS on DLPFC and M1 for motor learning

  Neuroimaging studies have shown that the dorsolateral prefrontal cortex (DLPFC) is recruited during motor skill learning. Nothing to mention, the primary motor cortex (M1) is also important. Then what happens when we apply TMS to DLPFC and M1? Would it be able to positively affect human motor learning? Is frequency also related to the effect? To investigate that, we are doing a TMS experiment.

• Functional delineation of the human striatum related to reward processing in motor skill learning

  The striatum is a major part of the basal ganglia to which the dopaminergic neurons are projected most, and it is important for motor control and reward processing. Thus, it is a locus of learning new motor skills based on reward feedback which reflects dopamine release. To disentangle the roles of the striatum, we present an fmRI experiment of motor learning guided by continuous visual feedback of performance which is explicitly provided through learning. We were able to delineate the anatomically defined striatum with high sensitivity and specificity using a parametric regressor encoding the learning performance. Our findings support the hypothesis that striata BOLD responses represent a dynamic change in dopamine release that is intimately linked to motor learning performance.

• Double dissociations of the effects of visual feedback on motor and somatosensory cortices during visuomotor learning

  The functional dissociation of M1 and S1 during human visuomotor learning, on the other hand, is poorly understood. To investigate the neural basis of the dissociation, we designed an fMRI study in which individuals learned a new motor skill. We manipulated the visual feedback of a moving effector (i.e., a cursor) so that participants learned to reach a target under two alternating conditions: online cursor feedback is available or unavailable except when a target is reached. As a result, the higher response was recorded in M1 when the online feedback was available but lower when it was not, and vice versa in S1.

General Interests

• 

  Neural substrates of de novo motor skill learning

  Human motor skill learning is a complicated process of generating a novel movement pattern to achieve a task goal guided by evaluative feedback such as rewards. The Basal Ganglia (BG) plays a …

  • Neural substrates of de novo motor skill learning

  • Human motor skill learning is a complicated process of generating a novel movement pattern to achieve a task goal guided by evaluative feedback such as rewards. The Basal Ganglia (BG) plays a central role in reward-based motor skill learning. Recent primate studies suggest rostrocaudally separated circuits in the BG for voluntary (early) versus automatic (late) behavior. However, little has been known about the separate circuits of the BG in reward-based human motor skill learning.
    
    To date, most neuroimaging studies investigating neural mechanism of motor skill learning have employed target-reaching, sequential force control, or sequence learning tasks. Here, we designed a novel fMRI experiment in which subjects learn a novel motor skill from scratch. Subjects wear a MR-compatible data-glove and learn to control a computer cursor over a 5-by-5 grid by manipulating fingers ( see figure).

    
    

    
    
    

    
    

    
    
    We collected behavioral and fMRI data from more than 25 subjects. All individual learned to control a cursor to reach targets after extensive training (see figure below).

    
    

    
    
    

    
    

    
    
    To investigate how extensive training changes neural representation of mapping between high-dimensional motor space and low-dimensional task space, subjects participated in two fMRI sessions separated by five training sessions. The extensive training decreased interaction between the motor and visual modules but increased interaction between the motor and reward modules. We also found the central executive, salience, and dorsal/ventral attention networks were strongly modulated by trial-by-trial reward in the early learning phase, but the extensive training reduced the sensitivity of these networks to the rewards.

    
    

    
    
    

    
    

    
    
    Most interestingly, we found fMRI evidences supporting the separate circuits in the caudate nucleus for early versus late stage of motor skill learning. As a result of the extensive training, the reward-modulated region shifted from the rostral to the caudal part of the caudate nucleus. However, there was no such a change in cortical area, vmPFC.

    
    

    
    
    

    
    

    
    
    To our best knowledge, for the first time, we report motor learning-induced transition of reward modulating regions of the human brain. In the future, we will use 7 T fMRI scanner to investigate the tail region of the caudate nucleus with higher spatial resolution. Additionally, we will use TMS targeting the primary motor cortex to understand how consolidation process would be modulated by the tail region of the caudate nucleus. (For downloading a poster presented in the annual meeting of Cognitive Neuroscience Society 2019, related this work, please click here)
• 

  Neuromodulation of learning & memory using non-invasive stimulation

  Stimulation of brain has a long history dating back to the 18th century, using transcranial electrical stimulation. In the era, people did not have enough knowledge about neurophysiology such …

  • Neuromodulation of learning & memory using non-invasive stimulation

  • Stimulation of brain has a long history dating back to the 18th century, using transcranial electrical stimulation. In the era, people did not have enough knowledge about neurophysiology such that they shocked their own brain regardless of irreversible permanent brain damage.
    
    Thanks to advances of neuroscience, we can see through brain and its activity with MRI and now seek to modulate brain functions.

    
    

    
     

    
    

    
    
    Transcranial magnetic stimulation (TMS) is considered to be a promising tool to induce reliable functional changes of human brains (see figure above). The principle of TMS is based on electromanetic theory of inducing electrical current as magnetic field changing in time. With current TMS technology, only cortical brain regions near the skull could be stimulated (but see dTMS). A research group in Northwestern University led by Prof. Joel Voss demonstrated that repetitive high frequency TMS (rTMS) could enhance associative memory function by targeted stimulation of hippocampus. They showed stimulating left parietal region showing the maximum correlation with hippocampus target in resting-state could significantly enhance hippocampal-cortical connectivity.
    
    I recently found targeted rTMS to the hippocampus also could enhance evoked activity of hippocampal-cortical network, especially in posterior-medial network involved in processing of item-context memory and associative memory recognition performance (see figures below, Kim et al., Science Advances, 2018).

    
    

    

    
    

    

    
    

    
    
     

    
    
    In the future, I continue this line of research to further understand distinct neural mechanism for various types of context-dependent declarative memory. Additionally, I will investigate on how TMS can modulate interaction between different memory systems, such as procedural memory (e.g., motor memory) and declarative memory.
    

    
    

    
    

    
    
    #TMS #fMRI #hippocampus #neuromodulation

• 

  Computational approaches to fMRI analysis

  Thanks to fast-increasing computing power, it is becoming more available to perform computationally intensive analysis on high-dimensional fMRI data. Also, during last two decades, traditional…

  • Computational approaches to fMRI analysis

  • Thanks to fast-increasing computing power, it is becoming more available to perform computationally intensive analysis on high-dimensional fMRI data. Also, during last two decades, traditional analysis of localizing brain regions associated cognitive functions has been challenged by connectivity analysis among multiple brain regions and multi-voxel pattern analysis considering spatial patterns of activity over ensemble of voxels. Currently, numerous algorithms and analysis tools have been adapted from fields of network theory and machine learning and available for cognitive neuroscientists analyzing fMRI data.
    
    Our lab has experiences in both of connectivity analysis and multi-voxel pattern classification (MVPA) for fMRI data. We have investigated how large-scale network interaction could support item-context memory formation (submitted). We used a relatively novel application of probabilisitic connectivity that considers inter-subject variability in functional networks. This method was superior to a conventional method averaging connectivity across subjects for group analysis in finding underlying modular structures of the network (see figure below). Additionally, we developed a novel algorithm comparing modular structures between two networks in a statistically rigorous way using normalized mutual information (NMI). We are currently investigating how motor skill learning can be represented as reconfiguration of network structures in a more efficient way using dynamic network analysis (see Basset et al.).
    

     
    

    
     

    
    For the technique of multi-voxel pattern classification, we have shown that spatial activity patterns in the cerebellum could gradually discriminate two motor tasks (see figure below and PLoS Biology, 2015). 

    
    

    
     

    This technique could be applied to discriminate activity patterns encoding different contexts in associative memory task and context-dependent learning task. More interestingly, both of connectivity analysis and MVPA could be applied to reveal how non-invasive stimulation such as transcranial magnetic stimulaton (TMS) changes large-scale brain network and spatial activity pattern in a way of enhancing cognitive functions.
    

• 

  Computational modeling of motor learning & memory

  Motor learning tasks are often categorized into motor adaptation, which learns recalibration of existing motor policies to perturbed environment, and motor skill learning, which learns novel …

  • Computational modeling of motor learning & memory

  • Motor learning tasks are often categorized into motor adaptation, which learns recalibration of existing motor policies to perturbed environment, and motor skill learning, which learns novel motor policies to achieve task goals. Learning to walk on the moon would be a good example of motor adaptation. On the moon, where gravity reduces to sixteen percent on the earth, you need to recalibrate your existing working patterns. For the motor skill learning, you can think of playing the piano. You need to learn new patterns of finger movements and how to distribute force on your fingers to play musical notes.

    
    

    
     

    
    
    Motor learning can be quantified easier than other types of learning, such as word learning and abstract-rule learning. Changes of reaction time, movement speed and direction, force could be measured as indicators of learning. Thus, motor learning is very attractive in the perspective of theoretical computation modeling. Our lab pursues to use computational models as tools to understand human behaviors and underlying neural mechanisms in motor adaptation and motor skill learning. In this blog, I will focus on motor adaptation, please see the other blog linked to brain-machine interface for motor skill learning.
    
    For motor adaptation models, we employed well-known multi-rate memory models as depicted in the figure below.
    
    According to this model, motor output (y) is represented as the sum of two memory states with different time scales, fast memory and slow memory.
    Each memory updates its state proportional to motor errors (e), which is the difference between the desired motor output (d) and the actual motor output (y). Fast memory updates its state from the current trial to the next trial faster and decays faster than slow memory. Additionally, one-fast and multiple slow memory could account for learning multiple tasks; initial interference in fast memory with gradual learning of separate representation of multiple tasks in distinct slow memory. A switching cue (c) plays a role in addressing corresponding slow memory depending on a current task.
    
    In recently published papers in PLoS One (2015), we showed how this simple computational model could account for contextual interference effects and spacing effects. Based on this model, we also searched neural substrates of motor memory with multiple time scales spatially distributed in the brain (PLoS Biology, 2015) as shown in the following figure. (https://www.youtube.com/watch?v=1tc98aBU67w) 

    
    

    
     

    
    Currently, we investigate on interaction between motor memory and episodic memory, which are two different form of memory, when two relevant tasks were sequentially presented. We found preliminary results showing bidirectional interference between the two memories not only in behaviors but also in neural activities using fMRI.
    
    In the future, we will further investigate causal link of this interference using non-invasive brain stimulation using transcranial magnetic stimulation (TMS).
    
    
    #motorlearning #computationalmodel #motoradaptation #fmri