Neural Dynamics Model of Visual Decision-Making: Learning from Human Experts
Jie Su Fang Cai Shu-Kuo Zhao Xin-Yi Wang Tian-Yi Qian Da-Hui Wang Bo Hong







1]Jie SU
These authors contributed equally to this work.
1]Fang CAI
These authors contributed equally to this work.

2]Shu-Kuo ZHAO
1]Xin-Yi WANG

[1]Tian-Yi QIANqiantianyi@qiyuanlab.com
[2]Da-Hui WANGwangdh@bnu.edu.cn
[1]Bo HONGhongbo@qiyuanlab.com

*[1]
    Qiyuan Laboratory,
    Beijing, China
    
[2]
    School of Systems Science,
    Beijing Normal University,
    
        No. 19, Xinjiekouwai St.,
        Beijing, 100875, China
    






    Uncovering the fundamental neural correlates
    of biological intelligence,
    developing mathematical models,
    and conducting computational simulations
    are critical for advancing new paradigms
    in artificial intelligence (AI).
    
    In this study, we implemented
    a comprehensive visual decision-making model
    that spans from visual input to behavioral output,
    using a neural dynamics modeling approach.
    
    Drawing inspiration from the key components
    of the dorsal visual pathway in primates,
    our model not only aligns closely with human behavior
    but also reflects neural activities in primates,
    and achieving accuracy comparable to
    convolutional neural networks (CNNs).
    
    Moreover, magnetic resonance imaging (MRI)
    identified key neuroimaging features
    such as structural connections and functional connectivity
    that are associated with performance in perceptual decision-making tasks.
    
    A neuroimaging-informed fine-tuning approach
    was introduced and applied to the model,
    leading to performance improvements that
    paralleled the behavioral variations observed among subjects.
    
    Compared to classical deep learning models,
    our model more accurately replicates
    the behavioral performance of biological intelligence,
    relying on the structural characteristics
    of biological neural networks rather than extensive training data,
    and demonstrating enhanced resilience to perturbation.







Neural Dynamics Model of Visual Decision-Making: Learning from Human Experts
    *
    September 9, 2024
============================================================================








§ INTRODUCTION



Abstracting the structures of biological neural systems
into mathematical models
and constructing artificial neural networks
based on these abstractions to address real-world problems,
represents a pivotal approach to innovation in artificial intelligence.

Over the past decades,
this approach has achieved significant success in various fields
such as pattern recognition and computer vision
<cit.>.

Visual perception, a fundamental process through which
humans and animals interpret and interact with the environment,
is a central topic in both neuroscience an artificial intelligence (AI).

Understanding the neural mechanisms underlying perceptual decision-making
not only provides insights into biological systems
but also has the potential to drive advancements in AI technologies.

Convolutional neural network (CNN) models,
inspired by the receptive fields and
parallel distributed processing in animal vision systems,
have achieved notable success in tasks such as
object detection, facial recognition
<cit.>,
and action recognition
<cit.>,
often surpassing human performance.

However, these models face significant limitations,
including the need for vast amounts of labeled data,
lack of biological interpretability,
and susceptibility to adversarial attacks.

These limitations highlight the gap between
current AI systems and biological neural networks
regarding robustness, efficiency, and flexibility.

Addressing these challenges requires a paradigm shift
towards models that more closely mimic the
underlying principles of biological neural computation.



Motion perception is critical for animals
to detect potential conspecifics, preys or predators,
and is vital for their survival.

The motion perception system starts with retinal input,
travels through the lateral geniculate nucleus (LGN)
to the primary visual cortex (V1),
and then projects to parietal areas along the dorsal visual pathway,
supporting spatial attention and eye movements
<cit.>.

Direction-selective (DS) neurons
are widely present in brain areas such as V1,
middle temporal area (MT),
and lateral intraparietal area (LIP)
<cit.>,
which are essential for motion perception.

The direction preference of V1 DS neurons depends on
the spatial pattern of LGN neurons to which they are connected
<cit.>.

Motion perception relies on the integration of
both spatial and temporal dimensions of information
<cit.>.

Spatial integration is achieved
by the larger receptive fields of MT neurons
<cit.>,
while temporal integration primarily relies on
the temporal integration properties of LIP neurons
<cit.>.



Based on the physiological structures and properties
of neurons and circuits in the LIP area,
Wang proposed a recurrent neural circuit model
<cit.>.

Within this model,
two groups of direction-selective excitatory neurons
are capable of integrating lower-level synaptic inputs
and performing cognitive tasks through
recurrent excitation and mutual inhibition mechanisms.

The model's attractor dynamics enhance its ability
to perform decision-making and working memory tasks,
replicate neural activity patterns observed in non-human primates,
and achieve performance consistent with experimental data.

Subsequent researches have extended the model by focusing on
its simplification, theoretical analyses and training methodologies
<cit.>.

These studies primarily emphasize
fitting and interpreting biological experimental data,
showcasing significant biological plausibility and interpretability.

However, neuron parameters are usually determined
based on averaged experimental data,
without considering the physiological characteristics
that underlie behavioral differences among individuals.

Moreover, the model has not been optimized
based on biological features found in primates,
which limits its impact and application in artificial intelligence.


Numerous studies have investigated the relationship
between the physiological or structural features
of the human brain and behavior

<cit.>.

These studies span various human behaviors,
including cognitive functions such as
working memory <cit.>,
language acquisition <cit.>,
theory of mind <cit.>,
and social functions like
social network size <cit.>.

Some research examines differences in
physiological characteristics between healthy and diseased groups,
for example, correlations between neurodegenerative diseases
and physiological indicators of white matter fiber tracts
<cit.>.

Modern neuroimaging technologies provide effective tools
for quantifying human brain physiological features, including
gray matter volume <cit.>,
cortical thickness <cit.>,
and myelination <cit.>
calculated from structural images,
measurements of white matter areas from diffusion images
<cit.>,
and the correlation of blood-oxygen-level-dependent (BOLD) signals
between brain areas in resting state,
known as resting-state functional connectivity (rest FC)
<cit.>.

These studies have revealed correlations between
brain structure, functional characteristics, and human behavior,
contributing to understanding disease mechanisms
<cit.>.

Despite these insights,
neuroimaging data has not been fully integrated into
the development or optimization of artificial neural network (ANN) models.

Integrating neuroimaging data with ANN models
offers a promising frontier for creating brain-inspired AI models
that replicates the behavioral performance of human subjects.




Addressing the gap in biologically inspired dynamics modeling
in artificial intelligence applications,
specifically the lack of tuning mechanisms
integrated with behavioral and neuroimaging data,
this study synthesizes known neural mechanisms
to construct a comprehensive biological neural dynamics model
of visual motion perception.

The model adheres to the physiological features of non-human primates
and facilitates cognitive decision-making behavior
in the Random Dot Kinematogram (RDK) task,
exhibiting biological-like behavioral and neuronal characteristics.

Furthermore, this study explores the correlation between
behavioral and neuroimaging features of human subjects
during visual decision tasks through
magnetic resonance imaging (MRI) techniques,
particularly focusing on the structural and functional characteristics
of corresponding brain regions in human experts.

Finally, we introduce a novel
neuroimaging-informed fine-tuning approach,
which leverages these neuroimaging characteristics
to optimize the artificial biological neural network model
and achieve notable improvements in performance.






§ RESULTS





 §.§ A Neural Dynamics Model of Visual Decision-Making with
Behavioral Performance and Neural Activities Similar to Primates



We constructed a neural dynamics model that
encompasses the four key brain areas of the dorsal visual pathway
(LGN, V1, MT, and LIP, Figs. <ref>a-c).
This model forms an artificial biological neural network
capable of motion perception and performing
the Random Dot Kinematogram (RDK) task.

The neurons in our model adopt the Leaky Integrate-and-Fire (LIF) model
and are interconnected through excitatory synapses (AMPA, NMDA)
and inhibitory synapses (GABA) (see  <ref>).

The spikes of different neuron groups were recorded
during the RDK task (Fig. <ref>d-f).



The electrophysiologic recordings show that
V1 neurons exhibit direction selectivity,
which is further enhanced in the MT neurons.

In the LIP area,
neurons that prefer a specific motion direction gradually dominate,
while those favoring the opposite direction are suppressed,
resulting in a “winner-take-all” effect.

Additionally, V1 and MT neuron activation increases with motion coherence
but stabilizes during the stimulus period.
In contrast, LIP neurons show a gradual ramping of activation,
with the ramping speed proportional to motion coherence
(Fig. <ref>d-f).

These simulated neural activities closely resemble those
recorded in electrophysiological experiments with macaque monkeys
<cit.>,
demonstrating the biological plausibility of our model.






The performance of our model was evaluated using the entire
Random Dot Kinematogram (RDK) dataset (see  <ref>).
We calculated the choice probability
and average reaction time for each coherence level.

Fig. <ref>a (upper panel)
presents the psychometric curve of the model (in blue),
which is similar to the psychometric curves of human subjects (in orange).
The model's sensitivity
(slope of the psychometric curve, k in Equation <ref>)
is 19.31± 0.17 (mean±SEM of 45 experiments),
significantly exceeding the average level of human subjects
(36 subjects, 15.1± 1.9, t=2.50, p=0.015, two-sample t-test).

Moreover, the model's decision time curve
aligns with the reaction times of human subjects.
As coherence increases (task difficulty decreases),
the decision time gradually becomes shorter,
as illustrated in Fig. <ref>a (lower panel).







In electrophysiological experiments,
microstimulation of specific brain regions
can induce behavioral changes
<cit.>.

To verify whether our neural dynamics model
exhibits similar characteristics to biological motion perception systems,
we conducted virtual cortical stimulation experiments.
External currents were injected
into different groups of neurons in our model
to investigate if the results matched those observed in biological experiments
<cit.>.


Applying continuous extra current
(040, Fig. <ref>b)
to MT neurons that prefer leftward motion
shifted the model's psychometric curve to the right,
indicating an increased leftward choice preference
(slope=-0.18, p<0.001),



This manipulation slightly reduced the model's sensitivity
(slope=-0.25, p<0.001),


and altered decision times,
decreasing for leftward and increasing for rightward motions,
aligning with observations in animal studies
<cit.>.

Conversely, applying extra current to neurons
that prefer rightward motion resulted in
a leftward shift of the psychometric curve
and a corresponding leftward shift in the decision time curve
(Fig. <ref>c,
intercept changed with external currents: slope=0.18, p<0.001;


sensitivity changes with external currents: slope=-0.26, p<0.001).



Moreover, when stimulating both groups of neurons
in the MT region without selection
(040),
the model's sensitivity slightly declined
(slope=-0.26, p<0.001),


but the decision time notably reduced
(Fig. <ref>e).

Correspondingly, stimulating all neurons in the V1 region
significantly altered the model's sensitivity
(slope=-0.50, p<0.001)
and decision time (Fig. <ref>d).



However, stimulating all excitatory neurons in the LIP area
did not affect the model's sensitivity
(slope=0.005, p=0.60)


or decision time (Fig. <ref>f).





 §.§ Model Optimization Inspired by Structural Connectivity of Human Experts



Structural connectivity estimated from neuroimaging techniques
reflects the mesoscopic properties of fibers,
which are directly related to behavioral outcomes
<cit.>.

To identify the key parameters
influencing the task performance,
we collected behavioral and neuroimaging data
from 36 human subjects performing the RDK tasks.



In the correlation analysis,
we found a significant negative correlation between
the mean fractional anisotropy (FA) value

of white matter in the left lateral occipital region
and subjects' behavioral performance
(r=-0.435, p=0.008, uncorrected Pearson correlation,
Fig. <ref>a).

This indicates that
subjects with lower mean FA values in this area
performed better in the behavioral experiment.

The lateral occipital region, located in the occipital lobe,
includes white matter pathways connecting
the primary visual cortex and the MT area.

Lower FA values may reflect reduced anisotropy
and predict a denser distribution of fiber bundles in this region.



The relative white matter volume in the right inferior parietal region
(i.e., the ratio of ROI volume to total brain volume,
estimated from MR T1 images)
was also significantly positively correlated
with subjects' behavioral performance
(r = 0.373, p = 0.025, uncorrected Pearson correlation,
Fig. <ref>c).

This suggests that subjects with better performance
had a larger white matter volume in this area,
indicating a broader range of fiber tracts.

The inferior parietal region,
located between the occipital and parietal lobes,
includes the white matter pathway connecting the MT and LIP regions,
which might be associated with decision-making in the brain.

These results demonstrate correlations between behavioral performance
and the structural features of the visual dorsal pathway.
No other structural features showed significant correlations.



In response to key parameters identified in MRI studies,
specifically the white matter connections
from V1 to MT and from MT to LIP,
which influence behavioral performance,
we employed a neuroimaging-informed fine-tuning approach
to adjust model parameters based on MRI analysis results.

For the mean FA of left lateral occipital area
(Fig. <ref>a),
we modified the average connection weight
of the V1-to-MT connections in the model
(Fig. <ref>b, upper panel) accordingly.

Increasing the connection weight at lower strengths
(mean of the connection matrix gradually increasing to 2)
enhanced model sensitivity and reduced decision times,
aligning with the observed behavioral correlations.

However, further increases in the connection weight
led to reduced decision times but a decline in sensitivity
(Fig. <ref>b, lower panel).

Similarly, adjusting the connection ratio between MT and LIP neurons
(Fig. <ref>d upper panel)
revealed that increasing the ratio from low levels
improved model sensitivity and reduced decision times,
aligning with neuroimaging and behavioral correlations.

However, beyond a certain range,
further increases in the connection ratio
decreased both model sensitivity and decision times
(Fig. <ref>d lower panel).









 §.§ Model Optimization Inspired by Functional Connectivity of Human Experts






The correlation between the functional connectivity indicators
and the behavioral performance in human subjects was also analyzed.

We found a significant positive correlation between
behavioral performance and resting-state functional connectivity (FC)
of the MT region and the anterior agranular insular complex (AAIC.
r=0.445, p=0.007, uncorrected Pearson correlation,
Fig. <ref>e).

The resting-state FC between the two brain regions
was more positively correlated in subjects with better performance,
which could indicate enhanced self-monitoring during the task.



The resting-state FC between the LIP region
and the suborbital region of the prefrontal lobe
was significantly negatively correlated with behavioral performance
(r=-0.472, p=0.004, uncorrected Pearson correlation,
Fig. <ref>g).

This suggests that a stronger negative correlation
between resting-state activities in these regions
is associated with better performance,
possibly reflecting higher engagement or attentiveness during the task.

No significant correlations were observed
between the resting-state FC features of other regions and subject behavior.



The functional MRI results highlighted brain areas
not included in the existing dynamics model.
These regions may provide top-down control
to areas involved in motion perception and decision-making,
thereby affecting perceptual decisions and behavior.

Based on findings from electrophysiological experiments
<cit.>,
we simulated the modulation of the MT region
by adjusting synaptic conductance in this module,
without adding new brain regions to the model.
The results are illustrated in Figs. <ref>f and h.

Increasing MT synaptic conductance enhanced
model sensitivity and reduced decision time,
aligning with observed subject behavior.

However, beyond a certain point,
further increases in connection strength
led to decreased sensitivity despite reduced decision time.

Similarly, we simulated the regulation of the LIP region
by adjusting connection efficiency between DS neurons.
This simulation, detailed in Fig.<ref>h,
showed that increasing connection efficiency
improved model performance and reduced decision time,
but excessive efficiency negatively affected performance.



These experiments illustrate how adjusting model parameters
based on structural and functional features
can enhance the neural dynamics model,
reflecting biological behaviors observed in neuroimaging studies.

This approach not only validates the model's biological plausibility
but also introduces a novel method for incorporating neuroimaging data
into the fine-tuning of artificial neural networks,
which we term neuroimaging-informed fine-tuning.






 §.§ Superior Robustness of Neural Dynamics Model Under Perturbation



To assess the robustness of our model,
we conducted four types of perturbation experiments
on both the neural dynamics model
and the convolutional neural network model,
as described in Section <ref>.

Noise was introduced by either
discarding or adding noise to the connection weights
or neurons in each module of the models.

Figs. <ref>b–i
show the changes in accuracy
of the CNN and neural dynamics models at each layer
with varying perturbation intensities
(see Table <ref> for statistics,
and Extended Fig. <ref> for changes in sensitivity).



From these figures,
it is evident that our neural dynamics model
performs better compared to the CNN model under perturbation.

The CNN's accuracy typically declines
more significantly with increased perturbation,
whereas the neural dynamics model remains more stable overall.

Notably, the model's performance remains largely unaffected
when noise is introduced into modules
corresponding to higher-level brain regions.

For example, discarding connections from MT to LIP
(Fig. <ref>f, red)
or adding noise to these connections
(Fig. <ref>g, red)
has minimal impact on model accuracy,
far less than the perturbation effects on the CNN
(Figs. <ref>b&c, red).
Adding noise to the input current of neurons
almost does not affect the neural dynamics model's performance
(Fig. <ref>i, except the purple).
Adding noise to LGN neurons causes model failure
(Figs. <ref>g&i, purple),
likely due to the sparse connections between V1 and LGN
leading to over-reliance on LGN signals.
Discarding neurons in the decision layer LIP
(Fig. <ref>g, blue)
also drastically reduces model performance,
as too few LIP neurons cannot maintain
the attractor states corresponding to decisions,
causing the model to rapidly degrade to a resting state
with a single attractor <cit.>,
effectively rendering it unable to make decisions.







Additionally, we found that changes in accuracy and sensitivity
with added noise in the neural dynamics model are consistent,
which contrasts with the behavior observed in CNN
In some cases, the CNN model retains high sensitivity,
but its accuracy significantly decreases with increasing model bias.
Furthermore, the CNN model exhibits greater variation with perturbation
(see scatter distribution in Figs. <ref>b–d,
see also Extended Figs. <ref>, <ref>),
while the neural dynamics model shows smaller bias
and maintains relatively stable performance even when disturbed.

Unexpectedly, reducing the number of LIP neurons
(Extended Fig. <ref>g, blue)
and adding noise to LIP neuron input currents
(Extended Fig. <ref>h, blue)
may improve model performance.
This is because our base model is not fully optimized;
weakening LIP neuron recurrent connections
or increasing Ornstein–Uhlenbeck noise may improve performance,
thus similar changes in perturbation
may also enhance model performance.







§ DISCUSSION




In this study, we developed a neural dynamics model
for motion perception and decision-making
that simulates the visual dorsal pathway.

The model incorporates biomimetic neurons and synapses
with dynamic characteristics and kinetic properties as its core elements.

These components enable the model to exhibit
neural activities and behavioral outputs
that are comparable to those of the biological brain,
including responses to virtual electrical stimulation.

By leveraging neuroimaging insights
such as the structural features identified from human experts,
we employed neuroimaging-informed fine-tuning
to optimize the model parameters,
leading to improved performance.

Compared to CNNs, our model achieves
similar performance with fewer parameters,
more closely aligns with biological data,
and exhibits greater robustness to perturbations.



Parameter optimization for neural dynamics model
is both critical and challenging due to
the complexity of nonlinear dynamic systems.

Previous studies have primarily employed
data fitting techniques using neural or behavioral data,
which are feasible for small-scale models.

However, as neural dynamics models increase in scale and complexity,
with larger number of parameters and higher computational demands,
optimization become significantly more computationally intensive and difficult.

while it is theoretically possible to optimize
key parameters in some simplified models through analytical methods,
such method is generally impractical for more complex models.

This study, for the first time,
explores the application of neuroimaging data in model optimization,
a process we term neuroimaging-informed fine-tuning.

While this approach may not achieve the global optimum,
it offers practical directions for parameter optimization
and significantly reduces the search space.

It is noteworthy that excessive adjustments to
parameters can impair model performance.
This observation implies the existence of an optimal range
for these parameters that maximizes performance.

Biological neural systems,
refined over billions of years of evolution,
regulate these parameters within such an optimal range,
with minor variations accounting for individual differences.



Targeting neuroimaging-informed fine-tuning,
we identified brain regions associated with
visual decision-making in human subjects
through behavioral and imaging experiments.

Subjects exhibiting superior performance in visual tasks
tend to have lower mean FA values in the occipital region,
indicating a higher degree of neural branching in the lateral occipital area.

Variations in white matter characteristics within the adults
are thought to result from inherent brain structure
and the long-term maturation process from childhood to adulthood
<cit.>,
suggesting that individuals with advanced visual capabilities
likely have undergone extensive training and development in visual function.

We also identified correlations between the functional connectivity of
MT–AAIC and LIP–suborbital with behavioral performance.
Variations in functional connectivity between subjects
are primarily influenced by their level of participation
and cognitive engagement during experiments <cit.>,
suggesting that expert subjects are more engaged and attentive during tasks.

We hypothesize that this process involves top-down regulation
from higher-level brain areas to task-related regions,
potentially enhancing synaptic transmission efficiency.

The structural and functional connectivity features
observed in human subjects account for their superior performance
from the perspectives of innate structure,
long-term development, and short-term participation.

These findings, applied to key parameter adjustments in the model,
indicate that the neural dynamics model aligns with biological intelligence
and shows good interpretability in physiological terms.
This proposes a potential path for enhancing the model's performance
through neuroimaging-informed fine-tuning.



In the model, LIP Neurons primarily rely on
differences in input currents for decision-making.
Larger input currents enable the model to reach
the decision threshold more quickly but reduce accuracy.

For instance, increasing the connection strength
between V1 and MT, between MT and LIP,
or increasing the number of MT neurons connected to LIP
enhances the total currents received by LIP neurons
(see Fig <ref>a).

This enhancement improves signal differentiation
among neuronal groups that prefer different directions,
thereby increasing accuracy (see Fig <ref>b&d, first half).

However, larger input currents cause attractors in decision space
to shift towards the diagonal
(see Fig. <ref>b, from 2 to 3) <cit.>.

In this scenario, inhibitory neurons are less effective
at suppressing DS neurons that prefer the opposite direction,
disrupting the winner-take-all dynamics
and deteriorating model performance
(see Figs. <ref>b&d, second half).



Similarly, modifying the recurrent connection weights
among LIP neuronal groups alters the energy landscape of the decision space.

Stronger recurrent connections eliminate the resting states attractor
and expand the size of the decision state attractors
(see Fig. <ref>c) <cit.>.

This modification improves performance during the initial phase
(see Fig. <ref>h, first half),
but excessively large decision attractors
can lead to rapid decision-making in the presence of noise,
thereby reducing accuracy (see Fig. <ref>h, second half).

Adjusting the number of LIP neurons also
alters the energy characteristics in the decision space.
If the parameters result in stable decision-state attractors,
the model can make decisions and maintain the states without external stimuli,
demonstrating a degree of working memory capability.

This enables decision-making with limited information
by integrating noisy evidence,
which can shorten decision time but introduces a risk of errors.

Conversely, if only one resting state attractor exists,
the model fails to reach the decision threshold
under any stimulus and cannot make decisions,
as evidenced by the perturbation experiment (Fig. <ref>h, blue),
where we deliberately chose the preferred direction
associated with higher firing rate neurons for decision.

In real-world scenarios,
the model would not make a choice in this situation.
Similarly, parameter adjustments can lead to another monostable mode,
where neuron groups preferring opposite directions
both exhibit high firing rates,
resulting in an inability to make reasonable decisions
(Figs. <ref>g&i, purple).








In 2-alternative forced choice (2-AFC) tasks,
researchers often model decision-making as evidence accumulation over time,
using approaches such as
the drift diffusion model (DDM)
<cit.>,
racing diffusion model <cit.>,
and linear ballistic accumulator (LBA) model <cit.>.

The temporal integration in the LIP module resembles the DDM
(see discussions in
<cit.>
), and can be directly related to it
<cit.>.

This type of evidence accumulation is not present in feedforward DCNN models.
Additionally, the dynamic nature of the LIP constitutes an attractor network
<cit.>,
where, through iterative processes,
the network's state stabilizes near attractors.

This differs fundamentally from traditional DCNNs,
which categorize through spatial partitioning within representational spaces,
and may explain why neural dynamics models
exhibit better stability and noise resistance.




In summary, integrating neural dynamics into AI models
can bridge the gap between biological intelligence and artificial systems,
leading to the development of next-generation AI
that is both interpretable and resilient like biological organisms.
By leveraging advances in neuroscience to construct models with neural dynamics,
we are able to create AI systems that better mimic biological behavior.

The neuroimaging-informed fine-tuning method enhances performance
while preserving the advantages of neural dynamics.
This approach ensures both high performance and explainability,
aligning with biological plausibility and computational efficiency.

This synthesis between advanced AI techniques
and in-depth neuroscientific insights represents a promising direction
for future research and development in both fields.






§ METHODS





 §.§ The RDK Dataset



Random Dot Kinematogram (RDK)
is a classic and common psychophysical paradigm
used to study visual motion perception.

In a typical RDK stimulus,
numerous small dots are randomly distributed on the screen
moving at different speeds in different directions,
without a clear pattern.

This randomness is crucial,
preventing subjects from relying solely on local motion cues
to judge overall motion characteristics.

Typically, a certain proportion of dots (signal dots)
move in a specified direction (target direction),
while the remaining dots (noise dots)
move randomly <cit.>.

Movement in a fixed direction is referred to as coherent motion,
and observers' ability to report the direction of coherent motion
increases with the percentage of coherent motion dots,
accompanied by shorter reaction times.

This proportion known as coherence,
is a measure of task difficulty
<cit.>,
making it the most important control parameter in RDK stimuli.

RDK can conveniently control the relative saliency of motion stimuli,
making it suitable for coherence threshold detection
or measuring changes in subject behavior with coherence.



In a 2-alternative forced choice (2-AFC) task,
with the proportion of coherent motion or
its logarithm as the independent variable,
a psychometric curve can be plotted
showing the probability of a subject choosing a certain direction.

This curve typically follows a sigmoid shape and can be described
by the equation (<ref>).

    p = 1/1+e^-kx+b.

Here, the slope k of the linear part
is referred to as sensitivity,
with higher sensitivity indicating a better ability
to distinguish between two directions of motion
and better behavioral performance.

The intercept b describes the bias of the decision-maker,
i.e., the extent to which the decision-maker
tends to choose one direction over the other.

A good decision-maker should have higher sensitivity and smaller bias.


To ensure consistency in data during model testing,
we generated different RDK stimuli using
the specialized neuroscience and experimental psychology software
PsychoPy <cit.>
with varying parameters and stored the stimuli
in the form of three-dimensional arrays.


Computer vision systems that use a camera as the perceptual input
typically measure input videos and images in pixels,
rather than in degrees as in studies of biological vision.
Therefore, the RDK animation used in this study is pixel-based.

Specifically, within a rectangular black background
with dimensions of 300 pixels by 300 pixels,
a 270-pixel diameter aperture is placed at the center.
Inside the aperture,
200 white circles with a diameter of 6 pixels are evenly distributed.
Some points (signal dots) are randomly selected to move left or right,
while the remaining points (noise dots) are randomly dispersed in all directions.
The circles move at a speed of 2 pixels per frame,
and their positions are randomly reset every 4 frames
or when they move out of the aperture.
Following these rules,
120 frames of images are continuously generated
and stored in 8-bit grayscale format
as a matrix of size 300x300x120.
This animation lasts for 2 seconds on a 60 monitor,
matching the stimulus duration of the human behavioral experiment.

The RDK dataset contains 100 levels of coherence
ranging from 099
with a step size of 1.

For each coherence level,
10 stimuli were generated for both leftward and rightward motion directions,
resulting in a total of 2000 stimuli.
This fine-grained variation in coherence levels allows for a detailed analysis
of the model's sensitivity to subtle differences in motion direction.





 §.§ The Neural Dynamics Model



The neural dynamics model constructed in this study
simulates four key areas of the dorsal pathway
involved in motion perception,
corresponding to the LGN, V1, MT and LIP regions
of the motion perception decision system
from primary to high levels (Fig. <ref>a).

We did not directly model lower visual pathway modules
(retina, ganglion cells, etc.) in detail,
but simplified them into temporal and spatial convolution calculations,
directly mapping visual input to the total input current
of LGN neurons as the model's input.



The LGN layer of the model was constructed
based on previous research
<cit.>.

The LGN is divided into two groups of ON and OFF neurons
adapting the Leaky Integrate-and-Fire (LIF) model,
each group containing 10000 neurons,
all with double Gaussian spatial receptive fields
(Equation <ref>, Fig. <ref>b middle).

These neurons cover a visual angle of 0.35
(x, y ∈ [-0.175, 0.175] in Equation <ref>),
corresponding to a 9x9 pixel area.

They are alternately overlapped and arranged regularly in space
(Fig. <ref>b top),
covering the entire 300x300 field of view,
forming a two-dimensional plane corresponding to the image space.

ON and OFF neurons have different temporal profiles,
with ON neurons responding slower than OFF neurons
(approximately 10,
Equation <ref>, Fig. <ref>b bottom).

Images of 300x300 pixels are convolved spatially
with the 9x9 spatial convolution kernels,
then the time step is reduced to 2
(120 frames expanded to 1000 frames)
through nearest-neighbor interpolation,
and temporal convolution with a sliding time window of 160 is performed.

At each time point,
two sets of 100x100 convolution results are obtained,
which serve as the stimulus-induced current
for each neuron in the ON and OFF neuron groups, respectively,
and together with Ornstein–Uhlenbeck noise
<cit.>
form the input current of LGN neurons.

    A(x, y) =
        α/πσ_α^2exp(
            -x^2+y^2/σ_α^2)
        - β/πσ_β^2exp(
            -x^2+y^2/σ_β^2)

where α = 1, β = 1,
σ_α = 0.0894,
σ_β = 0.1259
<cit.>.


    K(t) =
        αt^6/τ_0^7exp(
            -t/τ_0)
        - βt^6/τ_1^7exp(
            -t/τ_1)

where τ_0 = 3.66, τ_1 = 7.16,
α = 1, β = 0.8 for ON neurons, and
α = 1, β = 1 for OFF neurons
<cit.>.









The other three brain regions are composed of LIF neuron groups,
with each neuron group connected to the neurons
in the previous layer with specific structures and probabilities.

The V1 area has a total of 5000 neurons,
divided into two groups, namely G1 and G2.

Each V1 neuron receives input from
one ON cell and one OFF cell
arranged in a specific pattern
through AMPA synapses.

In the LGN projection received by the G1 group neurons,
the ON cells are always located to the left of the OFF cells.
Similarly, in the LGN projection received by the G2 group neurons,
the ON cells are always located to the right of the OFF cells
(Fig. <ref>c).

The MT area contains two groups of neurons, L and R,
each consisting of 400 neurons.

The L group neurons receive input from the G1 group neurons
within a certain receptive field,
while the R group neurons receive input from
the G2 group neurons within the same receptive field.

The connections between MT and V1 corresponding neuron groups
are also formed by AMPA synapses,
with a synaptic conductance of
g̅ = 2.0.


The LIP area is constructed according to the model
in literature <cit.>,
consisting of excitatory neuron groups A and B (each with 300 neurons)
and an inhibitory neuron group I (500 neurons).

The A group neurons receive random projections from L,
while the B group neurons receive projections from R
(50 of neurons for each group) with AMPA synapses.

Connections within excitatory groups A, B,
as well as connections from excitatory groups to inhibitory neuron group I,
are formed by AMPA and NMDA synapses with different temporal characteristics.

The inhibitory neuron group I inhibits neurons
in groups A and B through GABA synapses.

The connection strengths (synaptic conductance coefficients)
of the above connections follow a normal distribution
𝒩(g̅, 0.5g̅),
where the average conductance from MT to LIP is
g̅_MT = 0.1,
the average conductance among excitatory neurons in LIP is
g̅_AMPA = 0.05,
g̅_NMDA = 0.165,
the average conductance from excitatory neurons to inhibitory neurons in LIP is
g̅_AMPA = 0.04,
g̅_NMDA = 0.13.

If the connection strength between two neurons is less than 0,
there is no connection (Fig. <ref>c).
Additionally, the connection weights between neurons
with the same direction preference increase to
w = 1.3 times the original weight (Hebb-strengthened weight),
while the connection weights between neurons
with different direction preferences weaken to
w = 0.7 times the original weight (Hebb-weakened weight).


The neurons in the model are all LIF neurons,
a commonly used model in computational neuroscience
to approximate the behavior of biological neurons.

The resting membrane potential is set to V_r = -70,
and when the membrane potential of a neuron reaches -50,
it fires an action potential,
resetting to -55
during the refractory period <cit.>.

The LIF model was selected due to its
simplicity and computational efficiency,
while still capturing essential dynamics of neuronal firing.

In addition to synaptic currents and external currents,
each neuron receives Ornstein–Uhlenbeck noise,
a standard approach for modeling stochastic fluctuations in neuronal input.

The time constant of OU noise τ = 10,
and mean current of 400 with a variance of 100
were chosen to match the noise characteristics observed in biological neurons.

for LIP excitatory neurons,
a higher mean current of 550 was used to simulate
the enhanced excitatory drive from
other excitatory neurons without direction selectivity.

For excitatory neurons, the parameters are
C_m = 0.5,

g_l = 25,
and refractory period τ = 2.

For inhibitory neurons, the parameters are C_m = 0.2,
g_l = 20,
and refractory period τ = 1.

The AMPA, NMDA, and GABA synapses in the model
follow the settings in the literature
<cit.>.



















































We use the overall firing rate of neuron groups
A or B in the LIP region
as the basis for whether the model makes a decision.

When the average firing rate exceeds a threshold (30)
or when the stimulus finishes,
the model selects the direction preferred by
the neuron group with the higher firing rate as its final output.

By simulating the model with each stimulus in
the aforementioned RDK dataset,
we obtain the accuracy of the model's decisions
and the number of time steps taken for the decision (decision time)
under different coherence levels.

We can estimate a psychometric function through
least squares regression (Equation <ref>).
This method yields behavioral metrics,
including the model's psychometric curve and sensitivity indices.




Furthermore, our model,
which possesses a structure akin to biological systems
and neurons that mirror those of the biological nervous system,
allows for the execution of virtual electrophysiological experiments,
recording and analyzing the firing characteristics
of each neuron group in the model,
and even performing virtual electrical stimulation on neurons
by adding additional current inputs,
to study the characteristics of the model
(Figs. <ref>b&c).



Due to the randomness of RDK stimuli and the neural dynamics of the model,
to ensure the robustness of the results,
each stimulus was repeated twice in the model performance test,
and the neural dynamics model was re-initialized and repeated five times.

The selection probabilities and average decision times
for each coherence level were calculated and
estimated using the psychometric curve (Equation <ref>),
and the decision time curve was estimated using
a moving median smoothing algorithm with a coherence window of 10
(Fig. <ref>a upper panel).

For human subjects,
we conducted a bootstrap analysis on the behavioral data,
where the trials of each subject were randomly sampled with replacement,
and the median of 1000 bootstrap samples was calculated for each subject.
The final average performance over all subjects was plotted.





 §.§ A CNN Model for Motion Perception



A convolutional neural network (CNN) model (MotionNet) was built
similar in structure and scale to the neural dynamics model for comparison
(Fig. <ref>a).
The model receives video input with a resolution of 300x300 pixels,
processes it through spatial convolutions of 9x9 pixels
and temporal convolutions of 10 frames,
resulting in two 100x100 feature maps
matching the input of LGN neurons.
These feature maps are converted to two 50x50 maps
using 3x3 convolutional kernels.
The two channels are then processed with
11x11 convolutional kernels to generate two 20x20 feature maps.
After average pooling along the time dimension,
these features are mapped to a 400-dimensional vector by a linear layer,
and finally to output neurons using another linear layer.
All neurons utilize the ReLU activation function except the output layer,
which is passed through a softmax transformation
for fitting the one-hot encoded direction classification information.



To avoid overfitting the training data and ensure accurate model comparison,
MotionNet was not directly trained on the RDK dataset.
Instead, it utilized generated moving random images as the training data.
Specifically, a random grayscale image was generated and stretched randomly.
A window covering the image was moved in random directions
to obtain the necessary animation data.
The speed and direction of movement were randomly assigned,
and the training labels (supervision information, moving left or right)
were determined by the direction of movement along the horizontal axis.



MotionNet was trained with the cross-entropy loss function
and stochastic gradient descent (SGD) method.
The initial learning rate was set at 0.01,
reduced to 10 at the 5th and 15th epochs.
The momentum was set to 0.9,
with a batch size of 64,
and each epoch contained 500 batches.
Training stopped at the 20th epoch.

The model from the 20th epoch was selected for testing
based on its convergence and stability during training.

Similar analyses were then conducted on this model using the RDK dataset,
paralleling those applied to the neural dynamics model.

This approach ensured a fair comparison between
the two models' performance under the same experimental conditions.

Specifically, twenty consecutive frames were randomly selected from
a 120-frame animation as input for MotionNet.
The model's choices under various coherences were recorded
and psychometric curves were fitted.

For result stability, each trial was repeated twice.
Unlike the neural dynamics model, MotionNet,
being a CNN, lacks a concept of time,
so we focused on its psychometric curve
and related parameters (sensitivity).




 §.§ RDK Behavioral and Neuroimaging Experiments in Human Subjects



Subjects:
Thirty-six subjects participated in this RDK experiment
(12 males, 24 females, mean age 29 ±8 years).
Each subject conducted a behavioral experiment
with a series of random-dot motion-direction discrimination tasks
(consisting of a learning period, a practice period, and a test period)
and an MRI scanning experiment.

The MRI experiment consisted of
a high-resolution 3D-T1 structural scanning session,
a field-map scanning session for image distortion correction,
a functional-MRI session for localization of the MT region,
two functional-MRI sessions conducting the RDK tasks,
and a DWI session for white-matter fiber reconstruction.
The study was approved by the Ethics Committee,
and informed consent was obtained from each volunteer.


Behavioral experiment:
Random-dot stimuli were presented on a monitor
at a distance of 57 from the subject,
with a display resolution of 2560x1440 pixels
and a refresh rate of 60.
The stimulus program was written using PsychoPy
<cit.>.

The stimuli were a series of circular dots
(white dots presented on a black background).
Each trial was initialized with a fixation cross
(.33 x 0.33 dva,
lasting for a fixed duration of 500
plus a random duration sampled from a uniform distribution
with a maximum of 200).
Subjects were asked to gaze at the cross.
After that, a set of moving dots was presented
in a circular area of 5 dva diameter,
with a total number of 300 dots and
a diameter of 0.04 dva for each of these dots.
Some of the dots moved uniformly to the left or right,
while the rest of the dots moved randomly with a speed of

3.3.
The positions of all the dots were re-randomized every 5 frames
(i.e., the presentation of each dot lasted 5 frames).
Subjects were asked to recognize the uniform direction
of the moving dots and respond within
2000 (Extended Fig. <ref>a).

Each subject first performed a learning period.
The proportion of the uniformly moving dots for this period
was fixed at a coherence of 80
(i.e., 80 of the dots move in the same direction),
and the subject was informed with correct or incorrect feedback
after responding to the direction.

This period would not terminate
until the subject responded correctly 10 consecutive times.
After the learning period, the subject entered a practice period.

A total of 5 sessions were presented
and each session consisted of 70 trials
at randomized coherence levels
(1;2;3;4;5;6;8;10;15;20;25;30;40;50,
for a total of 14 coherence levels, each of which appeared 5 times).

During this period,
subjects were also informed of correct or incorrect feedback after responding.
Finally, the subject would be presented with a staircase test.
The coherence of the trials was set by the staircase method (3-down 1-up method),
with an initial coherence of 10,
a step size of 5 before the first reversal,
and a step size of 1 after the first reversal.
The behavioral experiment was completed after the 20th reversal or 150 trials in total.
A psychometric curve was then fitted for each subject's performance,
and a coherence corresponding to 79.4 accuracy
was estimated as the initial value of coherence in the following MRI experiment.


MRI experiment:
MRI experiments were performed using a MAGNETOM Prisma 3T
(Siemens Healthcare, Erlangen, Germany) MR scanner
with a 64-channel head-neck coil.
High-resolution 3D T1 structural images were acquired
using the MPRAGE sequence with the following scanning parameters:


TR=2530,
TE=3.34,
TI=1000,
flip angle=7,
iPAT=2,
FOV=[product-units=power]256x256,
number of slices=192,
resolution=1 isotropic.

Field-map images were scanned using a dual-echo 2D-GRE sequence
with the following parameters:


TR=747,
TE1=4.92,
TE2=7.38,
FOV=[product-units=power]208x208,
number of slices=72 slices,
resolution=2.0 isotropic.

Functional-MRI was performed using a 2D-GRE-EPI sequence
with simultaneous multi-slices scanning with the following parameters:


TR=1000,
TE=35,
flip angle=52,
FOV=[product-units=power]208x208,
number of slices=72 slices,
resolution=2.0 isotropic,
SMS factor=8.

The diffusion-weighted images were scanned using a 2D-SE-EPI sequence
with simultaneous multi-slices. Scanning parameters:
multiple b-values (b=0, 1000, 2500),
96 diffusion directions (32 b=1000 volumes and
64 b=2500 volumes <cit.>)
and 9 b=0 volumes,
TR=7400,
TE=70,
iPAT=2,
FOV=[product-units=power]204x204,
number of slices=96 slices,
resolution=1.4 isotropic,
SMS factor=2,
partial fourier=6/8,
phase-encoding direction=A>>P with only one b=0 volume P<<A.


The MT Localizer task used the classic random-dot
expansion-contraction paradigm
<cit.>.
The stimuli were presented on a monitor
at a distance of 170 from the subject,
with a resolution of 1920x1080 pixels
and a refresh rate of 60.
The program was written using PsychoPy.

The stimuli were a series of white circular dots presented on a black background.
The experimental paradigm was designed to be an  block design,
consisting of a 16 rest block
alternating with a 16 visual stimuli block
lasting for a total duration of 112.

A fixation cross was first presented
at the center of the screen for 16,
and subjects were asked to gaze at the cross;
three blocks were then presented.

Each block consisted of 200 dots (0.15 dva)
presented in a ring ranging from 0.5 dva to 9 dva.
A bigger dot of 0.3 dva was presented at the center of the screen,
and then all white dots except the central dot
started a contraction-expansion movement
with a speed of 8.
Subjects were asked to look at the central dot throughout the entire process.


The RDK test task in the MRI scanning used
the same paradigm as the behavioral test.
Subjects were asked to press two buttons with either the left or right hand.
The task adopted an  block design,
consisting of a 
4 rest block,
a 4 random viewing block,
and a 4 prompt-and-press block.
Each subject went through two sessions,
each consisting of 25 trials,
and a total duration of 304.
The experimental stimuli of each trial
were consistent with those presented in the behavioral experiment,
except that in the prompt-and-press block,
a new cue of `left-handed' or `right-handed' appeared
at the center of the screen
and the subjects followed the cue
to press the key with the corresponding hand
(Extended Fig. <ref>b).


In the MRI experiment,
we found that the RDK stimulus-induced activation was
mainly located in the primary visual cortex, MT, and LIP regions
(Extended Fig. <ref>c,
p<0.001, uncorrected, one-sample t-test).
This result was consistent with the ROIs modeled
in the neural dynamics model of motion perception.





 §.§ Neuroimaging Data Analysis Methods



High-resolution 3D–T1 structural images were
parcellated into gray matter and white matter
and reconstructed into cortex surfaces using Freesurfer
<cit.>.
A voxel-level segmentation of brain regions
based on the Destrieux atlas template
<cit.> was also obtained.
Several indices were estimated,
including the number of voxels per subregion,
cortical area, and average cortical thickness.
These processed data and structural features were used
in the subsequent correlation analysis.


The fMRI images were corrected using FSL,
including slice timing correction,
motion correction, and field-map correction.

The corrected images were then aligned to
the individual space of the T1 structural image.

The WM/GM/CSF binary maps were constructed
in the structural space for the subsequent GLM analysis
to remove white matter and cerebrospinal fluid voxels.

In the first-level analysis, the GLM model was used,
and the rest, stimuli, and response blocks of each trial
were considered as separate regressors.

The six motion parameters
(three translation and three rotation parameters per frame)
and the white matter and cerebrospinal fluid temporal signals
were used as nuisance regressors in the model.

The GLM model was fitted to each voxel,
and the contrast of `stimuli - rest' was constructed
to obtain the activation map in z-score.

In the second-level analysis, each subject's activation map in z-score
was projected to the individual cortical space
(reconstructed cortical surface),
and then projected to a standard cortical space (fsaverage5 template),
where the averaged activation map in z-score
was obtained by performing a one-sample t-test
(p<0.001, uncorrected, Extended Fig. <ref>c).



The resting-state functional connectivity in this study
was obtained from the task-fMRI data <cit.>.
By regressing out the task-dependent signals
and filtering the preprocessed results in the
0.010.08 band,
the background signals of the brain were obtained.

The resting-state functional connectivity matrix
was then calculated using two atlases:
the Destrieux structural atlas
and the Glasser functional atlas <cit.>.

Each element of the matrix was used to characterize each ROI pair
to correlate with the behavioral results.


Data preprocessing steps for diffusion images included
brain extraction, registration to T1 images,
field-map top-up correction, and eddy-current correction.

We applied a ball-and-stick model to regress the white matter voxels.

The model regression of voxels in the white-matter region
inferred the white-matter orientation distribution of single voxels,
and FA and MD values for each voxel were then obtained
<cit.>.

By registering to the white-matter segmentation image,
white-matter regions' mean FA and mean MD values
(Destrieux atlas, threshold: FA>0.2)
were calculated for the subsequent correlation analysis.
Mean FA and MD values were used to characterize
the structural connectivity of white-matter pathways
to correlate with the behavioral results.


To visualize the fiber bundles of the subjects,
fiber tract reconstruction was performed at the individual level
using the quantitative anisotropy of
the DSI Studio's gqi inference as a tracking index.

Segmented ROIs (lateral occipital and inferior parietal regions)
were used as spatial constraints for fiber tracking.
Deterministic fiber tracking was performed by the streamline Euler method
(tracking threshold =0,
angular threshold =0,
min length =30,
max length =300,
seed number =100000)
to obtain the morphology of fiber bundles
related to the visual decision-making brain regions
in the occipital and parietal lobes of the subjects.




 §.§ Neuroimaging-Informed Fine-Tuning



Data analysis:
The performance of the subjects in the behavioral RDK experiment
was quantitatively analyzed using psychometric curves,
and Pearson correlation analysis was performed
between the behavioral results and the structural and functional features
estimated from the MRI data.
For behavioral data,
a psychometric curve was fitted to each subject's staircase performance,
and perceptual ability was determined by taking

`1 - 79.4 accuracy'.


Pearson correlations were calculated between the above features from structural
(number of voxels per subregion, cortical area, average cortical thickness),
diffusion (mean FA, mean MD),
and functional (resting-state FC elements) images
and the perceptual ability indices of the subjects.

The ROIs were determined by the parcellation
and registration to the Glasser atlas
and the Destrieux atlas in each individual space.
We calculated the correlations between the ROIs' features
and the behavioral perceptual abilities
(p<0.01 as a threshold of statistical significance).



Connecting physiological data indicators found
in magnetic resonance imaging (MRI) with the model
is challenging due to the vastly different scales of neural elements involved.
In this study, a heuristic approach was used
to adjust and assess parameters potentially correlated with
structural and functional MRI indicators,
inspired by functional analogy.
White matter connections found in MRI were simulated
by adjusting the mean of the connection parameter distribution for V1 to MT
(0.23.0 with a step size of 0.2),
or by adjusting the proportion of connections between MT and LIP
(10100 with step 10).
According to Dale's principle,
non-positive connection weights were set to zero,
equivalent to removing those connections.

The adjusted model was retested on the RDK dataset five times,
and statistical analysis was performed on the sensitivity of
psychometric curves (slope)
to investigate the impact of parameter adjustments
on model performance and identify parameter combinations
that improve model performance.
Similarly, for functional connections found in fMRI,
the synaptic conductance of neurons in MT was altered
(0.010.15 with a step size of 0.01),
or the Hebb-strengthened weight
between LIP excitatory neurons was changed
(0.71.5 with a step size of 0.1)
to simulate the neural modulation from other brain regions.
The same statistical analysis was conducted
to examine the effect of parameter adjustments on model performance.




 §.§ Interference Resistance Experiment of the Model



In addition to requiring large amounts of data for training,
deep learning models are also prone to biases
and are easily affected by noise,
resulting in poor transferability, among other issues.
To address these problems, we designed a series of perturbation experiments
on the neural dynamics model to assess its noise resistance performance.



For each module of the neural dynamics model,
we separately tested the effects of deactivating a certain percentage of neurons,
discarding a certain percentage of synapses
(ranging from 090 with a step size of 10),
adding a certain level of Gaussian perturbation to the input current of neurons
(variance ranging from 0 to 2 times the
averaged absolute value of the group's input current,
normal distribution noise with a mean of 0),
and adding a certain level of Gaussian perturbation to the connection weights between neurons
(variance ranging from 0 to 2 times the
averaged absolute value of all connection weights,
normal distribution noise with a mean of 0).

For each parameter setting,
the model's behavioral performance was evaluated using the RDK dataset.

Our analysis concentrated on understanding
how changes in model parameters affected sensitivity and overall accuracy.

This evaluation highlights the significance of each parameter
and demonstrates the resiliency of our model.


For comparison, we conducted similar noise and damage experiments
on MotionNet as we did on the neural dynamics model.
For each layer of MotionNet,
we separately tested the effects of
deactivating a certain percentage of neurons,
discarding a certain percentage of connections
(ranging from 090 with a step size of 10),
adding a certain level of Gaussian perturbation to the output of neurons
(variance ranging from 0 to 2 times the
averaged absolute value of the group's neuron activation,
normal distribution noise with a mean of 0),
and adding a certain level of Gaussian perturbation to the connection weights between neurons
(variance ranging from 0 to 2 times the
averaged absolute value of all connection weights,
normal distribution noise with a mean of 0).
Due to the fact that the fully connected layer (fc4) of MotionNet
is mapped to the output layer with only two neurons,
the fc4 connections were neither deactivated nor perturbed during our experiment.
After each adjustment of the perturbation parameters,
we conducted a complete evaluation on the RDK dataset
and calculated the sensitivity of the current model
to the parameter changes.












§ EXTENDED DATA





























10








mcculloch1943logical
authorMcCulloch, W. S. & authorPitts, W.
titleA logical calculus of the ideas immanent in nervous
  activity.
journalThe Bulletin of Mathematical Biophysics
  volume5, pages115–133
  (year1943).


hopfield1982neural
authorHopfield, J. J.
titleNeural networks and physical systems with emergent
  collective computational abilities.
journalProceedings of the national academy of
  sciences volume79, pages2554–2558
  (year1982).

lecun1998gradient
authorLeCun, Y., authorBottou, L.,
  authorBengio, Y. & authorHaffner, P.
titleGradient-based learning applied to document
  recognition.
journalProceedings of the IEEE
  volume86, pages2278–2324
  (year1998).

lecun2015deep
authorLeCun, Y., authorBengio, Y. &
  authorHinton, G.
titleDeep learning.
journalNature volume521,
  pages436–444 (year2015).


hassabis2017neuroscience
authorHassabis, D., authorKumaran, D.,
  authorSummerfield, C. & authorBotvinick, M.
titleNeuroscience-Inspired Artificial Intelligence.
journalNeuron volume95,
  pages245–258 (year2017).


alex2012imagenet
authorKrizhevsky, A., authorSutskever, I. &
  authorHinton, G. E.
titleImagenet classification with deep convolutional
  neural networks.
journalAdvances in neural information processing
  systems volume25 (year2012).

szegedy2015going
authorSzegedy, C. et al.
titleGoing deeper with convolutions.
booktitle2015 IEEE
  Conference on Computer Vision and Pattern Recognition (CVPR),
  pages1–9 (publisherIEEE, addressBoston,
  MA, USA, year2015).


he2016resnet
authorHe, K., authorZhang, X., authorRen,
  S. & authorSun, J.
titleDeep Residual
  Learning for Image Recognition.
booktitle2016 IEEE
  Conference on Computer Vision and Pattern Recognition (CVPR),
  pages770–778 (publisherIEEE, addressLas
  Vegas, NV, USA, year2016).


ji2013conv3d
authorJi, S., authorXu, W., authorYang, M.
  & authorYu, K.
title3D Convolutional Neural Networks for Human
  Action Recognition.
journalIEEE Transactions on Pattern Analysis and
  Machine Intelligence volume35,
  pages221–231 (year2013).


maturana2015voxnet
authorMaturana, D. & authorScherer, S.
titleVoxNet: A
  3D Convolutional Neural Network for real-time object recognition.
booktitle2015
  IEEE/RSJ International Conference on Intelligent Robots and
  Systems (IROS), pages922–928 (publisherIEEE,
  addressHamburg, Germany, year2015).


mishkin1982contribution
authorMishkin, M. & authorUngerleider, L. G.
titleContribution of striate inputs to the visuospatial
  functions of parieto-preoccipital cortex in monkeys.
journalBehavioural Brain Research
  volume6, pages57–77 (year1982).

shadlen2001neural
authorShadlen, M. N. & authorNewsome, W. T.
titleNeural basis of a perceptual decision in the parietal
  cortex (area lip) of the rhesus monkey.
journalJournal of neurophysiology
  volume86, pages1916–1936
  (year2001).

katz2016dissociated
authorKatz, L. N., authorYates, J. L.,
  authorPillow, J. W. & authorHuk, A. C.
titleDissociated functional significance of
  decision-related activity in the primate dorsal stream.
journalNature volume535,
  pages285–288 (year2016).

yates2017functional
authorYates, J. L., authorPark, I. M.,
  authorKatz, L. N., authorPillow, J. W. &
  authorHuk, A. C.
titleFunctional dissection of signal and noise in MT and
  LIP during decision-making.
journalNature Neuroscience
  volume20, pages1285–1292
  (year2017).

chariker2021theory
authorChariker, L., authorShapley, R.,
  authorHawken, M. & authorYoung, L.-S.
titleA theory of direction selectivity for macaque primary
  visual cortex.
journalProceedings of the National Academy of
  Sciences volume118, pagese2105062118
  (year2021).

chariker2022computational
authorChariker, L., authorShapley, R.,
  authorHawken, M. & authorYoung, L.-S.
titleA computational model of direction selectivity in
  macaque v1 cortex based on dynamic differences between on and off pathways.
journalJournal of Neuroscience
  volume42, pages3365–3380
  (year2022).

burr2011motion
authorBurr, D. & authorThompson, P.
titleMotion psychophysics: 1985–2010.
journalVision research
  volume51, pages1431–1456
  (year2011).

amano2012human
authorAmano, K. et al.
titleHuman neural responses involved in spatial pooling of
  locally ambiguous motion signals.
journalJournal of Neurophysiology
  volume107, pages3493–3508
  (year2012).


nishida2018motion
authorNishida, S., authorKawabe, T.,
  authorSawayama, M. & authorFukiage, T.
titleMotion Perception: From Detection to
  Interpretation.
journalAnnual Review of Vision Science
  volume4, pages501–523
  (year2018).


shadlen1996motion
authorShadlen, M. N. & authorNewsome, W. T.
titleMotion perception: seeing and deciding.
journalProceedings of the National Academy of
  Sciences volume93, pages628–633
  (year1996).


wang2002probabilistic
authorWang, X.-J.
titleProbabilistic decision making by slow reverberation
  in cortical circuits.
journalNeuron volume36,
  pages955–968 (year2002).

wong2006recurrent
authorWong, K.-F. & authorWang, X.-J.
titleA recurrent network mechanism of time integration in
  perceptual decisions.
journalThe Journal of Neuroscience: The Official
  Journal of the Society for Neuroscience volume26,
  pages1314–1328 (year2006).

wang2008decision
authorWang, X.-J.
titleDecision making in recurrent neuronal circuits.
journalNeuron volume60,
  pages215–234 (year2008).

wang2020macroscopic
authorWang, X.-J.
titleMacroscopic gradients of synaptic excitation and
  inhibition in the neocortex.
journalNature Reviews. Neuroscience
  volume21, pages169–178
  (year2020).

song2017reward
authorSong, H. F., authorYang, G. R. &
  authorWang, X.-J.
titleReward-based training of recurrent neural networks
  for cognitive and value-based tasks.
journaleLife volume6,
  pagese21492 (year2017).

genon2022linking
authorGenon, S., authorEickhoff, S. B. &
  authorKharabian, S.
titleLinking interindividual variability in brain
  structure to behaviour.
journalNature Reviews Neuroscience
  volume23, pages307–318
  (year2022).

colom2007general
authorColom, R., authorJung, R. E. &
  authorHaier, R. J.
titleGeneral intelligence and memory span: evidence for a
  common neuroanatomic framework.
journalCognitive Neuropsychology
  volume24, pages867–878
  (year2007).

hamalainen2017bilingualism
authorHämäläinen, S., authorSairanen, V.,
  authorLeminen, A. & authorLehtonen, M.
titleBilingualism modulates the white matter structure of
  language-related pathways.
journalNeuroImage volume152,
  pages249–257 (year2017).

rice2015spontaneous
authorRice, K. & authorRedcay, E.
titleSpontaneous mentalizing captures variability in the
  cortical thickness of social brain regions.
journalSocial Cognitive and Affective Neuroscience
  volume10, pages327–334
  (year2015).

kanai2012online
authorKanai, R., authorBahrami, B.,
  authorRoylance, R. & authorRees, G.
titleOnline social network size is reflected in human
  brain structure.
journalProceedings of the Royal Society B:
  Biological Sciences volume279,
  pages1327–1334 (year2012).

forkel2022white
authorForkel, S. J., authorFriedrich, P.,
  authorThiebaut de Schotten, M. & authorHowells, H.
titleWhite matter variability, cognition, and disorders: a
  systematic review.
journalBrain Structure and Function
  pages1–16 (year2022).

schmithorst2010white
authorSchmithorst, V. J. & authorYuan, W.
titleWhite matter development during adolescence as shown
  by diffusion mri.
journalBrain and cognition
  volume72, pages16–25
  (year2010).

kantarci2017white
authorKantarci, K. et al.
titleWhite-matter integrity on dti and the pathologic
  staging of alzheimer's disease.
journalNeurobiology of aging
  volume56, pages172–179
  (year2017).

mechelli2005voxel
authorMechelli, A., authorPrice, C. J.,
  authorFriston, K. J. & authorAshburner, J.
titleVoxel-based morphometry of the human brain: methods
  and applications.
journalCurrent Medical Imaging
  volume1, pages105–113
  (year2005).

fischl2000measuring
authorFischl, B. & authorDale, A. M.
titleMeasuring the thickness of the human cerebral cortex
  from magnetic resonance images.
journalProceedings of the National Academy of
  Sciences volume97, pages11050–11055
  (year2000).

glasser2011mapping
authorGlasser, M. F. & authorVan Essen, D. C.
titleMapping human cortical areas in vivo based on myelin
  content as revealed by t1-and t2-weighted mri.
journalJournal of neuroscience
  volume31, pages11597–11616
  (year2011).

pievani2010assessment
authorPievani, M. et al.
titleAssessment of white matter tract damage in mild
  cognitive impairment and alzheimer's disease.
journalHuman brain mapping
  volume31, pages1862–1875
  (year2010).

sui2014function
authorSui, J., authorHuster, R., authorYu,
  Q., authorSegall, J. M. & authorCalhoun, V. D.
titleFunction–structure associations of the brain:
  evidence from multimodal connectivity and covariance studies.
journalNeuroimage volume102,
  pages11–23 (year2014).

wang2007altered
authorWang, K. et al.
titleAltered functional connectivity in early alzheimer's
  disease: A resting-state fmri study.
journalHuman brain mapping
  volume28, pages967–978
  (year2007).

song2008brain
authorSong, M. et al.
titleBrain spontaneous functional connectivity and
  intelligence.
journalNeuroimage volume41,
  pages1168–1176 (year2008).

hashemi2020bayesian
authorHashemi, M. et al.
titleThe Bayesian Virtual Epileptic Patient: A
  probabilistic framework designed to infer the spatial map of epileptogenicity
  in a personalized large-scale brain model of epilepsy spread.
journalNeuroImage volume217,
  pages116839 (year2020).

hanks2006microstimulation
authorHanks, T. D., authorDitterich, J. &
  authorShadlen, M. N.
titleMicrostimulation of macaque area lip affects
  decision-making in a motion discrimination task.
journalNature neuroscience
  volume9, pages682–689
  (year2006).

briggs2013attention
authorBriggs, F., authorMangun, G. R. &
  authorUsrey, W. M.
titleAttention enhances synaptic efficacy and the
  signal-to-noise ratio in neural circuits.
journalNature volume499,
  pages476–480 (year2013).

ye2021quantifying
authorYe, L. & authorLi, C.
titleQuantifying the Landscape of Decision Making
  From Spiking Neural Networks.
journalFrontiers in Computational Neuroscience
  volume15, pages740601
  (year2021).

lebel2008microstructural
authorLebel, C., authorWalker, L.,
  authorLeemans, A., authorPhillips, L. &
  authorBeaulieu, C.
titleMicrostructural maturation of the human brain from
  childhood to adulthood.
journalNeuroImage volume40,
  pages1044–1055 (year2008).

kuhn2021brain
authorKühn, S., authorForlim, C. G.,
  authorLender, A., authorWirtz, J. &
  authorGallinat, J.
titleBrain functional connectivity differs when viewing
  pictures from natural and built environments using fMRI resting state
  analysis.
journalScientific Reports
  volume11, pages4110 (year2021).

ratcliff1978theory
authorRatcliff, R.
titleA theory of memory retrieval.
journalPsychological review
  volume85, pages59 (year1978).

ratcliff2006modeling
authorRatcliff, R.
titleModeling response signal and response time data.
journalCognitive psychology
  volume53, pages195–237
  (year2006).

ratcliff2008diffusion
authorRatcliff, R. & authorMcKoon, G.
titleThe diffusion decision model: theory and data for
  two-choice decision tasks.
journalNeural computation
  volume20, pages873–922
  (year2008).

usher2001time
authorUsher, M. & authorMcClelland, J. L.
titleThe time course of perceptual choice: the leaky,
  competing accumulator model.
journalPsychological review
  volume108, pages550 (year2001).

tillman2020sequential
authorTillman, G., authorVan Zandt, T. &
  authorLogan, G. D.
titleSequential sampling models without random
  between-trial variability: the racing diffusion model of speeded decision
  making.
journalPsychonomic Bulletin & Review
  volume27, pages911–936
  (year2020).


brown2008simplest
authorBrown, S. D. & authorHeathcote, A.
titleThe simplest complete model of choice response time:
  Linear ballistic accumulation.
journalCognitive psychology
  volume57, pages153–178
  (year2008).

wei2017decision
authorWei, H., authorBu, Y. & authorDai,
  D.
titleA decision-making model based on a spiking neural
  circuit and synaptic plasticity.
journalCognitive Neurodynamics
  volume11, pages415–431
  (year2017).

roxin2008neurobiological
authorRoxin, A. & authorLedberg, A.
titleNeurobiological models of two-choice decision making
  can be reduced to a one-dimensional nonlinear diffusion equation.
journalPLoS computational biology
  volume4, pagese1000046
  (year2008).

umakantha2022relating
authorUmakantha, A., authorPurcell, B. A. &
  authorPalmeri, T. J.
titleRelating a Spiking Neural Network Model and
  the Diffusion Model of Decision-Making.
journalComputational Brain & Behavior
  volume5, pages279–301
  (year2022).

kim1999neural
authorKim, J.-N. & authorShadlen, M. N.
titleNeural correlates of a decision in the dorsolateral
  prefrontal cortex of the macaque.
journalNature neuroscience
  volume2, pages176–185
  (year1999).

roitman2002response
authorRoitman, J. D. & authorShadlen, M. N.
titleResponse of neurons in the lateral intraparietal area
  during a combined visual discrimination reaction time task.
journalJournal of neuroscience
  volume22, pages9475–9489
  (year2002).

britten1992analysis
authorBritten, K. H., authorShadlen, M. N.,
  authorNewsome, W. T. & authorMovshon, J. A.
titleThe analysis of visual motion: a comparison of
  neuronal and psychophysical performance.
journalJournal of Neuroscience
  volume12, pages4745–4765
  (year1992).

peirce2019psychopy2
authorPeirce, J. et al.
titlePsychopy2: Experiments in behavior made easy.
journalBehavior research methods
  volume51, pages195–203
  (year2019).

lansky2001ornstein
authorLánskỳ, P. & authorSacerdote, L.
titleThe ornstein–uhlenbeck neuronal model with
  signal-dependent noise.
journalPhysics Letters A
  volume285, pages132–140
  (year2001).

troyer1997physiological
authorTroyer, T. W. & authorMiller, K. D.
titlePhysiological gain leads to high isi variability in a
  simple model of a cortical regular spiking cell.
journalNeural computation
  volume9, pages971–983
  (year1997).

jahr1990voltage
authorJahr, C. E. & authorStevens, C. F.
titleVoltage dependence of nmda-activated macroscopic
  conductances predicted by single-channel kinetics.
journalJournal of Neuroscience
  volume10, pages3178–3182
  (year1990).

tian2022comprehensive
authorTian, Q. et al.
titleComprehensive diffusion mri dataset for in vivo human
  brain microstructure mapping using 300 mt/m gradients.
journalScientific Data
  volume9, pages7 (year2022).

michels2005visual
authorMichels, L., authorLappe, M. &
  authorVaina, L. M.
titleVisual areas involved in the perception of human
  movement from dynamic form analysis.
journalNeuroreport volume16,
  pages1037–1041 (year2005).

fischl2012freesurfer
authorFischl, B.
titleFreesurfer.
journalNeuroimage volume62,
  pages774–781 (year2012).

destrieux2010automatic
authorDestrieux, C., authorFischl, B.,
  authorDale, A. & authorHalgren, E.
titleAutomatic parcellation of human cortical gyri and
  sulci using standard anatomical nomenclature.
journalNeuroimage volume53,
  pages1–15 (year2010).

fox2016combining
authorFox, M. D. et al.
titleCombining task-evoked and spontaneous activity to
  improve pre-operative brain mapping with fmri.
journalNeuroImage volume124,
  pages714–723 (year2016).

glasser2016multi
authorGlasser, M. F. et al.
titleA multi-modal parcellation of human cerebral cortex.
journalNature volume536,
  pages171–178 (year2016).




