Our subject-dependent LDA-based classifier provided the highest accuracy mostly using 3–10 and 3–15 Hz frequency power bands. Spectral analysis indicated changes at theta band (3–6 Hz) as well as at alpha band (8–12 Hz) between different mental states in frontoparietal regions. Alpha band increase in the frontal area was observed for all conditions, whereas both theta and alpha bands increase in the parietal regions was larger for reading and calculation with respect to spatial navigation. Accordingly, high workload, that included both reading and calculation, compared to low workload—spatial navigation—showed significant power differences at 3–6 and 8–12 Hz bands. In particular theta band maximum was observed in the left frontal area, while alpha band peaked in the left frontal and bilateral parietal regions.

Despite intersubject variability and our small sample size, the observed alpha band increase in the bilateral parietal areas is in line with previous results reporting alpha changes in bilateral parietal and occipital brain regions associated with mental workload, task engagement or attention (Humphrey and Kramer, 1994; Pope et al., 1995; Kohlmorgen et al., 2007). The measured changes in the theta band power are also in line with previous studies indicating that theta oscillations are related to spatial navigation as well as encoding and retrieval of spatial information (Kahana et al., 1999; Bischof and Boulanger, 2003). In particular, high amplitude theta activity, mainly in the left frontal and right temporal cortices, has been measured during navigation in a virtual maze (Kahana et al., 1999). Other studies corroborated these results and showed that the frequency of theta episodes is directly associated with the difficulty of maze navigation (Bischof and Boulanger, 2003). In light of previous studies our results indicate that indeed the here adopted reading and calculation tasks required increased allocation of mental resources with respect to spatial navigation.

In addition, we observed delta frequencies (3 Hz) power changes during all conditions in frontal regions. These results are in line with previous studies indicating increased EEG oscillations in the range 1–3.5 Hz in frontal regions associated with different cognitive processes (Harmony, 2013), in particular during internal concentration and calculation (Fernandez et al., 1995).

Our mental states classifiers can equally be employed for real-time analysis of frequency bands. Online monitoring of alpha and theta bands power would be important for assessing participants' performance as these bands reflect cognitive and memory processing (Klimesch, 1999). In online MR-combined BCI applications the information stream provided to the user could be adapted to the current workload as indicated by alpha and theta oscillations. Kohlmorgen et al. (2007) proposed the use of ratios of activity in alpha (8–12 Hz) or theta (3–7 Hz) bands to compute an index of the user task engagement.

Alpha band is also critical for visual perception, in particular medium and lower amplitudes can reflect improved performance in somatosensory and visual discrimination tasks (Pfurtscheller and Lopes Da Silva, 1999; Hanslmayr et al., 2005; Palva and Palva, 2007; Van Dijk et al., 2008). Moreover, decrease in the alpha frequencies (8–12 Hz) before target onset was associated with augmented visual target detection (Ergenoglu et al., 2004). Further studies confirmed this observation by showing that the amplitude of prestimulus ∼10 Hz oscillations correlated with the detection of the upcoming target: the smaller the amplitude, the more likely the target would be detected (Van Dijk et al., 2008; Busch et al., 2009; Mathewson et al., 2009). On the contrary, stronger pre-stimulus alpha frequency band amplitude has been linked to increased cognitive performance (Neubauer and Freudenthaler, 1995; Klimesch, 1999). Thus, online inspection of alpha wave oscillations might be used for triggering stimuli presentation so as to optimize stimulus detection, as well as for improving interpretation of novel information and data mining since alpha band activity has also been associated with learning (Klimesch, 1999).

In conclusion, our LDA classifier is sufficiently flexible and powerful for the implementation of a MR-combined BCI system. Successful classification of mental states based on subject-specific single trials EEG indicates the possibility to combine BCI technology with the XIM so that brain activity could drive the adaptation of data representation. A possible way to refine our BCI in MR is the use of an asynchronous modality where participants do not need to follow a fixed repetitive scheme to switch from one mental task to another one. Asynchronous BCI, allowing individuals to decide when to perform a mental task and when to stop it and switch to another one, are more flexible and adaptive to different scenarios (Millán et al., 2002; Millan Jdel and Mourino, 2003). Ultimately, this approach would permit to model the user experience in the XIM as common product between the initial data representations and the changes made interactively as consequences of users' neurophysiological signals associated with spontaneous behavior.

# **ACKNOWLEDGMENTS**

The present study was supported by EU grants: FP7-ICT-2009- 258749 CEEDs: The Collective Experience of Empathic Data Systems; FP7-ICT-2013- 609593 BNCI Horizon 2020. The Future of Brain/Neural Computer Interaction: Horizon 2020; Italian Ministry of Health, GR-2009-1591908.

# **REFERENCES**

**Conflict of Interest Statement:** The Review Editor Dr. Emanuele Pasqualotto declares that, despite having collaborated with some of the authors, the review process was handled objectively. The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 07 June 2014; accepted: 12 November 2014; published online: 27 November 2014.*

*Citation: De Massari D, Pacheco D, Malekshahi R, Betella A, Verschure PFMJ, Birbaumer N and Caria A (2014) Fast mental states decoding in mixed reality. Front. Behav. Neurosci. 8:415. doi: 10.3389/fnbeh.2014.00415*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience.*

*Copyright © 2014 De Massari, Pacheco, Malekshahi, Betella, Verschure, Birbaumer and Caria. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Learned EEG-based brain self-regulation of motor-related oscillations during application of transcranial electric brain stimulation: feasibility and limitations

# *Surjo R. Soekadar 1,2,3\*, Matthias Witkowski 1,2,3, Eliana G. Cossio2,3, Niels Birbaumer 3,4 and Leonardo G. Cohen1*

*1 Human Cortical Physiology and Neurorehabilitation Section, NINDS, NIH, Bethesda, MD, USA*

*2 Applied Neurotechnology Lab, Department of Psychiatry and Psychotherapy, University Hospital Tübingen, Tübingen, Germany*

*3 Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany*

*4 Ospedale san Camillo, IRCCS, Venice, Italy*

#### *Edited by:*

*Andreas Meyer-Lindenberg, Central Institute of Mental Health, Germany*

#### *Reviewed by:*

*Alexander Fingelkurts, BM-Science - Brain and Mind Technologies Research Centre, Finland Andrew A. Fingelkurts, BM-Science - Brain and Mind Technologies Research Centre, Finland*

#### *\*Correspondence:*

*Surjo R. Soekadar, Applied Neurotechnology Lab, Department of Psychiatry and Psychotherapy, University Hospital of Tübingen, Calwerstr. 14, 72076 Tübingen, Germany e-mail: surjo.soekadar@ uni-tuebingen.de*

**Objective:** Transcranial direct current stimulation (tDCS) improves motor learning and can affect emotional processing and attention. However, it is unclear whether learned electroencephalography (EEG)-based brain-machine interface (BMI) control during tDCS is feasible, how application of transcranial electric currents during BMI control would interfere with feature-extraction of physiological brain signals and how it affects brain control performance. Here we tested this combination and evaluated stimulation-dependent artifacts across different EEG frequencies and stability of motor imagery-based BMI control.

**Approach:** Ten healthy volunteers were invited to two BMI-sessions, each comprising two 60-trial blocks. During the trials, learned desynchronization of mu-rhythms (8–15 Hz) associated with motor imagery (MI) recorded over C4 was translated into online cursor movements on a computer screen. During block 2, either sham (session A) or anodal tDCS (session B) was applied at 1 mA with the stimulation electrode placed 1 cm anterior of C4.

**Main results:** tDCS was associated with a significant signal power increase in the lower frequencies most evident in the signal spectrum of the EEG channel closest to the stimulation electrode. Stimulation-dependent signal power increase exhibited a decay of 12 dB per decade, leaving frequencies above 9 Hz unaffected. Analysis of BMI control performance did not indicate a difference between blocks and tDCS conditions.

**Conclusion:** Application of tDCS during learned EEG-based self-regulation of brain oscillations above 9 Hz is feasible and safe, and might improve applicability of BMI systems.

**Keywords: brain-machine interface (BMI) control, motor imagery, EEG, transcranial electric stimulation (TES), stimulation artifacts**

# **INTRODUCTION**

Brain-machine interfaces (BMI) translate physiological features of brain activity associated with the user's intention or state into control signals of a particular device or computer (Birbaumer and Cohen, 2007). BMIs are increasingly used in the context of neurorehabilitation, e.g., after stroke (Ang et al., 2011; Ramos-Murguialday et al., 2013) with two main purposes: (1) to drive assistive devices or computers that surrogate a lost or impeded function, or (2) to facilitate motor recovery related to BMIbased neurofeedback training (Wang et al., 2010; Soekadar et al., 2011). The best-established BMI in stroke is based on volitional modulation of mu-rhythm (8–15 Hz, also called sensorimotor rhythm, SMR) recorded over the sensori-motor cortex using electro- or magnetoencephalography (EEG/MEG) (Buch et al., 2008; Ramos-Murguialday et al., 2013). Motor planning, imagery or execution is associated with a decrease in amplitude of mu-rhythm (Pfurtscheller and Neuper, 1997; Llanos et al., 2013). This decrease can be quantified as event-related desynchronization (ERD) (Pfurtscheller and Aranibar, 1979) and used by a BMI system to control an exoskeleton moving a patient's paralyzed hand (Soekadar et al., 2011).

Depending on the physiological features, several sessions of BMI training are typically required to achieve stable and reliable BMI control (Soekadar et al., 2011). Training-related BMI learning, however, is often substantially impeded in patients with brain lesions, such as stroke, traumatic brain injury or other brain disorders resulting in compromised learning capacity (Buch et al., 2008). Thus, it would be desirable to identify strategies to improve BMI learning and performance in these populations.

Several studies suggest that the application of weak electric currents in the form of transcranial direct current stimulation (tDCS) can improve motor learning (Zimerman et al., 2012), cognition (Metuki et al., 2012), memory consolidation (Dockery et al., 2009; Reis et al., 2009) and emotional processing (Nitsche et al., 2012). While the underlying physiological mechanisms of these effects are still unknown, it has been shown that tDCS can result in polarity-dependent shifts of membrane potentials and modulate cortical excitability (Nitsche and Paulus, 2000; Pellicciari et al., 2013). Recent work indicates that anodal tDCS applied before a motor imagery-based BMI session might improve BMI control (Wei et al., 2013) and modulate motor imagery-related mu-ERD (Matsumoto et al., 2010). A series of experiments suggests that learning is faster when tDCS is applied during a task compared to when it is applied prior to the task (Stagg et al., 2011). It was concluded that such timingdependency might be of particular importance for the development of plasticity-inducing stimulation protocols (Nitsche et al., 2007). Furthermore, in addition to being time-saving in the clinical context when applied during motor or BMI training, direct effects of tDCS on brain oscillatory activity in the context of such training can be assessed, allowing for a better understanding of how transcranial electric currents modulate task-related brain activity.

While a novel strategy using magnetoencephalography (MEG) for *in vivo* assessment of neuromagnetic brain oscillations during transcranial electric brain stimulation was recently introduced (Soekadar et al., 2013a), the immobility and costs of MEG constrains broader clinical applications, e.g., for BMI training in the context of neurorehabilitation. EEG, in contrast, is inexpensive and widely used in clinical environments. However, the application of electric currents at voltages of up to 20 volts to the human head while recording EEG in the range of millivolts is particularly challenging. This is due to two main reasons: (1) Most EEG systems use wet electrodes to improve conduction between electrodes and the skin. Likewise, the application of electric currents to the head also requires good conduction, mostly provided by using electrolyte gels or pastes. If the conductive agents used for the EEG electrodes and stimulating electrodes are in direct contact, the stimulation currents will saturate the EEG amplifier and impede any physiological recordings. (2) Electric stimulation of the head can be associated with additional stimulation-depended signals picked up by the EEG amplifiers that might reduce classification accuracy in BMI control. This would be disadvantageous for both assistive and neurofeedback BMI applications. While denoting any stimulation-dependent signal during neurophysiological recordings of brain activity as artifacts or noise (i.e., an irregular fluctuation of the measured signal that does not contain meaningful information or obscures the information of interest) is somewhat appealing, it should be emphasized that the separation between the physiological responses or effects of the stimulation and stimulator-dependent noise is difficult. While recently systems were introduced that integrate electric stimulation and recording of EEG (e.g., Starstim® by Neuroelectrics, Barcelona, Spain) allowing for simultaneous EEG monitoring during tDCS (Schestatsky et al., 2013), it remained unclear how EEG signals across different frequency bands recorded at different distances from the stimulating electrode are affected by tDCS, and whether learned electroencephalography (EEG)-based brain-machine interface (BMI) control e.g., using motor imagery (MI) during tDCS is feasible, reliable, and safe.

Here, we investigated such a combination choosing an electrode montage that places the stimulation electrode as close as 1 cm to the EEG electrode recording brain oscillations used for BMI control (see **Figure 1A**). To allow generalization to other BMI paradigms, we first characterized stimulation-dependent signals at different EEG locations. We then compared the signal power in different frequencies (delta: 0.1–4 Hz; theta: 4–9 Hz; alpha: 9–15 Hz; and beta: 15–30 Hz) before stimulation and during stimulation, and identified those frequencies significantly influenced by stimulation-dependent signals. Finally, we calculated the effect size of the stimulation conditions for each frequency band. Potential side effects of stimulation, like tingling, itching or burning (Brunoni et al., 2011) were assessed throughout the sessions.

# **MATERIALS AND METHODS**

# **PARTICIPANTS AND EXPERIMENTAL DESIGN**

Ten healthy volunteers (5 males, 5 females, mean age: 26.7 years ± 4.3) exhibiting reliable and stable motor imagery-based BMI control in a previous investigation (Soekadar et al., 2011) were invited for two BMI sessions on consecutive days. All participants were right handed according to the Edinburgh Handedness Inventory (Oldfield, 1971). Each session was divided into two blocks of 60 trials (block 1, block 2). While no stimulation was delivered during block 1 of both sessions, either sham stimulation (session A) or anodal stimulation (session B) was applied during block 2. Sessions were conducted in random order. All participants gave written informed consent before entering the study. The study protocol was approved by the National Institute of Neurologic Disorders and Stroke Institutional Review Board (NINDS IRB).

### **ELECTROENCEPHALOGRAPHIC (EEG)-RECORDINGS AND FEEDBACK OF mu-EVENT-RELATED DESYNCHRONIZATION (mu-ERD)**

Participants were seated comfortably in an armchair facing a computer monitor. EEG was recorded from the following conventional EEG-recording sites (F3, FC5, C3, P3, Fz, AFz, Cz, FCz, F4, FC6, C4, P4 according to the international 10/20 system) using a 12-channel active electrode EEG system (Acti-cap® and BrainAmp MRplus®, BrainProducts, Gilching, Germany) with the reference electrode placed at FCz and the ground electrode at AFz. For translation of neurophysiological signals into visual feedback, BCI2000, a multipurpose standard BCI platform, was used (Schalk et al., 2004).

All participants were instructed to use visuo-kinesthetic MI of moving their left hand modulating right-hemispheric murhythms when they saw the visual cue indicating the initialization of each trial.

To rule out overt movements during MI, electromyography (EMG) was recorded from the first dorsal interosseus muscle (FDI), extensor digitorum communis (EDC), extensor carpi ulnaris (ECU) and flexor carpi radialis (FCR) during the sessions. Skin/electrode resistance was kept below 10 k-. EMG signals were recorded at a sampling rate of 1 kHz and high-pass filtered at 2 Hz (BrainAmp ExG®, Brainproducts, Gilching, Germany). Trials in which EMG activity exceeded that recorded during rest by two standard deviations were interrupted and excluded from further analysis. The number of excluded trials in which

EMG activity was present was comparable across participants and ranged between 5 and 10%.

yellow circle). The reference stimulation electrode was placed over the left supraorbital region (blue). **(B)** BMI paradigm. Electric brain activity recorded

Amplitude of mu-rhythm event-related desynchronization (mu-ERD) and synchronization (mu-ERS) was visually fed back to the participants during each trial (see **Figure 1B**). While increasing mu-ERD was indicated by up-movements of a ball above the horizontal midline of the screen, mu-ERS resulted in down-movements below the midline. Visual feedback was continuously updated every 100 ms. Computation of mu-ERD/mu-ERS included the power spectrum estimation (an autoregressive model of order 16 using the Yule–Walker algorithm) of each incoming sample at the optimal frequency for mu-ERD detection. The optimal frequency was identified in a screening run before the first session (11 Hz in all participants). Resulting values were compared with mean power values of the preceding inter-trial-intervals (ITI) that were continuously updated during BMI control according to the method of Pfurtscheller and Aranibar (1979) and as previously implemented into a mu-ERD BMI system (Soekadar et al., 2011):

$$\overline{R} = \frac{1}{N} \sum\_{t=1}^{N} R\_t \tag{1}$$

$$ERD\left(t\right) = \frac{T\_t}{\overline{R}} - 1\tag{2}$$

Where *t* represents the recorded sample block, *Tt* the eventrelated task condition period and *Rt* the power estimate in a given frequency band of *t*. *R* (reference value) represents power estimates during the rest (task-free) condition.

Each session consisted of 2 blocks with 60 trials. After the first block either sham stimulation (session A) or anodal stimulation (session B) was applied during the following 60 trials in a

instructed to keep the ball above the dotted horizontal line during the task

# **TRANSCRANIAL DIRECT CURRENT STIMULATION (tDCS)**

to hit the target (indicated by red bar).

randomized order.

tDCS was applied via two conducting 4 × 6 cm rubber electrodes and attached to the participant's head using a conductive paste (Ten20®, D.O. Deaver, Aurora, CO, USA). The adhesive features of this paste prevented any sliding or dislocation of the electrodes during the attachment of the EEG cap. A bipolar electrode montage (right M1 and left supraorbital area) was used (see **Figure 1A**) to deliver a current of 1 mA (current density 0.04 mA/cm2; total charge 0.048 C/cm2 using the DC-STIMULATOR PLUS, neuroConn GmbH, Germany). During sham stimulation, the DC stimulator was set up to apply an anodal current for 15 s and—at the offset—decrease stimulation intensity in a ramp-like fashion, a method shown to achieve a good level of blinding (Gandiga et al., 2006). Prior to the first block of the first session, the M1 hand-area was localized in all participants based on the motor evoked potential (MEP) hotspot of the first digit's interosseus muscle (FDI) using transcranial magnetic stimulation (TMS). After the end of block 2 of sessions A and B, all participants rated possible discomfort, pain, tingling, itching or burning associated with the stimulation on visual analog scales (VAS) to assess safety and tolerability of tDCS during BMI control.

#### **OFFLINE ANALYSIS**

For all outcome measures, assumption of a normal distribution (Shapiro–Wilk test of normality) was tested. Parametric tests were corrected by Greenhouse-Geisser estimates if Mauchly's sphericity test indicated significance. To compare signal power across conditions, fast Fourier transformations were performed for all EEG data collected during the first and second blocks of session A and session B (see **Figure 2**). A repeated-measures ANOVA (rmANOVA) with factors "block" (block 1, block 2) and "frequency" (delta, theta, alpha, beta) was performed based on the raw EEG signal power recorded from electrode C4 (in immediate proximity to the stimulation electrode) and P3 (at ∼8–10 cm distance from the stimulation electrode) to investigate stimulation-dependent changes in different EEG channels. *Posthoc* paired-samples Students *t*-tests were used when applicable and corrected for multiple comparisons (Bonferroni). Timefrequency representations (TFR) were plotted for both conditions and tested for statistical differences at two different electrode positions (C4 in immediate proximity of the stimulation electrode, and P3) using a cluster-based permutation test (Maris and Oostenveld, 2007). Effect size of stimulation-dependent signal differences in each frequency band was calculated using Cohen's *d* transformed into a regression coefficient *r* where *r* < 0.3 is considered a small, *r* < 0.5 is considered a medium, and *r* > 0.5 is considered a large effect (Cohen, 1988). A rmANOVA with factors "session" (session A, session B) and "block" (block 1, block 2) was used to evaluate changes of BMI control across sessions in the absence (block 1) and presence (block 2) of anodal tDCS. BMI control was defined as the time during each trial in which the ball was above midline (indicating mu-ERD). All analyses were performed in SPSS 17.0. Significance level was set to *p* < 0.05. Variance is defined as the standard error of the mean.

# **RESULTS**

### **STIMULATION-DEPENDENT CHANGES OF SIGNAL POWER ACROSS DIFFERENT FREQUENCY BANDS AND CONDITIONS**

#### *Stimulation electrode in close proximity (***∼***1 cm) to the recording EEG channel (C4)*

While rmANOVA of data recorded during session A (sham stimulation during block 2) from electrode C4 (**Figure 2A**) showed a main effect for "frequency" [*F*(1, 27) = 37.156, *p* < 0.0001], indicating an expected difference between power values across the investigated frequency bands, there was no effect for "block" [*F*(1, 27) = 0.233, *p* = 0.641] and no interaction between the factors (*p* = 0.188). *Post-hoc t*-tests showed no significant differences between power values of block 1 and 2 in any frequency band (delta: *p* = 0.729; theta: *p* = 0.963; alpha: *p* = 0.946; beta: *p* = 0.784).

significant during sham stimulation (left column), anodal stimulation

right panel), while alpha (9–15 Hz) and beta (15–30 Hz) frequencies showed no difference between conditions.

When analyzing data of session B (anodal stimulation during block 2) from the same location, we found a main effect for both "frequency" [*F*(1, 27) = 77.536, *p* < 0.0001] and "block" [*F*(1, 27) = 33.16, *p* < 0.0001] and an interaction between the two [*F*(1, 3) = 39.584, *p* < 0.0001]. *Post-hoc t*-tests showed a significant difference between power values of block 1 and 2 in the delta band (*p* < 0.0001) and a trend in the theta band (*p* = 0.068), but no significant differences in the alpha (*p* = 0.482) or beta (beta: *p* = 0.336) bands.

## *Stimulation electrode at* **∼***8–10 cm distance to the recording EEG channel (P3)*

Analysis of EEG data recorded ∼8–10 cm away from the stimulation electrode (P3) (**Figure 2B**) during session A, revealed a main effect for "frequency" [*F*(1, 27) = 28.28, *p* < 0.0001], but no effect for "block" [*F*(1, 27) = 1.202, *p* = 0.301] and no interaction between the factors (*p* = 0.056). We found no significant differences between power values of block 1 and 2 in any frequency band (delta: *p* = 0.655; theta: *p* = 0.984; alpha: *p* = 0.937; beta: *p* = 0.923).

The same analysis performed for data acquired during session B (anodal stimulation during block 2) at electrode position P3, showed a main effect for both, "frequency" [*F*(1, 27) = 34.39, *p* < 0.0001] and "block" [*F*(1, 27) = 11.34, *p* < 0.01] and a significant interaction between the two [*F*(1, 3) = 14.152, *p* < 0.0001]. *Posthoc t*-tests indicated a significant difference between power values of block 1 and 2 in the delta band (*p* < 0.05), but not in the theta (*p* = 0.217), alpha (*p* = 0.445) or beta (beta: *p* = 0.482) bands.

Fourier transformations and TFR calculated for both sessions and blocks separately showed an increase in signal power during the second block of session B (see **Figures 2**, **3**), which was highest in the slow frequency bands showing a decay of ∼12 dB per decade. While not significant, we found a slight increase in broadband noise across all frequencies during anodal stimulation (second block, session B; see **Figure 3B**, right panel). Statistical analysis using a non-parametric cluster-based permutation test indicated significant stimulation-dependent changes of signal power in frequencies below 8 Hz (**Figure 3C**, right panel), but not in frequencies above 9 Hz. Calculation of the stimulationdependent signal difference's effect size on each frequency band indicated a large effect on the delta band (*d* = 0.9893, *r* = 0.443) which was weaker in the theta band (*d* = 0.5721, *r* = 0.275) and small in the alpha (*d* = 0.2988, *r* = 0.122) as well as beta band (*d* = 0.2791, *r* = 0.115).

#### **BRAIN-MACHINE INTERFACE (BMI) CONTROL ACROSS SESSIONS AND CONDITIONS**

RmANOVA indicated no main effects for "session" [*F*(1, 9) = 0.30, *p* = 0.597] or "block" [*F*(1, 9) = 1.097, *p* = 0.322] and no interaction between the factors [*F*(1, 1) = 0.349, *p* = 0.569]. There also was no difference between block 1 of session A and session B (*p* = 0.541), nor a difference between block 1 and block 2 of session A (*p* = 0.880) or session B (*p* = 0.470) (**Figure 4**).

#### **SAFETY AND TOLERABILITY OF SIMULTANEOUS tDCS DURING EEG-BASED BRAIN-MACHINE INTERFACE (BMI) CONTROL**

Six of ten participants reported light tingling (rated at 2–3 on a VAS, mean value across all participants 1.6 ± 0.4 with 0 = none and 10 = unbearable, extreme tingling) in at least one of the sessions, and five reported light itching (mean value 1.4 ± 0.8; 0 = none, 10 = unbearable, extreme itching), mainly at the beginning of block 2. None of the participants reported any form of severe discomfort or pain. Participants were unable to distinguish sensations between session A and session B.

#### **DISCUSSION**

Our study investigated the influence of simultaneous tDCS on EEG recordings across different frequency bands and shows that online extraction of physiological signals during learned selfregulation of brain oscillations for online MI-based BMI control is feasible and safe. We found that application of tDCS is associated with a significant signal increase across slower frequency bands below 9 Hz (delta and theta) in direct proximity of the stimulation electrode, and delta band (<4 Hz) recorded at larger distance (>∼8 cm). However, signals oscillating above 9 Hz (e.g., 11 Hz) were not influenced by stimulation and could be successfully used for reliable, motor imagery-based BMI control. Our results indicate that any BMI paradigm driven by modulation of brain oscillations in the alpha (9–15 Hz) or beta range (15–30 Hz) or even higher frequencies (>30 Hz) is possible, given that the signal/noise ratio allows proper linear or non-linear classification of the neurophysiological features used for BMI control.

An important aspect in the combination of tDCS and BMI control is the stimulation montage. Due to the conduction properties of the human head, most electric currents pass through the

**mu-ERD)-based brain-machine interface (BMI) performance was defined as the percentage of time mu-ERD was detected during trials.** BMI performance was comparable between both sessions and did not exhibit differences between block 1 and block 2. There was neither a difference between block 1 of session A and session B (*p* = 0.541), nor a difference between block 1 and block 2 of session A (*p* = 0.880) or session B (*p* = 0.470).

skin and cerebro-spinal fluid (CSF), while only a fraction enters the gray or white matter (Sadleir et al., 2012). The path of the electric currents in any given case depends on many individual characteristics, such as bone thickness, shape of the skull, density of bone-passing veins or volume of the outer CSF space (which is increased, for instance, in brain atrophy).  Depending on the montage, the area with the highest magnitude of the cortical electrical field might be in larger distance from the stimulating electrode not directly underlying the current source (Edwards et al., 2013). Thus, the technique described here which allows for the application of electric currents as close as 1 cm to the EEG recording channel might be used for modulating cortical activity of brain areas functionally related to BMI control.

When using a different stimulator than the one used in this investigation, corresponding characteristics of this device should be tested first before combining it with EEG-based BMI systems to rule out other stimulation-dependent contaminations of the EEG signal.

While some previous studies suggested that tDCS can have immediate effects on motor imagery-related ERD (Matsumoto et al., 2010) and BMI control (Wei et al., 2013), we did not find such immediate effects. This might be due to the fact that participants were not BMI-naïve and exhibited already high and stable BMI control before admission to the study. tDCS, here applied as anodal tDCS, might not have improved BMI performance further, as a ceiling in BMI control may have been reached in these participants. Another reason could be the montage of the stimulation electrode placed 1 cm anterior of the C4, which might have resulted in the highest magnitude of cortical electrical fields in brain areas not related to motor imagery-based BMI control. A different placement of the stimulation electrode might have led to other results.

The immediate effects of electric currents on oscillatory activity in the human brain and their relatedness to behavior are still poorly understood (Dayan et al., 2013). As noted previously, a sensor-space based approach does not allow unambiguous differentiation between signals with a physiological origin opposed to stimulation-dependent signals deriving from the electric circuit of the stimulator. It is conceivable that the observed slow frequency signal power elevation during tDCS is in part of physiological origin and might reflect mechanisms that also underlie the previously well-described after-effects of tDCS, for instance polarityspecific modulation of cortical excitability and improvements of cognition and learning. Due to the design of the paradigm, the effect of tDCS on slow cortical potentials (SCP) could not be investigated here, but might be of interest in future studies to improve general understanding of the physiological effects of tDCS in the context of learned brain self-regulation during motor imagery-based BMI control. Implementation of algorithms with noise-canceling features, e.g., online source-reconstruction using beamformers (Soekadar et al., 2013b) might help to further investigate these mechanisms. The combination of electrical stimulation during multimodal EEG-MEG recordings in the context of brain self-regulation might help to shed light on these issues and further improve understanding of the exact neural substrates and mechanisms underlying the learning of abstract skills, like volitional modulation of brain oscillatory activity in the context of BMI control (Koralek et al., 2012).

## **CONCLUSION**

tDCS delivered at 1 mA in close proximity (1 cm) to an EEG channel used for learned self-regulation of brain oscillatory activity above 9 Hz is feasible and safe. While associated with a signal power elevation across slower frequencies, brain signals above 9 Hz were unaffected by the stimulation allowing simultaneous application of electric currents during motor imagery-based online BMI control. Such combination might substantially improve the applicability and practicality of BMI use in patient populations, for instance, in the context of neurorehabilitation, and allow systematic investigation of the relatedness between learned brain self-regulation, brain oscillatory activity and behavior.

#### **ACKNOWLEDGMENTS**

This work was supported by the Intramural Research Program (IRP) of the National Institute of Neurological Disorders and Stroke (NINDS), USA; the Center for Neuroscience and Regenerative Medicine (CNRM), Uniformed Services University of Health Sciences, USA; the Advanced Convergence Research Center at the Daegu Gyeongbuk Institute of Science and Technology, Korea; the German Federal Ministry of Education and Research (BMBF, grant number 01GQ0831, 16SV5838K to Surjo R. Soekadar and Niels Birbaumer); the Deutsche Forschungsgemeinschaft (DFG, grant number SO932-1 to Surjo R. Soekadar and Reinhart Koselleck Project support to Niels Birbaumer); the European Commission under the project WAY (grant number 288551 to Surjo R. Soekadar and Niels Birbaumer); the Volkswagenstiftung and the Baden-Württemberg Stiftung, Germany. We thank Sook-Lei Liew and Birgit Teufel for their assistance in preparing the manuscript.

# **REFERENCES**

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 12 January 2014; accepted: 03 March 2014; published online: 18 March 2014. Citation: Soekadar SR, Witkowski M, Cossio EG, Birbaumer N and Cohen LG (2014) Learned EEG-based brain self-regulation of motor-related oscillations during application of transcranial electric brain stimulation: feasibility and limitations. Front. Behav. Neurosci. 8:93. doi: 10.3389/fnbeh.2014.00093*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience. Copyright © 2014 Soekadar, Witkowski, Cossio, Birbaumer and Cohen. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Is neural hyperpolarization by cathodal stimulation always detrimental at the behavioral level?

# *Cornelia Pirulli 1†, Anna Fertonani 1† and Carlo Miniussi 1,2\**

*1 Cognitive Neuroscience Section, IRCCS Istituto Centro San Giovanni di Dio Fatebenefratelli, Brescia, Italy*

*2 Neuroscience Section, Department of Clinical and Experimental Sciences, University of Brescia, Brescia, Italy*

#### *Edited by:*

*Niels Birbaumer, University of Tuebingen, Germany*

#### *Reviewed by:*

*Andrea Brovelli, Centre National de la Recherche Scientifique, France Niels Birbaumer, University of Tuebingen, Germany*

#### *\*Correspondence:*

*Carlo Miniussi, Department of Clinical and Experimental Sciences, University of Brescia, Viale Europa 11, 25123 Brescia, Italy e-mail: carlo.miniussi@ cognitiveneuroscience.it*

*†These authors have contributed equally to this work.*

Cathodal transcranial direct current stimulation (c-tDCS) is usually considered an inhibitory stimulation. From a physiological perspective, c-tDCS induces hyperpolarization at the neural level. However, from a behavioral perspective, c-tDCS application does not always result in performance deterioration. In this work, we investigated the role of several important stimulation parameters (i.e., timing, presence of pauses, duration, and intensity) in shaping the behavioral effects of c-tDCS over the primary visual cortex. In Experiment 1, we applied c-tDCS at two different times (before or during an orientation discrimination task). We also studied the effects of pauses during the stimulation. In Experiments 2 and 3, we compared different durations (9 vs. 22 min) and intensities (0.75 vs. 1.5 mA) of stimulation. c-tDCS applied before task execution induced an improvement of performance, highlighting the importance of the activation state of the cortex. However, this result depended on the duration and intensity of stimulation. We suggest that the application of c-tDCS induces depression of cortical activity over a specific stimulated area; but to keep reactivity within given limits, the brain react in order to restore the equilibrium and this might result in increased sensitivity in visual performance. This is a further example of how the nervous system dynamically maintains a condition that permits adequate performance in different environments.

**Keywords: transcranial direct current stimulation, perceptual learning, metaplasticity, facilitation, cathodal tDCS, homeostasis, NIBS, neural noise**

# **INTRODUCTION**

Transcranial direct current stimulation (tDCS) is a technique that allows the modulation of cortical excitability. A direct current of low-level intensity (∼2 mA) is applied to electrodes that sit on the subject's scalp. This current passes through the scalp and crosses the extra cortical layers to reach the cortex, modulating the membrane polarity of neurons within a region of underlying neural tissue. The low strength of the current is not able to induce depolarization at a threshold level of "inactive" neurons (i.e., inducing an action potential). However, if there is ongoing activity (i.e., background activity determined by state or taskinduced activity), the change in membrane potential induced by tDCS can promote more effective "excitation" or "inhibition" in a polarity-specific manner (Creutzfeldt et al., 1962; Bindman et al., 1964; Stagg and Nitsche, 2011). These tDCS-induced changes in the neuronal threshold during stimulation result from changes in membrane permeability, including depolarization of the soma by anodal stimulation (a-tDCS) and hyperpolarization by cathodal stimulation (c-tDCS) (Liebetanz et al., 2002; Nitsche et al., 2003b, 2004a,b). Polarization effects outlast the tDCS period (Nitsche and Paulus, 2000, 2001), and these after-effects are due to changes in receptor activity at the synaptic level, in addition to membrane polarity shifts (Nitsche et al., 2003a, 2004a,b).

From a behavioral standpoint, strong *a priori* assumptions are often made in neuromodulation studies using tDCS where physiological effects are directly mapped on to behavioral effects. The application of a-tDCS during a task is thought to induce facilitation, while c-tDCS is assumed to induce inhibition of performance. While the concept of a-tDCS facilitating and ctDCS worsening performance seems well established, these effects are mainly valid for tDCS in the motor system (Nitsche et al., 2008, but see Wiethoff et al., 2014). Behavioral effects of tDCS have been identified in several functional areas, but the relationship between facilitation and inhibition is often quite complex (Jacobson et al., 2012; Miniussi et al., 2013). tDCS is thought to prime the behavioral system by increasing/decreasing cortical excitability and producing behavioral effects in the cognitive system. Nevertheless, the final effect depends on the stimulated area and on its involvement in a given task (i.e., the type of process and the final goal of the task) (Dockery et al., 2009; Nitsche and Paulus, 2011; Vallar and Bolognini, 2011; Berryhill and Jones, 2012; Jacobson et al., 2012; Tseng et al., 2012; Weiss and Lavidor, 2012; Filmer et al., 2013; Hsu et al., 2014; Nozari et al., 2014).

Several examples of contradictory final behavioral outcomes for c-tDCS can be found at different processing levels (Jacobson et al., 2012). Monti et al. (2008) found facilitation in a naming task using c-tDCS over Broca's area. Antal et al. (2004b) found that c-tDCS applied to the visual middle temporal area improved performance in a visuomotor coordination task. In a different study from the same group (Antal et al., 2001) using a Gabor patch stimulus, only c-tDCS showed a significant change in static and dynamic contrast sensitivities. Vicario et al. (2013) showed that c-tDCS applied to the posterior parietal cortex enhanced temporal accuracy in a time reproduction task compared to sham stimulation (see also Dockery et al., 2009; Moos et al., 2012; Weiss and Lavidor, 2012; Filmer et al., 2013). Moreover, several studies reported no effect of c-tDCS on task performance (e.g., Kincses et al., 2004; Iyer et al., 2005; Sparing et al., 2008; Cerruti and Schlaug, 2009; Fertonani et al., 2010; Kraft et al., 2010). Contrasting effects of c-DCS have also been found at the neurophysiological level (see Matsunaga, 2004; Pellicciari et al., 2013). A magnetoencephalography study has reported that the two tDCS polarities induce the same cortical EEG power density (Venkatakrishnan et al., 2011). Antal et al. (2004a) showed that c-tDCS over the primary visual cortex (V1) decreased the amplitude of an early visual evoked potential (N70), whereas Accornero et al. (2007) reported the opposite result with an increased amplitude of the early P100 potential.

This complex pattern of results may be explained by the fact that the ability or the efficacy of tDCS to induce modifications of membrane polarity—and consequently behavioral performance—depends on several methodological and technical parameters: current density, duration, timing of application, and pauses between stimulations (for a review, see Brunoni et al., 2012). Clearly, all these methodological aspects will determine the "reaction of the brain" to the DC stimulation in relation to the subject's state and task/protocol. As an example, the importance of duration and of the presence of pauses during the period of stimulation has been recently demonstrated by Fricke et al. (2011). These authors measured the motor evoked potential (MEP) amplitude after repeated tDCS. The authors found that 5 or 10 min of c-tDCS decreased excitability and suppressed MEP amplitude for 5 and 30 min, respectively, indicating the importance of tDCS duration. However, if a pause of 3 min was inserted between two stimulations of 5 min, there was an inversion of the effect, resulting in MEP amplitude enhancement (see also Monte-Silva et al., 2010). The stimulation intensity is also an important factor. Batsikadze et al. (2013) showed that 1 mA c-tDCS decreased MEP amplitude, but the application of 2 mA resulted in increased cortical excitability (see also Teo et al., 2011; Moos et al., 2012; Hoy et al., 2013). In addition, behavioral performance induced by tDCS depends on the timing of application in relation to task execution (Stagg et al., 2011; Pirulli et al., 2013; Fertonani et al., 2014). Several studies have shown that the same type of stimulation may have different behavioral effects (facilitation vs. inhibition vs. null-effect) depending on whether it is applied before or during the task execution.

Given the heterogeneity of the effects induced by c-tDCS, the aim of this work was to explore the outcome of applying c-tDCS on V1. We chose an orientation discrimination task (ODT) because it is a well established task to study visual perceptual learning (Vogels and Orban, 1985; Shiu and Pashler, 1992), and it has been showed to involves primarily V1 (Schoups et al., 2001; Li et al., 2004). Nevertheless it should be considered that we cannot be totally sure that stimulation delivered by tDCS is focalized only under the stimulation electrode (Miranda et al., 2006; Wagner et al., 2007). We applied c-tDCS before or during the execution of ODT, with or without pauses during stimulation, and at different intensities and durations. Based on the previous reports, our expectation was that the final outcome of the tDCS in terms of response facilitation or inhibition would depend on the interaction between the brain state and when and how c-tDCS was applied. Therefore, c-tDCS would not necessarily induce a univocal behavioral inhibition. Given that the brain is constructed to keep certain important parameters within given limits, the brain would react proportionally when it is shifted from these limits in order to restore the equilibrium, as suggested by the concept of homeostasis (see Bernard, 1878 and Cannon, 1929 in Cooper, 2008). Therefore, an initial down-regulation induced by an inhibitory stimulation, given for a longer time, at higher intensity can be reverted, rendering the involved neurons more easily responsive. Nevertheless to explain the effects of tDCS during a behavioral task it has been proposed that a neural noise framework should be consider (see non-linear systems and stochastic resonance; Miniussi et al., 2013), suggesting that the outcome of applying tDCS depends on the noise present in the system and the level of tDCS and task -induced activity, rather than solely on the stimulation polarity.

# **MATERIALS AND METHODS**

#### **ORIENTATION DISCRIMINATION TASK**

We chose an ODT that is a widely studied VPL task and involves V1 neurons (Vogels and Orban, 1985; Shiu and Pashler, 1992). This task has been previously described in detail by our group (Fertonani et al., 2011; see Pirulli et al., 2013). Given the ODT characteristics, is likely that a local (i.e., V1) circuit of neuronal populations is dedicated to execute the task, nevertheless we cannot exclude that tDCS effects are due to a more complex neuronal network, involving others parietal areas. Briefly, throughout the experiment, participants were comfortably seated in an armchair in a quiet, dimly illuminated room. The subjects had to decide as quickly and accurately as possible whether the presented stimulus (a target line) was tilted clockwise or counter clockwise relative to the previously presented stimulus (reference line) (see trial structure in **Figure 1**). After each response an auditory feedback informed the subjects about the correctness of their responses (an high tone indicated the right response while a low tone the wrong response).

Each block of the ODT consisted of 64 trials and lasted approximately 4 min. The ODT consisted of 5 experimental blocks plus a training block. The training block contained 8 trials and an increased rotation angle between the two stimuli (10◦ clockwise or counter clockwise). In Experiments 1 and 2, the angular difference between the reference and the target was ±1.10, 1.21, 1.33, and 1.46◦. All of the experimental parameters were balanced and randomized between blocks. In Experiment 3 (control experiment), the task was made easier by replacing the smallest degree of rotation (1.10◦) with 1.60◦ (1.21, 1.33, 1.46, and 1.60◦). All of the other task characteristics were unchanged, except for the presence of a baseline block before the stimulation.

#### **TRANSCRANIAL DIRECT CURRENT STIMULATION**

tDCS was delivered by a battery-driven current stimulator (Eldith-Plus, NeuroConn GmbH, Ilmenau, Germany) through

a pair of saline-soaked surface sponge electrodes. The "active" electrode (16 cm2) was placed over the occipital cortex in the area corresponding to V1, which was defined as 10% of the nasion-inion distance above the inion (mean position = 3.5 ± 0.2 cm above the inion). The reference electrode (60 cm2) was fixed extra-cephalically on the right arm. The electrodes were kept in place with elastic bands, and an electro-conductive gel was applied under the electrodes to help reduce impedance to the electrical current. When tDCS was applied, the polarity of the active electrode over V1 was always cathodal. For active tDCS, the current was ramped up over 8 s (fade-in phase), held constant for the experimental time, and then ramped down over 8 s (fade-out phase). In the sham c-tDCS, the current was ramped up (8 s) and down (8 s) and stayed at level for 15 s.

#### *Experiment 1—"Timing and pauses"*

In this experiment, we investigated the effect of the timing of stimulation during task execution, either online or offline (before task execution), and the presence of pauses during the stimulation (intervals of 2 min between blocks). In all conditions, we applied c-tDCS for 22 min at 1.5 mA (current density of the active electrode 0.094 mA/cm2 of the reference 0.025 mA/cm2) as shown in **Figure 2A**. In the continuous stimulation, current was administered without pauses, while in the paused stimulation condition,

the stimulation was turned on at the beginning of each experimental block and maintained until the end of the block. c-tDCS was applied for approximately 4 min during each of the 5 experimental blocks, with 2 min of pauses between blocks (i.e., 4 min of stimulation—2 min of pause—4 min of stimulation, and so on). In the online condition, the stimulation was applied during the task execution. In the offline condition, stimulation was applied before task execution while the subjects were listening to an audio book played on an audio device, maintaining the same time intervals used in the online condition. The duration of the entire experimental session was approximately 30 min for the online conditions and approximately 60 min for the offline conditions. The procedure is described in **Figure 2A**.

#### *Experiment 2—"Duration and intensity"*

In this experiment, we investigated the effect of the duration and intensity of stimulation for the offline condition, as shown in **Figure 2B**. Subjects were stimulated for 9 min at an intensity of 1.5 mA or for 22 min at an intensity of 0.75 mA (current density of the active electrode 0.047 mA/cm2 of the reference 0.013 mA/cm2). Data were compared with those for 22 min of stimulation at an intensity of 1.5 mA in the offline condition collected in Experiment 1 (see **Figure 2B**).

#### *Experiment 3—Control experiment*

In Experiment 3, we corroborated the effects of the stimulation intensity for the offline protocol with a modified experimental design (see the description of the ODT task, above). We facilitated the ODT and added a baseline block before the beginning of the offline stimulation. The stimulation was applied continuously for 22 min with an intensity of 1.5 or 0.75 mA (see **Figure 2C**).

### *Sensation questionnaire*

In all of the experiments, at the end of the experimental session, we asked all subjects to complete a questionnaire (Fertonani et al., 2010) about the tDCS-induced sensations that they experienced during the different conditions so that we could evaluate if different stimulation protocols (e.g., active vs. sham) induced different sensations.

### **SUBJECTS**

A total of 139 healthy subjects participated in the three experiments. All of the participants were right-handed except for 6 subjects tested in Experiment 3, who were equally distributed in the experimental groups. All participants had normal or corrected-to-normal vision. Subjects with a history of seizures, implanted metal objects, heart problems or any neurological disease were not recruited. Moreover, subjects who had a task performance below chance (no learning) were excluded from the study. Based on these criteria, 17 participants were excluded. The remaining 122 subjects (61 males, mean age ± standard deviation 22.0 ± 2.9 years; range 19–33 years) participated in the experiments.

### *Experiment 1—"Timing and pauses"*

Seventy-two subjects were assigned to one of the five groups stimulated for 22 min at 1.5 mA: online paused (14 subjects, 7 males; 21.7 ± 2.6 years), offline paused (14 subjects, 7 males; 21.6 ± 2.6 years), online continuous (10 subjects, 5 males; 21.7 ± 0.8 years), offline continuous (10 subjects, 5 males; 23.0 ± 3.6 years) and placebo stimulation (sham, 24 subjects, 12 males; 21.7 ± 3.6 years). In the sham group, 14 subjects were stimulated online and 10 subjects were stimulated offline. The data for the online conditions were collected in a previous experiment (Fertonani et al., 2011; for details see Pirulli et al., 2013).

# *Experiment 2—"Duration and intensity"*

We tested two additional groups (20 Subjects): 10 subjects (5 males; 22.1 ± 2.0 years) stimulated for 9 min at 1.5 mA (offline-9 min) and 10 subjects (5 males; 21.4 ± 2.0 years) stimulated for 22 min at 0.75 mA (offline-0.75 mA). We compared these two groups with the offline and sham groups of Experiment 1.

# *Experiment 3—Control experiment*

We tested three new groups (30 Subjects) stimulated for 22 min at 0.75 mA (10 subjects, 5 males, 23.0 ± 4.4 years), 1.5 mA (10 subjects, 5 males, mean age 20.8 ± 1.8 years) and sham (10 subjects, 5 males; 23.6 ± 4.1 years).

The study was approved by the Ethics Committee of the IRCCS Centro San Giovanni di Dio Fatebenefratelli, Brescia, Italy. Safety procedures were used in accordance with non-invasive brain stimulation indications (Iyer et al., 2005; Poreisz et al., 2007; Rossi et al., 2009), and written informed consent was obtained from all participants prior to the beginning of the experiments.

# **DATA ANALYSIS**

The average orientation sensitivity was calculated as a d prime value (d ) from measurements of the hit rate and false-alarm rate, for each subject and each block separately for each stimulation condition. We have chosen the d as a measure of accuracy because it is roughly invariant when response bias is manipulate, whereas simple indexes such as proportion correct don't have this property. As a first index of learning rate, we analyzed the relationship between d values and block numbers using linear regression analysis. This analysis allowed us to associate a slope value with each subject. A second index called the "learning index" was calculated, for Experiment 3, by subtracting the mean baseline d value from the mean d value of block 5 for each subject.

The Kolmogorov-Smirnov test confirmed the normality of the distribution of all data (d values, slope, learning index), and subsequently data were analyzed using a repeated-measures analysis of variance (ANOVA). The data sphericity was tested using the Mauchly test, where appropriate. When the test results were statistically significant, the data were corrected using the Huynh-Feldt correction. The effect size is reported using the partial Eta squared value. A *p*-value *<* 0.05 was considered significant for all statistical analyses. For multiple comparisons, we used Fisher's Least Significant Difference (LSD) method to test our specific "a priori" hypotheses (i.e., to compare different timings of application, intensities and durations). For all other comparisons, the *p*-values were corrected using a Tukey correction.

Data from the sensations induced by c-tDCS were analyzed using the Kruskal–Wallis one-way analysis of variance and, subsequently, with multiple comparisons.

# **RESULTS**

# **ORIENTATION SENSITIVITY—d-**

### *Experiment 1—"Timing and pauses"*

We performed a repeated-measure ANOVA with *block* (from 1 to 5) as a within-subjects factor and *stimulation* (online paused, online continuous, offline paused, offline continuous, and sham) as a between-subjects factor. We observed a significant main effect for *block* [*F*(4*,* 268) = 24*.*105; *p <* 0*.*001; η2 *P* = 0*.*265] and *stimulation* [*F*(4*,* 67) = 3*.*272; *p* = 0*.*016; η2 *P* = 0*.*163]. The interaction between *block* and *stimulation* was not statistically significant [*F*(16*,* 268) = 1*.*005; *p* = 0*.*452].

For *block*, multiple *post-hoc* comparisons revealed a statistically significant difference between block 1 and blocks 2, 3, 4, and 5 (all *p <* 0*.*01), between block 2 and blocks 4, and 5 (all *p <* 0*.*001), and between block 3 and block 5 (*p* = 0*.*043).

For *stimulation*, multiple *post-hoc* comparisons revealed that the offline paused (mean d ± standard error of the mean s.e.m. = 0.615 ± 0.122) and offline continuous (0.623 ± 0.114) conditions were significantly different from the online paused (0.304 ± 0.121), online continuous (0.346 ± 0.118), and sham (0.368 ± 0.083) (all *p <* 0*.*05) conditions (see **Figure 3**).

These results support the initial hypothesis that c-tDCS, applied before the ODT, modulates behavior, while c-tDCS applied during the task does not modify the final outcome. We found an improvement in the subject's accuracy when c-tDCS was applied offline.

These data highlight the absence of a difference between the conditions with or without pauses. Confirmation was obtained with an ANOVA with *block* (from 1 to 5) as a within-subjects factor and *timing* (online vs. offline) and presence of *pauses* (paused vs. continuous) as between-subjects factors. We observed a significant main effect of *block* [*F*(4*,* 176) = 18*.*438; *p <* 0*.*001; η2 *P* = 0*.*295] and *timing* [*F*(1*,* 44) = 11*.*026; *p* = 0*.*002; η2 *P* = 0*.*200]. The factor *pauses* was not statistically significant [*F*(1*,* 44) = 0*.*077; *p* = 0*.*782]. No interaction was statistically significant. For *block*, multiple *post-hoc* comparisons revealed a statistically significant difference between block 1 and blocks 3, 4, and 5 (all *p <* 0*.*001) and between block 2 and block 3 (*p* = 0*.*042), 4 (*p <* 0*.*001), 5 (*p <* 0*.*001) (see **Figure 3**).

Having verified that the presence of pauses during stimulation does not influence the effect of stimulation, we collapsed the two online conditions (continuous and paused) and the two offline conditions (continuous and paused) into one online and one offline condition (hereafter, all conditions with the initial parameters, i.e., 22 min duration and 1.5 mA intensity, will be called "online" and "offline").

#### *Experiment 2—"Duration and intensity"*

We performed a repeated-measure ANOVA with *block* (from 1 to 5) as a within-subjects factor and *stimulation* (online, offline, offline-9 min, offline-0.75 mA, and sham) as a between-subjects factor. We observed a significant main effect for *block* [*F*(4*,* 348) = 20*.*286; *p <* 0*.*001; η2 *P* = 0*.*189] and *stimulation* [*F*(4*,* 87) = 4*.*727; *p* = 0*.*002; η2 *P* = 0*.*163]. The interaction between *block* and *stimulation* was not statistically significant [*F*(16*,* 348) = 1*.*512; *p* = 0*.*092].

For *block*, multiple *post-hoc* comparisons revealed a statistically significant difference between block 1 and blocks 2, 3, 4, and 5 (all *p <* 0*.*001), between block 2 and blocks 4, and 5 (all *p <* 0*.*001), and between block 3 and block 5 (*p* = 0*.*003).

**FIGURE 4 | Results of Experiment 2.** Data are presented as the mean d values. The lines represent the fit of each condition. The blue line represents the offline 1.5 mA for the 22 min condition, the green line represents the online 1.5 mA for the 22 min condition, the turquoise line represents offline 1.5 mA for the 9 min condition, and the double blue line represents the offline-0.75 mA for the 22 min condition. The sham condition is shown in black.

For *stimulation*, multiple *post-hoc* comparisons revealed that offline (0.618 ± 0.085) was different from sham (0.368 ± 0.083), online (0.321 ± 0.086) and offline-0.75 mA (0.264 ± 0.109). Moreover, offline-9 min (0.577 ± 0.123) was different from online and offline-0.75 mA and showed a marginally significant difference from sham (*p* = 0*.*069, all other *p <* 0*.*05) (see **Figure 4**).

#### *Experiment 3—Control experiment*

We performed a repeated-measure ANOVA with *block* (from baseline to 5) as a within-subjects factor and *stimulation* (offline, offline-0.75 mA, and sham) as a between-subjects factor. We observed a significant main effect for *block* [*F*(5*,* 135) = 7*.*843; *p <* 0*.*001; η2 *P* = 0*.*225] and an interaction between *block* and *stimulation* [*F*(10*,* 135) = 1*.*980; *p* = 0*.*128]. The factor *stimulation* was not statistically significant [*F*(2*,* 27) = 1*.*434; *p* = 0*.*256].

For *block*, multiple *post-hoc* comparisons revealed a statistically significant difference between block baseline and blocks 1, 2, 3, 4, and 5 (all *p <* 0*.*01). The interaction between *block* and *stimulation* revealed that stimulation influences the block trend. In the offline condition, block baseline was different from blocks 1, 2, 3, 4, and 5; block 1 was different from blocks 4 and 5; and block 2 was different from block 5. In the offline-0.75 mA condition, block baseline was different from block 4. In the sham condition, block baseline was different from blocks 1 and 5 and block 1 was different from block 2.

A One-Way ANOVA on block baseline showed that in this block, d was not different between the stimulation conditions [*F*(2*,* 27) = 0*.*064; *p* = 0*.*938].

In Experiment 3, the presence of a block of baseline allowed us to show a different rate of learning between the stimulation conditions. For this purpose, we executed two different analyses. A One-Way ANOVA on slope [*F*(2*,* 27) = 4*.*630; *p* = 0*.*019; η2 *P* = 0*.*255] demonstrated that offline was different from sham (*p* = 0*.*008) and 0.75 mA (*p* = 0*.*027). A One-Way ANOVA on the learning index [*F*(2*,* 27) = 3*.*658; *p* = 0*.*039; η2 *P* = 0*.*213] showed that offline was different from sham (*p* = 0*.*034) and 0.75 mA (*p* = 0*.*022) (see **Figure 5**).

#### *Sensations induced by different conditions*

In the tDCS sensation questionnaire (Fertonani et al., 2010), each participant reported having tolerated the stimulation without discomfort. The results of the questionnaire are reported in **Table 1** (Experiments 1 and 2) and **Table 2** (Experiment 3). In Experiments 1 and 2, the analysis did not reveal any differences

between the stimulations for pain, burning, heat, iron taste, and fatigue sensations. The analysis demonstrated a statistically significant difference between stimulations with respect to itchiness [*H*(6*, N* = 92) = 17*.*382, *p* = 0*.*008] and pinching [*H*(6*, N* = 92) = 18*.*974, *p* = 0*.*004]. Subsequently, multiple comparisons were performed for these two sensations. For itchiness, offline continuous was significantly different (*p* = 0*.*03) from sham, whereas for pinching, online paused and offline paused were significantly different from sham (respectively *p* = 0*.*03 and *p* = 0*.*01). The subjective influence on performance was equal for all the stimulations. In Experiment 3, the analysis did not reveal any difference between the stimulation conditions in the perceived sensations.

#### **DISCUSSION**

In this work, we demonstrated that c-tDCS, which is often considered to be inhibitory at the behavioral level, can actually induce facilitatory effects, enhancing subjects' performance. In the Experiment 1, we showed that the effects of c-tDCS are dependent on the timing of stimulation. Only c-tDCS before the task induced an improvement of performance. Moreover, in this experiment, we applied 22 min of stimulation interspersed by pauses of 2 min and compared this protocol to a continuous protocol. We found that short pauses do not play any role in shaping the final outcome. In Experiment 2, we applied offline stimulation and found that 9 min of c-tDCS facilitated the performance only slightly less. Instead, current density induced a facilitatory effect in our protocol. Finally, Experiment 3 confirmed the importance of current density and showed that the facilitatory effect is not due to skill differences between subjects (i.e., differences in accuracy) because all the groups had the same performance at baseline.

The presence of short pauses (2 min) during c-tDCS, both in the online and in the offline conditions, did not influence the results. In previous works, it has been shown that interstimulation intervals determine the effects of c-tDCS on the

**Table 1 | Transcranial direct current stimulation (tDCS)-induced sensations—Experiments 1 and 2: mean intensity of the sensations reported by subjects after tDCS and the percentage of subjects who reported each sensation.**

*Sensation intensity was evaluated on a 5-point scale: 0 none, 1 mild, 2 moderate, 3 considerable, and 4 strong. The column "Effect on performance" indicates the subjective feelings of the participants relative to the effect of the tDCS-induced sensation on performance (valid only for the online condition).*

**Table 2 | Transcranial direct current stimulation (tDCS)-induced sensations—Experiment 3 (See Table 1 description).**

motor cortex (Monte-Silva et al., 2010; Fricke et al., 2011). These previous studies found that varying the length of the pause between two stimulations modified the effects on cortex excitability. Fricke et al. (2011) demonstrated that short inter-stimulation intervals (3 min) induced an inversion of the inhibitory effect of c-tDCS, resulting in MEP amplitude enhancement. Monte-Silva et al. (2010) showed that the inhibitory effects of c-tDCS were more efficacious if a second period of stimulation was applied during the after-effects of the first stimulation with an interval of 3 or 20 min. However, with long inter-stimulation intervals, the c-tDCS-induced inhibitory after-effects were diminished. In Experiment 2 of our study, we confirmed that c-tDCS applied before the task induces a significant improvement in performance, regardless of the presence of pauses during stimulation. A possible explanation for these partially contrasting data could be the differences in stimulation parameters (i.e., the intensity and number of pauses\blocks of stimulation). A higher intensity of stimulation (1.5 mA) may induce a stable excitability shifts in relation to the execution of the task, and therefore, the additional presence of pauses might not further affect the cortical state (see Batsikadze et al., 2013). Similarly the presence of successive multiple pauses\blocks of stimulation might decrease further changes of the cortical state through adaptation.

During the online application of c-tDCS, while the subjects were performing the ODT, we expected a worsening of performance. However, it was difficult to observe a decline in subject accuracy in our learning task because subjects could not have a performance level lower than chance ("floor effect"). This result suggests that the behavioral level of task performance at baseline is a key factor in determining a null effect of c-tDCS. Additionally, compensatory networks can be activated during stimulation (Sack et al., 2005), and therefore, a functional compensation might intervene, maintaining behavior after neuronal challenge (O'Shea et al., 2007). Nevertheless, the absence of inhibition by online c-tDCS is in line with previous data (Jacobson et al., 2012).

Our most important result involves the fact that c-tDCS applied before the task at 1.5 mA induced a facilitatory effect on subjects' accuracy during the ODT (see also Dockery et al., 2009; Moos et al., 2012; Weiss and Lavidor, 2012; Filmer et al., 2013). Most studies that have applied 1 mA c-tDCS in the motor system found decreased cortical excitability; this has also been shown with a decrease in MEP amplitude (see Nitsche and Paulus, 2011; Medeiros et al., 2012). However, these physiological effects are not linear, but they seem to depend on several parameters. In a recently published work, Batsikadze et al. (2013) showed that 20 min of c-tDCS at 2 mA applied to the motor cortex significantly increased MEP amplitudes, while 1 mA of the same stimulation decreased cortico-spinal excitability. This result highlights that an increase in intensity (2 mA) and duration (20 min) of stimulation induces an opposite outcome compared to standard parameters. Batsikadze et al. (2013) suggested that this result might be due to the direction of plasticity from the amount of neuronal calcium influx caused by the stimulation protocol: whereas low postsynaptic calcium enhancement induced by lowintensity c-tDCS causes long-term depression, higher intensity c-tDCS induces a large calcium increase, resulting in long-term potentiation (LTP) (Cho et al., 2001; Lisman, 2001; Batsikadze et al., 2013). Our data are in line with this result; nevertheless, at a systems level, different intensities of stimulation might induce different adaptive responses by the brain, as discussed below.

Importantly, our stimulation was applied before the task execution. The time at which the stimulation is applied has been investigated at the behavioral level in the motor (Stagg et al., 2011) and visual domains (Pirulli et al., 2013). Stagg et al. (2011) showed that a-tDCS on the motor cortex has opposite effects if applied during or before a sequence—a motor learning task. However, the effects of c-tDCS do not seem to be timing dependent. c-tDCS application during or before an explicit motor learning task induces a slowing in reaction time (Stagg et al., 2011). Nevertheless, in a previous work (Pirulli et al., 2013), we demonstrated that the same type of current can induce different effects on visual performance depending on the state of activation of the cortex.

Here, we demonstrated that the application of c-tDCS before the VPL induces an improvement in performance that is not present if c-tDCS is applied during the execution of the task. Therefore, c-tDCS causes different effects depending on the state level of the neurons at the moment of stimulation (Dockery et al., 2009; Tseng et al., 2012; Weiss and Lavidor, 2012; Filmer et al., 2013; Hsu et al., 2014; Nozari et al., 2014). While an online effect would rely mainly on the depolarization or hyperpolarization induced by tDCS interacting with task execution (see Miniussi et al., 2013), an offline protocol would rely more on a change in the state of the stimulated area induced by the stimulation i.e., a shift in the input strength needed for the final response (see Figure 5 in Miniussi et al., 2013). Thus, all these aspects, which can be defined as "relatively simple" technical parameters, influence brain activity in response to exogenous stimulation.

Artificially altering neuronal function can trigger homeostatic changes at the synaptic level (Turrigiano and Nelson, 2004). For example, if a synapse is constantly over-inhibited, there can be a compensatory increase in receptor activity at the postsynaptic membrane, termed up-regulation. Homeostatic plasticity is a fundamental physiological mechanism that maintains neural functions within predefined optimal ranges (Bienenstock et al., 1982; Turrigiano and Nelson, 2004; Abraham, 2008). The basis of homeostatic plasticity is that the threshold for LTP induction is not stable but varies depending on previous neuronal activity induced by non-invasive stimulation (Ziemann and Siebner, 2008; Siebner, 2010). Therefore, the application of tDCS before and during the task of interest may result in different functional states. The application of c-tDCS before the execution of the task would lower the level of postsynaptic neural activity, causing a decrease in the threshold for the induction of successive facilitatory mechanisms. The induced neural modification could then facilitate LTP-like mechanisms and consequently induce an improvement in behavioral performance (Bienenstock et al., 1982; Abraham, 2008). Homeostatic mechanisms can stabilize cortical excitability within a range (Siebner et al., 2004). Thus, an initial down-regulation induced by c-tDCS can be reverted. Applying this theory to our data, pre-conditioning the V1 cortex with 22 min of inhibitory stimulation could render neurons that are involved in task execution more easily excitable. In classical studies, a release phenomenon (rebound) has been found at the end of the cathodal DC (Creutzfeldt et al., 1962). However, the timing of this inversion might depend, at least in part, on stimulation parameters like intensity and duration. Indeed, stimulating offline with a different duration or intensity changes the behavioral effects. Our data highlighted the fact that the final behavioral response, obtained by applying tDCS, depends on the history of the stimulated area (i.e., metaplasticity). This has been previously demonstrated in animal studies in which the direction and magnitude of synaptic plasticity depends on the previous history of postsynaptic activity (Huang et al., 1992; Wang and Wagner, 1999).

From a physiological perspective, c-tDCS over the scalp can induce a hyperpolarization at the soma of perpendicularly oriented neurons (Jefferys, 1981; Bikson et al., 2004). However, we are measuring behavior and therefore should consider a network level of reasoning rather than just a cellular one and present data testify the complexity of a neural network response. Indeed, changes in hyperpolarization alter the sensitivity of the entire system and therefore its response threshold, but these changes are ultimately expressed on subject performance. The final response to the task depends on the strength of the signal and on the signal-to-noise ratio, where the signal is the neural activity operational to the task and the noise is random neural activity. Clearly, in this case, the signal-to-noise ratio relies on a system that has changed its state after c-tDCS, a system that has adapted to exogenous stimulation. Adaptation occurs when receptors\neurons change their sensitivity to a stimulus. In the visual system, adaptation to reduced light intensity increases the visual system's ability to detect a stimulus. In the same way, the reduced cortical activity induced by c-tDCS causes an improved threshold for orientation sensitivity (d ) by reducing background noise. Therefore, an inhibitory stimulation may increase the signal-to-noise ratio in the system and facilitate perceptual learning (Miniussi et al., 2013).

In conclusion, the present data show that an inhibitory stimulation does not always induce a deterioration in performance. The effects of c-tDCS should be considered in relation to the timing and the application parameters that will alter the state of the cortical network carrying out a task. We suggest that when applying c-tDCS before a task, it is necessary to consider the involvement of cognitive and non-cognitive adaptation mechanisms. The application of a tDCS protocol that induces a depression in cortical activity over a specific stimulated area might result in increased sensitivity in visual performance. This is a further example of how the nervous system maintains a dynamic state to maintain performance in different environments.

# **REFERENCES**

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 02 May 2014; accepted: 05 June 2014; published online: 27 June 2014. Citation: Pirulli C, Fertonani A and Miniussi C (2014) Is neural hyperpolarization by cathodal stimulation always detrimental at the behavioral level? Front. Behav. Neurosci. 8:226. doi: 10.3389/fnbeh.2014.00226*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience. Copyright © 2014 Pirulli, Fertonani and Miniussi. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Real-time fMRI brain-computer interface: development of a "motivational feedback" subsystem for the regulation of visual cue reactivity

#### **Moses O. Sokunbi 1,2\*, David E. J. Linden1,2 , Isabelle Habes 1,2 , Stephen Johnston3 and Niklas Ihssen1,2**

1 MRC Centre for Neuropsychiatric Genetics and Genomics, Institute of Psychological Medicine and Clinical Neurosciences, Cardiff School of Medicine, Cardiff University, Cardiff, UK

2 Cardiff University Brain Research Imaging Centre (CUBRIC), Cardiff University, Cardiff, UK

3 Department of Psychology, Swansea University, Swansea, UK

#### **Edited by:**

Sergio Ruiz, Pontificia Universidad Catolica de Chile, Chile

#### **Reviewed by:**

Corinde Wiers, Berlin School of Mind and Brain, Germany Ralf Veit, Institute of Medical Psychology, Germany

#### **\*Correspondence:**

Moses O. Sokunbi, MRC Centre for Neuropsychiatric Genetics and Genomics, Institute of Psychological Medicine and Clinical Neurosciences, Cardiff School of Medicine, Cardiff University, Hadyn Ellis Building, Maindy Road, Cathays, Cardiff, CF24 4HQ, Wales, UK

e-mail<EMAIL_ADDRESS>
Here we present a novel neurofeedback subsystem for the presentation of motivationally relevant visual feedback during the self-regulation of functional brain activation. Our "motivational neurofeedback" approach uses functional magnetic resonance imaging (fMRI) signals elicited by visual cues (pictures) and related to motivational processes such as craving or hunger. The visual feedback subsystem provides simultaneous feedback through these images as their size corresponds to the magnitude of fMRI signal change from a target brain area. During self-regulation of cue-evoked brain responses, decreases and increases in picture size thus provide real motivational consequences in terms of cue approach vs. cue avoidance, which increases face validity of the approach in applied settings. Further, the outlined approach comprises of neurofeedback (regulation) and "mirror" runs that allow to control for non-specific and task-unrelated effects, such as habituation or neural adaptation. The approach was implemented in the Python programming language. Pilot data from 10 volunteers showed that participants were able to successfully down-regulate individually defined target areas, demonstrating feasibility of the approach. The newly developed visual feedback subsystem can be integrated into protocols for imaging-based brain-computer interfaces (BCI) and may facilitate neurofeedback research and applications into healthy and dysfunctional motivational processes, such as food craving or addiction.

**Keywords: brain-computer interface (BCI), hunger, visual cue reactivity, food craving, functional magnetic resonance imaging (fMRI), neurofeedback, self-regulation**

# **INTRODUCTION**

Recent advancements in real-time functional magnetic resonance imaging (fMRI) have been based on the increased availability of high-field (in excess of 1.5 tesla) MRI scanners, fast data acquisition sequences, improved real-time pre-processing, improved statistical analysis techniques and methods of data visualization and feedback. These developments have made the implementation of fMRI brain-computer interfaces (fMRI-BCI) and neurofeedback possible (Weiskopf, 2012). In fMRI-BCI, the self-regulation of brain activity in a target region can be achieved by providing real-time feedback of brain activation levels in that target region. While paradigms more commonly utilize feedback signals based on overall activity levels in a given region, more complex metrics of brain activation are being used based on multivariate statistical techniques (LaConte et al., 2007; Sitaram et al., 2008) or correlation of activity in several brain areas (Zilverstand et al., 2014).

#### **NEUROFEEDBACK AND VISUAL CUE REACTIVITY**

In recent years an emerging but technically challenging field in neurofeedback research has been the pairing of feedback signals (and regulation instructions) with simultaneous visual stimulus presentation. Such approaches are especially interesting for research into neural and behavioral responses that are passively triggered by salient emotional or motivational cues, such as fear responses evoked by emotional pictures or craving elicited by drug-related pictures. The degree of drug cue-elicited neural activation within circuits of the orbitofrontal cortex (Ernst et al., 2014), the nucleus accumbens and medial prefrontal cortex (PFC; Wiers et al., 2014) has been shown to determine the degree of cue approach vs. avoidance behavior in alcohol dependent patients. Moreover, alcohol cues automatically evoke cognitive processes that are biased towards approaching the cue and correlate with drinking behavior in adolescents (Peeters et al., 2012). Neurofeedback training to control these emotional/motivational responses should thus aim at the down-regulation of blood oxygen level dependent (BOLD) responses, especially when the targeted neurocognitive process is considered to be dysfunctional. To date, however, most fMRI neurofeedback studies have trained participants to up-regulate activation of a target area. Further, a direct comparison between up- and down-regulation training of the same brain region (anterior insula) showed that down-regulation was more difficult to learn (Veit et al., 2012). In light of its promising clinical applications, this underscores the need for methodological development and research into the area of neurofeedback related to visual cue reactivity. Here we present a novel neurofeedback subsystem for the presentation of motivationally relevant feedback during exposure to appetitive pictures.

### **THE CHOICE OF FEEDBACK TYPE**

When setting up visual cue reactivity neurofeedback paradigms, researchers are confronted with several challenges. One relates to the choice of feedback type: With the simultaneous visual presentation of feedback information and task stimulus, there is a potential risk that monitoring the feedback stimulus distracts attention from the emotional/motivational cue which will confound any effortful down-regulation of the targeted neural response. Since the pioneering neurofeedback studies at the beginning of the last decade, various techniques have been employed to present visual feedback for the self-regulation of BOLD activation over time. Weiskopf et al. (2003, 2004) and deCharms et al. (2004) have used scrolling time series graphs and curves of BOLD activation in the region of interest (ROI) to provide immediate information to the participant. Sitaram et al. (2005) introduced the "thermometer" as a type of visual feedback for the display of brain activity. Positive BOLD activity, compared to a baseline period is shown as an increasing number of red "hot" bars, whereas negative BOLD activity is similarly shown in blue. The authors also introduced a virtual reality feedback system (Sitaram et al., 2005) where volunteers controlled a 3D animated character (a fish in water). Other visual feedback approaches include the presentation of functional maps of the brain (Yoo and Jolesz, 2002) or video-based feedback that has been used to train stroke patients to self-regulate ventromedial premotor cortex (Sitaram, 2007a).

Most studies using simultaneous visual stimulation during feedback-guided regulation have adopted the thermometer approach as described above but temporally separated the feedback presentation from the regulation/visual stimulation period (Posse et al., 2003; Li et al., 2013). In contrast to such delayed feedback, continuous/on-line feedback during regulation has the advantage that participants can adaptively test different mental strategies to optimize regulation. Only few studies have paired the presentation of pictorial cues with BOLD signal feedback in a simultaneous manner. Brühl et al. (2014) trained participants to down-regulate activity of the right amygdala while being exposed to negative emotional faces. During the 20-sec down-regulation blocks, a feedback stimulus reflecting activation levels of the amygdala was presented to the participants, showing two colored rectangles at the picture edges which color-coded activity levels in a similar way as the thermometer approach described above. Rather than using a peripheral feedback stimulus, Veit et al. (2012) embedded the thermometer inside the simultaneously presented emotional cues that consisted of blocks of aversive pictures. Also, Mathiak et al. (2010) provided a positive feedback through facial expression (smiling) when activity in the anterior cingulate cortex (ACC) increased and gradually returned to a neutral expression when the activity dropped.

Critically, such spatial or feature-based distinction between the feedback and the task stimulus can induce distraction or interference effects as seen in dual tasks. As a consequence, responses to the task stimulus (e.g., emotional picture) may be reduced, confounding the effects of effortful down-regulation. Here, we delineate a novel approach in which feedback-guided self-regulation is based on visual changes in the stimulus responsible for the targeted brain responses itself. Specifically, the present paradigm uses appetitive food pictures to evoke responses in brain circuits related to hunger and food craving. The success of regulating those responses during picture exposure in the neurofeedback session is represented as gradual changes in image size. Compared to the feedback stimuli used by Veit et al. and Brühl et al. our paradigm has the advantage that distracting/dual-task effects associated with monitoring the feedback stimulus during cue exposure are minimized. At the same time, by linking decreases in image size to successful downregulation and image size increases to failure, respectively, the paradigm provides real motivational consequences of the participant's regulation effort, i.e., the stimuli mimic avoidance behavior during successful down-regulation and approach behavior during unsuccessful down-regulation. Such behavioral/motivational relevance of the task can increase face validity of the neurofeedback training and may thus facilitate its therapeutic use in pathologies of motivational systems, such as obesity or addiction.

#### **THE ISSUE OF HABITUATION**

Another challenge of visual cue reactivity neurofeedback is that repeated exposure to the same or similar visual stimulus will lead to a decrease of neural responses. Such effects have been well documented in brain imaging tasks using fMRI adaption or "repetition suppression" techniques, in which the reduction of BOLD responses associated with the repeated presentation of identical stimuli is used as a tool to characterize the neural representation of visual objects (Grill-Spector et al., 2006). Similar effects of neural adaptation can also occur in brain circuits involved in emotional processes. For instance, multiple presentations of threatening pictures will lead to a gradually weaker activation of the amygdala (Wright et al., 2001). In the present context of down-regulation of cue-induced BOLD activation patterns, it is thus crucial to control for such effects, as a gradual habituation effect over successive regulation blocks may be misinterpreted as reflecting successful regulation. In previous research such control measures have been implemented, for instance, by including a passive viewing condition (either within- or between-runs) where the same set of pictures and the feedback stimulus shown during regulation is repeated but participants are instructed not to regulate their brain responses (Brühl et al., 2014). Here, we took a similar approach by including "mirror" runs in the paradigm, in which participants were exposed to the same size sequence that was "produced" during a previous regulation block but instructed not to regulate target area activation.

# **COMPUTATIONAL TRANSLATION AND RESPONSE RANGE ADAPTATION OF THE BOLD SIGNAL**

All BCIs include a series of steps for converting the measured brain signal, such as percentage signal change, into commands (Linden, 2014). First, the relevant feature is extracted from the wealth of information that the measurement device picks up, ideally in real time. Secondly, the extracted feature needs to be converted into an output signal for the participant to use through a translation algorithm. Finally, the translation from the extracted feature of the brain signal to the output needs to be adaptive. Peoples' individual neurophysiological responses vary widely yet all must ultimately appear in an identical summary form, for their use, projected onto a computer screen. Thus the conversion has to be adapted to an individual's response range. This adaptation to the original signal also has to take into account the fluctuation of the signal over time and the improvement with training as well as an individual's training capacity. For example the duration of the training has to be adjusted to psychological factors like motivation and fatigue, which again corresponds to well-known general principles from teaching and training.

A well designed feedback system is an important criterion in successfully training participants to self-regulate their BOLD response. Contingent feedback following the participant's response with minimum lag and with reliable information content pertaining to task success improves learning (Sitaram et al., 2008). As detailed below, in the current project we developed a solution for steps two (translation algorithm) and three (adaptation) of the motivational BCI approach outlined above.

# **MATERIALS AND METHODS**

# **fMRI BRAIN-COMPUTER NEUROFEEDBACK ARCHITECTURE**

**Figure 1** describes the fMRI-BCI architecture for neurofeedback training at our center. Our architecture is a closed-loop system with the following major subsystems; signal acquisition, signal analysis and signal feedback. In **Figure 1**, the MRI image pool is the end point of the signal acquisition subsystem and is connected to the signal analysis and signal feedback subsystems through a local area network (LAN). The signal analysis subsystem communicates with the signal feedback subsystem and they both reside on the same computer. The signal feedback subsystem is connected to the projector screen, where the feedback signal is presented.

At the signal acquisition subsystem, localized brain activity of the participant while viewing images on the projector screen is measured by fMRI using a BOLD sequence; a contrast is then made between the signal elicited by the target stimuli (food pictures) and neutral control stimuli (household objects in localiser run, see below), or a fixation baseline (regulation runs). A 3 T whole body scanner (General Electric, Milwaukee, USA) with an 8-channel head coil is used. The parameterisation of the BOLD sensitive echo planar imaging (EPI) sequence is as follows: TR = 2 s, TE = 45 ms, flip angle = 80◦ , 30 slices, FOV = 192 mm, image matrix 64 × 64, in-plane voxel size = 3 mm × 3 mm, slice thickness = 4 mm and a gap of 1 mm. Image reconstruction, distortion correction and image averaging are performed on the MRI scanner computer and stored in the MRI image pool.

The signal analysis subsystem is performed using Turbo-Brainvoyager (TBV) version 3.0 (Brain Innovation, Maastricht, The Netherlands; (Goebel, 2001)). TBV retrieves reconstructed fMRI images from the MRI pool via the LAN and performs online 3D motion correction, temporal filtering, spatial smoothing, spatial normalization and online statistical analysis calculating beta parameters from an incremental general linear model (GLM) based on the predictors of interest (e.g., food vs. neutral pictures). A static ROI is selected by drawing an area on the functional map (3D BOLD signal) computed in TBV. All supra-threshold voxels (according to an investigatorchosen statistical threshold between 2 and 3) are included in the target area for signal extraction. Average BOLD signal values (betas) from the target area are extracted by TBV and stored in a continuously updated real-time protocol file (rtp file).

The signal feedback subsystem retrieves the rtp files, processes and analyses the BOLD signal values contained within. Feedback is presented to the participant in real-time as food image sizes corresponding to the percentage change in BOLD signal values of the ROI during food picture presentation relative to fixation baseline using a moving average across three consecutive TRs. Feedback is presented to the participant with a delay that depends on the time for signal acquisition, signal analysis and signal feedback processing. Minimizing the delay is critical for volitional control (Sitaram et al., 2007b). Applying the general experience from operant conditioning experiments, the maximum delay for successful learning of biological responses is 2 min (Yoo et al., 2004). The design and implementation of the visual feedback subsystem for the regulation of hunger or food craving is described in the following sub-sections.

# **DESIGN**

The visual feedback subsystem involves two types of functional imaging run; the neurofeedback and mirror/control runs. The present paradigm involves four runs of each type presented in an alternating order. The neurofeedback runs are the experimental runs, where the participants learn to control their fMRI signal in a block design with periods of rest followed by periods of down-regulation. During the rest blocks, a fixation cross is displayed for 20 s. During the down-regulation block, also 20 s in duration, one food image is presented which varies in size dependent on the percent fMRI signal change, during the block, relative to the preceding fixation block.  During the down-regulation block, the size of the food image is updated every TR (2 s) leading to a consecutive display of 11 different image sizes in total. Image sizes can vary between 10% and 100% of the original image size (1013 by 760 pixels) that are distributed across the whole size range. The size of the food image displayed at the first TR of each down-regulation block is set to 50% of the maximum image size, which corresponds to the percent signal change (PSC) at the first TR (FPSC) relative to the preceding rest

block. Here, the FPSC is taken as the reference scaling point for the calibration of subsequent PSCs. The increase or decrease in image size calibration of the PSC is described using the following calibration algorithm:

```
def PSCDisplay(PSC):
  if PSC < FPSC - 100:
    Display 10% of the image size
  elif FPSC - 100 <= PSC < FPSC - 75:
    Display 15% of the image size
  elif FPSC - 75 <= PSC < FPSC - 50:
    Display 20% of the image size
  elif FPSC - 50 <= PSC < FPSC - 25:
    Display 30% of the image size
  elif FPSC - 25 <= PSC < FPSC:
    Display 40% of the image size
  elif PSC == FPSC:
    Display 50% of the image size
  elif FPSC + 25 >= PSC > FPSC:
    Display 60% of the image size
  elif FPSC + 50 >= PSC > FPSC + 25:
    Display 70% of the image size
  elif FPSC + 75 >= PSC > FPSC + 50:
    Display 80% of the image size
  elif FPSC + 100 >= PSC > FPSC + 75:
```
Display 90% of the image size elif PSC > FPSC + 100: Display 100% of the image size retun Display

This manner of calibrating the PSCs ensures that the effect of the FPSC is reflected in all the image sizes and adapts the feedback subsystem to peoples' individual neurophysiological response range. Each neurofeedback block uses as its feedback signal a different food image from a pool of 16 pictures selected at random. By default the range of the response is set to a 1% deviation from baseline (upwards or downwards), which implies that a 1% upregulation will result in presentation of the full-size image and 1% down-regulation in presentation of the smallest image. Further up- or down-regulation will then not be reflected in further in- or decreases of the image size. However, the gain of the conversion from the extracted brain signal feature (% BOLD signal change) to image size can be set freely to reflect an individual's response range, and can be adjusted adaptively to reflect dynamic changes in self-regulation ability. The food images were taken from the International Affective Picture System (Lang et al., 2005) and the Internet.

After each neurofeedback run, an associated mirror/control run is presented to the participant. In the mirror run the exact same picture (size) sequence (using the same picture

exemplar) is repeated that was shown in the preceding neurofeedback run. The only difference is that during the mirror run participants are instructed NOT to regulate their brain responses during exposure to the food images but should passively watch the image (size) sequences instead. By this means, the mirror run can serve as a perceptual control: when BOLD responses from the mirror run are (offline) subtracted from BOLD responses in the neurofeedback run, any decrease in brain activation (in the target ROI or elsewhere in the brain) *cannot* be attributed to differences in physical stimulation—which may include diminishing brain activation to motivational cues as a result of habituation over time/successive runs—but will likely reflect the down-regulation effort. Importantly, each mirror run is always presented *after* its corresponding regulation run showing the same picture exemplar and size sequence. Any habituationrelated reduction of brain responses caused by repeating the same stimulation in the mirror run thus *cannot* lead to an erroneous detection of down-regulation success (difference regulation mirror) as it works in the opposite direction. **Figures 2A,B** depicts the temporal structure of the neurofeedback and mirror runs.

#### **IMPLEMENTATION**

**Figures 3**, **4** depict flow-charts of the execution path of the neurofeedback and mirror runs respectively. The flow-charts were implemented by writing the sequence of executed events in the Python programming language and executed using the PsychoPy graphical user interface (Peirce, 2007). During the neurofeedback run, the signal feedback subsystem accesses the average BOLD signal values of the ROI time courses (in the rtp files) computed by TBV and presents the feedback display in **Figure 2A**. The mirror run presents a visually identical feedback display (see **Figure 2B**) which uses the average BOLD signal values of the neurofeedback run stored in a separate directory (termed "mirror files"). The mirror files are processed and displayed in synchronization with the average BOLD signal values of the ROI time courses (rtp files) computed by TBV during the mirror run (scanner files).

# **PILOT STUDY TESTING THE FEASIBILITY OF "MOTIVATIONAL NEUROFEEDBACK"**

Data from 10 female participants (mean age *M* = 21.40 years, Standard Deviation (SD) = 2.27) were acquired to test whether the newly developed motivational neurofeedback paradigm can be successfully used to down-regulate brain activation in response to appetitive food pictures. Participants were asked not to eat for 4 h before the scanning session to increase motivational brain responses related to hunger or food craving. Written informed consent was obtained in accordance with the local ethics committee prior to the start of the study.

The experiment began with the functional localizer run followed by an alternating sequence of four neurofeedback and four mirror runs. During the localizer run the participants passively viewed five blocks of food pictures and five blocks of

pictures showing neutral household objects in alternating order. The picture blocks contained five pictures randomly selected from the relevant category, presented for 2 s each. Blocks were interleaved with a 10 s fixation period (with the exception of an initial fixation period at the start of the experiment which was 12 s in duration), resulting in a total run length of 222 s. The sequence of four neurofeedback and mirror runs were presented using the trial/run structure described above. Due to technical problems, for three participants only three neurofeedback/mirror runs could be acquired. For ROI analyses testing the feasibility of down-regulation of the target area in the neurofeedback runs, we used the pre-processed functional images created by the TBV software, which were co-registered to individual high-resolution T1-weighted anatomical images and spatially normalized to Talairach space. Individual volume time courses were spatially smoothed using a kernel with a full

width at half maximum (FWHM) of 4 mm and temporally filtered with a high pass filter of 2 cycles/time course. For each run and participant separately, mean beta estimates in the target area were then extracted based on individual whole-brain GLMs and three different predictors for BOLD signal changes during (i) mirror, (ii) regulation (neurofeedback); and (iii) rest blocks. **Table 1** shows the associated brain regions, mean Talairach coordinates and size of activation clusters selected as target areas for neurofeedback-guided down-regulation in the pilot sample.

# **RESULTS AND DISCUSSION**

Based on a whole brain GLM analysis for the contrast between food and neutral pictures during the localiser run we visually selected for each participant individually a target area (ROI) encompassing the cluster showing the strongest response in the

**Table 1 | Associated brain region (Left/Right), mean Talairach coordinates and size (1** × **1** × **1 mm3 voxels) of activation clusters selected as target areas for neurofeedback-guided down-regulation in the pilot sample**.

Target areas were functionally selected using a localizer scan with food and neutral pictures. The table also includes mean beta differences for the regulation/neurofeedback vs. mirror/passive viewing condition across runs. Negative values indicate successful down-regulation of target area activation during neurofeedback runs.

statistical activation maps (see **Table 1**). The feedback signals (and its corresponding picture sizes) in the subsequent neurofeedback runs were computed as the average percent signal change from all significantly activated voxels within a rectangle drawn over the target region across three axial slices. We constrained the selection to regions with a known involvement in emotional-motivational processes and to non-visual brain areas.

During the neurofeedback run participants were asked to reduce the size of the food images displayed on the projector screen by reducing the average fMRI signal strength in the target ROI. During the mirror run the participants were instructed to passively view the food image sequence repeated from the previous neurofeedback run. **Figure 5** shows mean beta estimates for the pilot sample, separated for the four consecutive regulation and mirror runs. Paired *t*-tests showed that the regulation condition led to significantly lower target area activation in run 2, *t*(9) = 2.49, *p* = 0.034, and in run 3, *t*(9) = 2.77, *p* = 0.022. Activation levels were not significantly different between regulation and passive viewing in run 1 and run 4, *t*s < 1.0. However, as indicated in **Figure 5** this was caused by increased activation in the mirror condition in the first and fourth run while target area activation during regulation was low throughout the session, with beta estimates not exceeding 0.1. Across runs, 8 out of 10 participants successfully reduced target area activation during neurofeedback, showing lower mean beta values for the regulation vs. passive viewing condition (see **Table 1**). To summarize, pilot data suggests that our paradigm enables participants to successfully downregulate brain areas involved in processing motivational cues, such as appetitive food pictures. These results may be relevant to the increasing interest in the combination of Pavlovian and instrumental techniques in neurofeedback research (Mendelsohn et al., 2014).

### **MOTIVATIONAL NEUROFEEDBACK—RELEVANCE FOR CLINICAL APPLICATIONS**

Providing motivationally relevant neurofeedback through changes in stimulus size may also provide an avenue to help patients gain control over dysfunctional motivational processes, such as craving elicited by environmental cues in substance dependence. It is well known that visual cue reactivity, and specifically craving responses to drug-related cues (e.g., in the media or during social interactions) are a major determinant of relapse after treatment of addiction (Weiss, 2005). Moreover, maladaptive brain responses to visual drug cues can be identified in early stages of addictive disorders (Ihssen et al., 2011). Identifying a suitable neurofeedback approach that can help to alter these activation patterns directly would thus provide an important clinical tool complementing traditional psychological and pharmacological interventions. A few studies have begun to test the effects of neurofeedback in the context of addiction. For instance, Li et al. (2013) and Hanlon et al. (2013) trained treatment-seeking smokers to regulate brain responses in craving-related brain areas ACC and PFC during exposure to smoking-related pictures. Pictures were presented in blocks of 22 s during which participants were asked to regulate ACC/PFC activity, followed by a thermometer feedback shown for 4 s. However, these studies did not control for habituation effects as described above, presenting regulation runs always after passive viewing control runs. Moreover, feedback was implemented with a delay. The present approach overcomes these limitations. It may also be especially suitable in an applied setting as it has high face validity and presents to the participant visible, quasi-behavioral consequences of his/her mental regulation effort: Visual cues, such as picture showing high-calorie food or alcoholic beverages, can be directly manipulated through the regulation, i.e., "pushed away" through successful down-regulation or "dragged towards oneself " through up-regulation (or non-regulation). Providing such approach and avoidance consequences of neural selfregulation may increase the likelihood that learned strategies are transferred to a natural environment. The importance of such motivational factors for drug addiction is demonstrated by behavioral interventions showing that making avoidance movements (pushing a joystick) in response to alcohol pictures can change the automatic approach bias and improve treatment outcomes of alcohol-dependent patients (Wiers et al., 2011; Eberl et al., 2013).

On the other hand, the present paradigm allows participants to directly manipulate and control the visual cue which can be predicted to increase the participant's sense of agency and perception of self-efficacy (Bandura, 1977)—factors that are central for the maintenance of drug-abstinent behavior (Greenfield et al., 2000). One limitation of our paradigm relates to the effects that visual changes in picture size may themselves have on motivational brain responses. However, changes in picture size have been shown to affect neural responses of emotional brain networks only marginally (De Cesarei and Codispoti, 2006). More importantly, in the present paradigm such effects are also controlled for by the mirror runs, which repeat the size changes of the regulation run. Nonetheless, care should be taken when defining maximal and minimum picture sizes for a specific scanner-projector set-up in order to remain within a visible range.

#### **SUMMARY AND CONCLUSIONS**

The visual feedback subsystem we have developed has been tailored to specifically allow (i) feedback-guided regulation of visual cue reactivity, i.e., to control brain responses to pictorial cues *during* exposure to those cues; and (ii) to increase motivational relevance of visual feedback by using decreases (avoidance) or increases (approach) of stimulus size as an indication of successful or unsuccessful down-regulation. Further, the feedback subsystem is able to adapt to peoples' individual neurophysiological response range. These individual ranges are scaled to fall within a standard range before being mapped to the corresponding image sizes for display. Importantly, the introduction of the mirror run controls for physical/perceptual confounds, allowing separation of BOLD response changes resulting from successful regulation from those related to variations of visual stimulus properties or visual responses, such as habituation. Finally, the present feedback subsystem displays a new food image size every 2 s allowing good performance and providing a minimum delay in line with existing visual feedback subsystems (Weiskopf et al., 2003, 2004; Sitaram et al., 2007b, 2008). To conclude, our approach may facilitate the control of brain activation during neurofeedback training involving simultaneous presentation of visual cues and may thus help the translation of neurofeedback into clinical applications, such as the regulation

of craving responses to substance-related visual cues in addictive disorders.

# **ACKNOWLEDGMENTS**

This study was supported by the BRAINTRAIN grant, funded by the European Union's Seventh Framework Programme for research, technological development and demonstration under grant agreement no. 602186, a seed corn grant of the MRC Centre for Neuropsychiatric Genetics and Genomics, Cardiff University and MRC grant G1100629.

# **REFERENCES**

**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 31 March 2014; accepted: 20 October 2014; published online: 25 November 2014*.

*Citation: Sokunbi MO, Linden DEJ, Habes I, Johnston S and Ihssen N (2014) Real-time fMRI brain-computer interface: development of a "motivational feedback" subsystem for the regulation of visual cue reactivity. Front. Behav. Neurosci. 8:392. doi: 10.3389/fnbeh.2014.00392*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience*.

*Copyright © 2014 Sokunbi, Linden, Habes, Johnston and Ihssen. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms*.

# Social reward improves the voluntary control over localized brain activity in fMRI-based neurofeedback training

Krystyna A. Mathiak 1, 2, 3 \* † , Eliza M. Alawi 1, 2 †, Yury Koush1, 2, 4, 5, Miriam Dyck 1, 2 , Julia S. Cordes 1, 2, Tilman J. Gaber 2, 3, Florian D. Zepf 6, 7, Nicola Palomero-Gallagher 8 , Pegah Sarkheil 1, 2, Susanne Bergert 1, 2, Mikhail Zvyagintsev 1, 2 and Klaus Mathiak 1, 2

1 Department of Psychiatry, Psychotherapy and Psychosomatics, Behavioral Psychobiology, RWTH Aachen University, Aachen, Germany, 2 Translational Brain Medicine, Jülich-Aachen Research Alliance, Jülich, Aachen, Germany, 3 Department of Child and Adolescent Psychiatry, Psychotherapy and Psychosomatics, RWTH Aachen University, Aachen, Germany, 4 Department of Radiology and Medical Informatics, University of Geneva, Geneva, Switzerland, 5 Institute of Bioengineering, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland, 6 Department of Child and Adolescent Psychiatry, School of Psychiatry and Clinical Neurosciences and School of Paediatrics and Child Health, Faculty of Medicine, Dentistry and Health Sciences, The University of Western Australia (M561), Perth, WA, Australia, 7 Specialised Child and Adolescent Mental Health Services, Department of Health in Western Australia, Perth, WA, Australia, 8 Research Centre Jülich, Institute of Neuroscience and Medicine (INM-1), Jülich, Germany

#### Edited by:

Francisco Javier Zamorano, Universidad del Desarrollo, Chile

#### Reviewed by:

Annette Beatrix Bruehl, University of Zurich, Switzerland Frauke Nees, Central Institute of Mental Health, Germany

#### \*Correspondence:

Krystyna A. Mathiak, Department of Psychiatry, Psychotherapy and Psychosomatics, RWTH Aachen University, Pauwelsstr. 30, 52074 Aachen, Germany<EMAIL_ADDRESS>
† These authors have contributed equally to this work.

Received: 14 November 2014 Accepted: 11 May 2015 Published: 03 June 2015

#### Citation:

Mathiak KA, Alawi EM, Koush Y, Dyck M, Cordes JS, Gaber TJ, Zepf FD, Palomero-Gallagher N, Sarkheil P, Bergert S, Zvyagintsev M and Mathiak K (2015) Social reward improves the voluntary control over localized brain activity in fMRI-based neurofeedback training.

> Front. Behav. Neurosci. 9:136. doi: 10.3389/fnbeh.2015.00136

Neurofeedback (NF) based on real-time functional magnetic resonance imaging (rt-fMRI) allows voluntary regulation of the activity in a selected brain region. For the training of this regulation, a well-designed feedback system is required. Social reward may serve as an effective incentive in NF paradigms, but its efficiency has not yet been tested. Therefore, we developed a social reward NF paradigm and assessed it in comparison with a typical visual NF paradigm (moving bar). We trained twenty-four healthy participants, on three consecutive days, to control activation in dorsal anterior cingulate cortex (ACC) with fMRI-based NF. In the social feedback group, an avatar gradually smiled when ACC activity increased, whereas in the standard feedback group, a moving bar indicated the activation level. In order to assess a transfer of the NF training both groups were asked to up-regulate their brain activity without receiving feedback immediately before and after the NF training (pre- and post-test). Finally, the effect of the acquired NF training on ACC function was evaluated in a cognitive interference task (Simon task) during the pre- and post-test. Social reward led to stronger activity in the ACC and reward-related areas during the NF training when compared to standard feedback. After the training, both groups were able to regulate ACC without receiving feedback, with a trend for stronger responses in the social feedback group. Moreover, despite a lack of behavioral differences, significant higher ACC activations emerged in the cognitive interference task, reflecting a stronger generalization of the NF training on cognitive interference processing after social feedback. Social reward can increase self-regulation in fMRI-based NF and strengthen its effects on neural processing in related tasks, such as cognitive interference. A particular advantage of social feedback is that a direct external reward is provided as in natural social interactions, opening perspectives for implicit learning paradigms.

Keywords: neurofeedback, real-time fMRI, social communication, reward, smile, avatar, Simon task, cognitive interference

# Introduction

People constantly control their brain activity by engaging in voluntary actions that are linked to activation of specific brain regions (deCharms et al., 2005; deCharms, 2007). This does not always work well: to excel in difficult skills, to suppress unwanted emotions or to override automatic actions requires a long and difficult learning process or even sometimes fails altogether. Although we do control the brain activity indirectly via our actions, typically we cannot exert direct control over specific brain regions. Brain imaging techniques, such as functional magnetic resonance imaging (fMRI), help to understand the link between the physiological processes taking place within the brain and our subjective awareness (deCharms, 2007, 2008). Neurofeedback (NF) based on real-time fMRI (rt-fMRI) takes us even a step further: Subjects can voluntarily change the activity in a selected brain region and directly see the effect of the evoked brain activation. Although this particular method is still relatively new and subject to certain limitations, its potential implications are vast.

NF based on electroencephalography (EEG) is wellestablished for the treatment of attention-deficit/hyperactivity disorder (Monastra, 2005) and epilepsy (Sterman and Egner, 2006) for over 4 decades now. fMRI-based NF, enabling the regulation of a precisely selected brain region, became available much later with the development of brain-computer interfaces (BCIs) based on rt-fMRI (for a review, see Weiskopf et al., 2004b, 2007). rt-fMRI NF proved to have a good anatomical resolution and to elicit behavioral changes (for a review, see Birbaumer et al., 2009). However, its use as a clinical method is still limited. One of the limitations is that only about two thirds of the people subjected to this particular method succeed in controlling computerized devices with brain signals, while the remaining one third fails to do so (Friedrich et al., 2014). A number of factors determine how well this control can be achieved, including the training protocol, instructions, tasks, mode of feedback as well as psychological traits such as motivation and expected reward, mood, locus of control, and empathy. Indeed, according to Goebel et al. (2004), the social component of the study paradigm, that is the willingness to compete and win against a real opponent, may lead to very fast and effective learning. In accordance with these observations, deCharms (2008) proposed to develop new types of task paradigms for rt-fMRI NF, where participants would be trained without engaging in a deliberate cognitive process.

In a standard fMRI NF paradigm, participants are presented with a visual display of a color bar moving up-and-down or a fluctuating thermometer that reflect brain activity in a region of interest (ROI). Their task is to raise the level on such a bar display by regulating the brain activity in the selected ROI. It is proposed that successful learning follows the principles of operant conditioning, involving a reward when the required threshold is achieved (McCarthy-Jones, 2012).

**Abbreviations:** ACC, anterior cingulate cortex; BCI, brain-computer interface; EEG, electroencephalography; EPI, echo planar imaging; FWE, family-wise error (correction); NF, neurofeedback; ROI, region of interest; rt-fMRI, real-time functional magnetic resonance imaging; SPM, Statistical Parametric Mapping.

In EEG-based NF, the reward is often explicit, involving appointing points or making a game character move on the screen (Egner and Sterman, 2006) or make a Lego robot move forward (Mirkovic et al., 2013 ´ ). In published rt-fMRI NF studies, the reward is often less direct such as the subject's own satisfaction with successful control of the bar display (or receiving social reward from an experimenter at the end of the task). deCharms et al. (2005) used task-related feedback stimulus, namely images of a fire changing its size to reflect a successful regulation of a pain-related area (rACC). Sokunbi et al. (2014) extended this particular approach, introducing feedback-guided self-regulation based on changing size of appetitive food pictures to regulate brain circuits related to hunger and food craving. They argued that the stimuli mimic avoidance behaviors during successful down-regulation and approach behaviors during unsuccessful down-regulation, increasing the face validity of the used training. Two remarkable studies applied explicit rewards in fMRI NF: Bray et al. (2007) offered monetary rewards when subjects successfully modified their motor cortex activity, and Goebel et al. (2004) added a social rivalry aspect in a so-called Brain-Pong game, where subjects played virtual ping-pong against each other, using their brain activity to control a racket ("brain-pong"). Although all the above attempts were successful, none was shown to be superior to standard feedback signals.

In daily life, we control our brain activity to change our facial expression, prosody, body posture, and other behavior based on subtle feedback signals that we receive from our partners in social interactions. Social reward, such as smile, can activate reward-related areas of the brain, similarly to other reward types, e.g., money (Izuma et al., 2008). We demonstrated in a pilot study that social reward can directly reinforce localized brain activity (Mathiak et al., 2010). In contrast to monetary reward (Bray et al., 2007), social reward can even be provided in realtime, i.e., by displaying positive facial expressions. Similar to the Brain-Pong setup (Goebel et al., 2004), where the inclusion of motivating (but not directly rewarding) social competition improved performance, social reward in the present approach may improve NF training by enhancing the motivation. Here, we investigated the impact of the feedback mode—smiling avatar face vs. standard bar display—on regulation performance during fMRI NF of the anterior cingulate cortex (ACC).

The ACC has been the focus of many studies due to its key role in regulating emotions, goal-directed behaviors, attentional processes, response selection, motor functions (Bush et al., 2000; Carter and van Veen, 2007), and above all in conflict monitoring and error perception (Botvinick et al., 2004; Kerns et al., 2004). The ACC can be successfully controlled using fMRI-based NF with standard paradigms (Weiskopf et al., 2003; deCharms et al., 2005; Emmert et al., 2014; Rance et al., 2014). However, in contrast to some visual and motor areas, no evident strategy emerged that yields activity increases without feedback mechanisms, so that the ACC is a suitable ROI to study learning in rt-fMRI NF. The ACC is reliably activated in both the colorword Stroop and the Simon tasks, which are both based on introducing interfering task-irrelevant stimuli (Peterson et al., 2002). In the Simon task, reactions to a target stimulus are slowed when the location of the target and the response side do not correspond, even though this is task-irrelevant. Both, the Stroop and the Simon task involve the ACC. However, in a direct comparison the Simon task led to significantly stronger ACC activations (Liu et al., 2004). Thus, we applied the Simon task to test for altered activation in the ACC region after NF training (generalization).

Twenty-four healthy participants were randomly allocated to one of two groups: a social and a standard feedback group. As for social reward, an avatar started gradually smiling when ACC activity increased. A bar display indicated the activation levels in the standard feedback group. To control for non-specific effects of the NF procedure, the subjects attempted to up-regulate their brain activity without receiving any feedback directly before (pretest) and after the NF training (post-test). Further during the pre- and post-test, a cognitive interference task (Simon task) investigated change of ACC activity in a novel setting, also without feedback. The ability to voluntarily activate the ACC in an identical paradigm with no feedback served as measure for transfer of the NF training; the impact of the NF training on the ACC activity in the novel setting without feedback assessed generalization of the NF training (after Poppen et al., 1988; Simon and Gluck, 2013).

# Hypotheses

We expected successful NF training in both groups. In comparison with the control group, the social feedback group should demonstrate:

# Materials and Methods

#### Participants

Twenty-four right-handed subjects (13 females; age 25.62 ± 4.79) participated in the study. They were allocated based on the order of their inclusion in the study either to the social (12 subjects, 6 females, age 24.75 ± 2.80) or the standard feedback group (12 subjects, 7 females, χ 2 (1) = 0.168, p > 0.682; age 26.5 ± 6.1, t(22) = −0.891, p > 0.383). The alternating strategy did not exclude selection bias but minimized chronological bias (see Tamm and Hilgers, 2014). All participants were naïve to NF and reported the absence of any acute or history of major neurological or psychiatric disorder, any current use of psychoactive drugs as well as any contraindication for MRI. Written informed consent was obtained prior to participation. Afterwards, demographic information about age, gender, and education was collected. In addition, the participants were asked to complete The Positive and Negative Affect Schedule (PANAS; Watson et al., 1998) before and after NF training on each day. The study protocol was approved by the Ethics Committee of the Medical Faculty of the RWTH Aachen University, Germany, and the study was carried out in accordance with the Declaration of Helsinki.

# Experimental Stimuli and Task Training