One interpretation is offered by the framework of grounded or embodied cognition (Barsalou, 1999). As discussed in the introduction, this framework assumes that processing of affective stimuli can be purely conceptual, or can be accompanied by somatosensory reactions (Niedenthal, 2007; Winkielman et al., 2015b). If an incidental stimulus elicits a somatosensory reaction, and the mechanisms underlying a particular behavior draw on the same set of somatosensory resources, one should observe a congruent influence (Winkielman et al., 2007; Knutson et al., 2008). This interpretation is consistent with existing literature, discussed earlier, showing that viewing pictures of emotional faces or emotional scenes leads to greater physiological effects than viewing emotion words (Harris et al., 2003; Larsen et al., 2003).

Why do emotional pictures exert this influence but not words? One possibility is that the evolutionarily "old" affect system is particularly suited for dealing with pictorial information. A picture of a smiley face, a juicy cake, a slithering snake, or a bloody knife may be quickly interpreted by early visual areas connected to subcortical structures involved in generation of physiological responses (de Gelder et al., 2011). In comparison, words are polysemous—even simple nouns like "cake" or "gun" can appear in contexts that flip its affective meaning (cake=dung, guns=biceps). So, before a word is able to elicit a robust physiological response, it needs to be processed more deeply by more advanced mechanisms, and perhaps even "translated" into a specific sensory representation (Paivio, 1971; Vandenberghe et al., 1996; Naccache et al., 2005; Niedenthal et al., 2009). Furthermore, without the physiological response to guide subsequent processing in a valence-congruent fashion, words might be subject to more "cold" cognitive operations.

A different theoretical perspective suggests that the power of affective pictures comes not from their ability to trigger physiological responses, but from their privileged access to the cognitive semantic system (Potter et al., 1986; Glaser, 1992), including the semantic network representing evaluative information (De Houwer and Hermans, 1994). Consistent with this assumption, some studies found stronger impact of pictures and words in a fast evaluative priming task (Spruyt et al., 2002, but see Kiefer et al., 2017). However, this "privileged semantic access", unlike the "embodiment" explanation, does not predict greater impact of pictures on consumption behavior a finding we observed. It also predicts greater impact of primes on evaluation of drink qualities and evaluation of one's mood—findings we failed to observe. Further, the privileged semantic access account would predict greater impact of supraliminal, than subliminal words, and high-frequency than low-frequency words—again, findings that we failed to observe. On the other hand, this verdict against this explanation needs to be tentative until future studies directly examine the relative availability of semantic concepts after picture and word priming and the relative sensitivity of behavioral and self-report measures.

More generally, the privileged semantic access explanation, may on the first glance seem contradictory to the embodied explanation. However, it can be seen as compatible. After all, theories of embodied cognition propose that semantic processing is supported by somatosensory simulation involving "perception-like" processes. In fact, categorization literature suggests that when participants try to access a deep meaning of an emotional concept, they generate concrete images and develop a specific "situated" conceptualization (Barsalou, 1999; Niedenthal et al., 2005; Oosterwijk et al., 2015). However, that simulation process takes motivation, effort, and time (Wilson-Mendenhall et al., 2011). From that perspective, pictures offer "pre-packaged" situated conceptualizations—an idea which explains their privileged access and greater spontaneous impact on physiology.

One weakness of both the embodiment and the "privileged access" explanations is that they both predict that the impact of words should simply be weaker primes than pictures. However, even though most of the individual effects were not significant, a general tendency was for words to have an opposite effect. One speculative explanation is that because our words were concrete nouns, referring to specific exemplars rather than general traits, they might have triggered comparison contrast operations, thus creating a response tendency opposite to the prime implications (Dijksterhuis et al., 1998). On the other hand, emotional pictures also represent concrete exemplars, albeit their exemplar status might be qualified by a broad affective activation.

Finally, it is important to acknowledge that words can be powerful triggers of affect and actions. Occasionally, a word may be worth a thousand pictures. Humans are a symbolic species and much of their affairs, including those of the heart, take root and express in symbolically-mediated interactions (e.g., via political manifestos, love letters, or arguments presented in journal articles). Words can do that because they "invite and guide" us to develop mental models that real images may not match (Barsalou, 1999; Bergen, 2012). As a result, words can trigger powerful emotional reactions, with all their physiological consequences. Some of them require a deep understanding of the implications of what has been said ("when you left house today, you forgot to turn off the hot iron"). Other words, such as taboo words, terms of endearment, or native-language emotional terms can work directly through previously pre-computed meanings (Harris et al., 2003; Baumeister et al., 2017). Future research may focus on specifying different conditions under which words and pictures are influential as affective primes of judgment and action, and explore more precisely the underlying psychological and physiological mechanisms. In any case, we hope that the words and pictures presented in this article offer an initial contribution toward this goal, and help us move toward better integration of theories of cognition and emotion.

### **ETHICS STATEMENT**

This study was carried out in accordance with the recommendations of UCSD Human Research Protection Program with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the HRPP human subject committee.

## **AUTHOR CONTRIBUTIONS**

YG collected, entered, did preliminary analyses of the data and described the procedures and methods. PW conceptualized the study and wrote the introduction, results, and discussion sections.

# **REFERENCES**

# **FUNDING**

PW was supported by UCSD Academic Senate Grant and by SWPS BST research fund.

### **ACKNOWLEDGMENTS**

Data were collected by the second author and parts were reported in her UCSD Honor's Thesis (unpublished). This work is supported by Academic Senate Grant from UCSD. Data and materials from this research are available and hosted on UCSD server under the following link: http://pages.ucsd. edu/~pwinkiel/koolaid-picture-words.zip. The second author was unreachable during the preparation of the final version of this article.

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2018 Winkielman and Gogolushko. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# **APPENDIX**

# Interplay between Narrative and Bodily Self in Access to Consciousness: No Difference between Self- and Non-self Attributes

Jean-Paul Noel1,2,3, Olaf Blanke1,2,4, Andrea Serino1,2,5 and Roy Salomon1,2,6 \*

1 Laboratory of Cognitive Neuroscience, Faculty of Life Science, Brain Mind Institute, Ecole Polytechnique Federale de Lausanne, Lausanne, Switzerland, 2 Center for Neuroprosthetics, Ecole Polytechnique Federale de Lausanne, Lausanne, Switzerland, 3 Vanderbilt Brain Institute, Vanderbilt University, Nashville, TN, USA, 4 Department of Neurology, University Hospital, Geneva, Switzerland, 5 Department of Psychology, Alma Mater Studiorum – Università di Bologna, Bologna, Italy, 6 Gonda Multidisciplinary Brain Research Center, Bar-Ilan University, Ramat Gan, Israel

#### Edited by:

Guido Hesselmann, Charité, Germany

#### Reviewed by:

Timo Stein, University of Trento, Italy Bianca Van Kemenade, University of Marburg, Germany

> \*Correspondence: Roy Salomon<EMAIL_ADDRESS>
#### Specialty section:

This article was submitted to Consciousness Research, a section of the journal Frontiers in Psychology

Received: 30 October 2016 Accepted: 12 January 2017 Published: 31 January 2017

#### Citation:

Noel J-P, Blanke O, Serino A and Salomon R (2017) Interplay between Narrative and Bodily Self in Access to Consciousness: No Difference between Self- and Non-self Attributes. Front. Psychol. 8:72. doi: 10.3389/fpsyg.2017.00072 The construct of the "self" is conceived as being fundamental in promoting survival. As such, extensive studies have documented preferential processing of self-relevant stimuli. For example, attributes that relate to the self are better encoded and retrieved, and are more readily consciously perceived. The preferential processing of self-relevant information, however, appears to be especially true for physical (e.g., faces), as opposed to psychological (e.g., traits), conceptions of the self. Here, we test whether semantic attributes that participants judge as self-relevant are further processed unconsciously than attributes that were not judged as self-relevant. In Experiment 1, a continuous flash suppression paradigm was employed with "self" and "non-self" attribute words being presented subliminally, and we asked participants to categorize unseen words as either self-related or not. In a second experiment, we attempted to boost putative preferential self-processing by relation to its physical conception, that is, one's own body. To this aim, we repeated Experiment 1 while administrating acoustic stimuli either close or far from the body, i.e., within or outside peripersonal space. Results of both Experiment 1 and 2 demonstrate no difference in breaking suppression for self and non-self words. Additionally, we found that while participants were able to process the physical location of the unseen words (above or below fixation) they were not able to categorize these as self-relevant or not. Finally, results showed that sounds presented in the extra-personal space elicited a more stringent response criterion for "self" in the process of categorizing unseen visual stimuli. This shift in criterion as a consequence of sound location was restricted to the self, as no such effect was observed in the categorization of attributes occurring above or below fixation. Overall, our findings seem to indicate that subliminally presented stimuli are not semantically processed, at least inasmuch as to be categorized as self-relevant or not. However, we do demonstrate that the distance at which acoustic stimuli are presented may alter the balance between self- and non-self biases.

Keywords: self, criterion bias, peripersonal space, multisensory, audio, consciousness

# INTRODUCTION

fpsyg-08-00072 January 27, 2017 Time: 15:14 # 2

The inherent complexity of the construct of the self, has led to distinct conceptualizations typically including several levels of "selfhood" (James, 1890; Damasio, 1996, 1999; Parnas, 2003; Northoff and Bermpohl, 2004; Zahavi, 2005). At a foundational level of the self is a pre-semantic and nonconceptual understanding of the bodily self, which may serve as a scaffold for more complex and abstracts understandings of the self (Damasio, 1996, 1999; Gallagher, 2005; Legrand, 2006, 2007; Blanke and Metzinger, 2009). The pre-reflective level of self-representation has been suggested to be derived from multisensory-motor correspondences (Blanke, 2012; Blanke et al., 2015) and is thought to be developed at an early age (Rochat, 1995, 2003; Filippetti et al., 2013). It is only at later stages of development that higher-level semantic representations of the self – such as the narrative or autobiographical self (Northoff and Bermpohl, 2004; Northoff et al., 2006) – representing reflective conceptualizations of the self across time and space are established (Rochat, 1995, 2003; Damasio, 1996, 1999; Baumeister, 1998). Thus, a core self-representation is suggested to be grounded in the bodily self, while semantic and psychological representations of the self are developed later, putatively scaffolded upon the bodily-self representation, and involving memory and cognitive constructs of oneself across time and space (Arzy et al., 2008; Peer et al., 2015).

Both the bodily and the narrative conceptions of the self have been described as a fundamental construct bearing on human survival (Keenan et al., 2000; Humphrey, 2006) and correspondingly, the processing of self-relevant information has been shown to enjoy a privileged status (Yu and Blake, 1992; Tong and Nakayama, 1999; Sugiura et al., 2000; Salomon et al., 2011, 2013). At the sensory level, an example of this privileged selfprocessing can be observed in the so-called "cocktail party effect" (Cherry, 1953; Wood and Cowan, 1995) in which one's name is automatically picked up by the auditory system among a sea of sensory noise (see Schäfer et al., 2016, for a recent demonstration of self-prioritization effects across various sensory modalities). A similar "self-reference effect" (Rogers et al., 1977) has been described for mnemonic processes in which information that is encoded as being self-relevant is a posteriori better retained (Sui and Humphreys, 2015). Further, it has recently been shown that even an arbitrary and externally imposed relationship between stimuli and the self causes enhanced processing of these stimuli (Sui et al., 2012, 2013; Stein et al., 2016). Thus, seemingly both at the sensory and at higher-order cognitive levels, selfrelated stimuli may enjoy of enhanced or prioritized access to awareness.

The bifurcation between the study of the bodily-self and a narrative-self has nonetheless arguably led to neglecting possible interactions between the sensory signals which are at the basis of the bodily self and higher-order level processes relating to the narrative self (see Canzoneri et al., 2016 for a recent exception).

Regarding the bodily-self, recent studies have shown that coupling masked visual stimuli with related bodily signals affects the emergence of the visual stimuli into awareness.  Our group, for example, has demonstrated that pairing unseen visual stimuli with a congruent body posture causes the unseen visual stimuli to break continuous flash suppression (CFS; Tsuchiya and Koch, 2005; Jiang et al., 2007; Stein et al., 2011) faster than when the stimuli is paired with incongruent proprioceptive signals (Salomon et al., 2013). That is, when participants are presented with highly visible and salient stimuli to their dominant eye (e.g., "Mondrians"), and visual stimuli with a lesser contrast or salience to their non-dominant eye (e.g., CFS paradigm), this latter stimulus will break the Mondrian suppression and thus access consciousness more readily when paired with a congruent (vs. incongruent) body posture. Further, the co-localization of sound and visual stimuli has equally been demonstrated to thrust this latter one into awareness under a breaking CFS paradigm (Aller et al., 2015). Similarly, using a related visual dichoptic presentation paradigm, namely, binocular rivalry (see Alais and Blake, 2005, for review), Lunghi et al. have demonstrated striking interactions between the administration of touch and/or audiotactile stimulation and congruently presented visual stimuli in propelling the latter one into awareness (Lunghi and Alais, 2013; Lunghi et al., 2014). Further, sensory signals presented not only at the body, but also near the body, in the peripersonal space (PPS; Serino et al., 2015), have been demonstrated to heavily influence self-processing (Blanke et al., 2015; Noel et al., 2015a, 2016; Salomon et al., 2016a), and to reference conceptual processing (Canzoneri et al., 2016). Thus, sensory signals from the body or within the PPS may affect the processing of visual and self-related stimuli (see Faivre et al., 2015 for review).

On the other hand, in relation to the preferential processing of the semantic self, the scope and limits of unconscious semantic processing have revealed contradictory results – arguably in great measure as a consequence of the utilization of different experimental paradigms (Hesselmann and Moors, 2015), but also due to distinct levels of association between physical and psychological conceptions of the self. While researchers employing unconscious priming (which may arguably primarily index non-conscious processing, as opposed to access to consciousness, as CFS does; Stein et al., 2011) have demonstrated powerful effects of unconscious social primes influencing posterior behavior (Bargh et al., 1996; Dijksterhuis and Van Knippenberg, 2000), these subliminal effects are not always present – in particular when scrutinizing the possibility of observing subliminal semantic processing (Zimba and Blake, 1983; Blake, 1988; Kang et al., 2011). In fact, researchers have argued that the default stance should be not to expect much (self or non-self) semantic unconscious processing during incongruent dichoptic presentation (Hesselmann and Moors, 2015), and Blake (1988) inclusively reported that even the first name of a specific observer, which represents a fundamental constituent of the semantic self (Joubert, 1991) proved insufficient to alter dominance of one percept (self) over another (non-self) under a condition of dichoptic stimulation. Tacikowski and Ehrsson

(2016) have recently reported a similar effect, in that even one's own name, when masked, was insufficient to act as a self-prime.

Contrarily, recent studies utilizing a breaking CFS paradigm have demonstrated a host of effects presumably relying on unconscious semantic processing. Sklar et al. (2012), for instance, have shown that semantically incoherent expressions broke suppression (that is, "emerged" into visual awareness) more readily than semantically coherent expressions. Similarly, these researchers showed that increasing the negative affect of expressions lowered suppression time significantly, and that effortful arithmetic equations can be solved without awareness (Sklar et al., 2012). Yet other groups have demonstrated that emotional information is processed during suppression (Yang et al., 2007), scene congruency information can be extracted in the absence of visual awareness (Mudrik et al., 2011), and that word meaning (Costello et al., 2009) and word valence (Yang and Yeh, 2011) can be processed unconsciously. Further, particularly with regard access to consciousness of self-relevant information under the context of breaking CFS, Geng et al. (2012) have demonstrated that one's own face is detected more rapidly than other faces, while Stein et al. (2016), did not show the analogous effect when a visual stimulus (gabor) was arbitrarily labeled "You" as opposed to "Other" (Stein et al., 2016).

Overall, the recent demonstrations of semantic processing under CFS beget the question whether an unconscious semantic representation of the self can be processed unconsciously. The prospect of revealing semantic self-processing unconsciously has been troublesome, but evidence indicates that this prospect may be bolstered by investigating semantic self-processing in combination with lower-level understandings of the bodily self. Here we investigated if participants are (i) able to process self-relevant semantic stimuli unconsciously and (ii) if this is modulated by task irrelevant stimuli, which tap into a specific multisensory dimension of the bodily self, i.e., the PPS. In a first experiment, we present participants with self-attributed personality trait to their non-dominant eye, while presenting dynamic high-contrast stimuli to their dominant one in order to mask the self-attribute. Participants were then to categorize the presented but unseen words as either self-related or not. We tested whether participants were able to semantically process self-related information inasmuch to appropriately categorize the words as self-relevant or not. Successful categorization of unseen words as either self-related or not is taken to index unconscious self-processing and is thus the dependent variable of interest here. That is, we questioned whether subjectively invisible words could be nonetheless accurately classified as self vs. non-self. In a second experiment, we tested whether such putative self-effect could be modulated by a low-level manipulation prioritizing the representation of the bodily self. One key aspect of bodily self is its link to the body and to the space immediately surrounding it, i.e., the PPS (Blanke, 2012; Blanke et al., 2015; Noel et al., 2015a, 2016; Canzoneri et al., 2016; Salomon et al., 2016a). Thus, here, we tested the hypothesis that presenting an acoustic stimuli within, as opposed to outside, PPS may bolster unconscious semantic self-processing. In this manner, we aimed at testing a direct link between bodily and narrative representations of the self and how they interact to modulate awareness.

Continuous flash suppression was utilized in order to mask presented stimuli as it has been suggested that this paradigm allows for some sort of non-conscious physical/perceptual (Geng et al., 2012), but not the psychological/conceptual self-processing (Stein et al., 2016). Hence, it represents an ideal candidate to reveal prioritized non-conscious processing by the psychological self, when this latter one is related to its physical conception. Lastly, self-rated personality attributes were used as these tap into the core of the psychological self while equally being malleable in time and self-selected by the participants themselves. That is, in contrast to other experiments that have used a fixed set of stimuli (e.g., Blake, 1988), here we aimed at having a set of words that were always the same yet for some participants fell within the "self " category, while for other fell in the "non-self " category.

#### MATERIALS AND METHODS

#### Participants

Forty-five native French speakers (nine females, mean age = 22.2 years old, range = 18–30 years old) completed a self-attribution questionnaire (see below), out of which 23 (six females, mean age = 21.4 years old, range = 18–27 years old) partook in both Experiment 1 and Experiment 2. The rest of participants were discarded from participation in the experimental phase, as their self-reports (see below) showed a bias in the attributes they categorized as "self " or "non-self " in terms of either the word-length, or the frequency of the select word in the French language. All participants were right-handed, had normal or corrected-to-normal visual acuity, and reported normal hearing. The study was approved by the Brain Mind Institute Ethics Committee for Human Behavioral Research of the EPFL, and conducted in line with the Declaration of Helsinki. All participants gave informed consent prior to participation and were remunerated with 20 Swiss Francs for their time.

## Self-attribution Reports

#### Materials, Procedure, and Analyses

Anderson's Likableness ratings of 555 personality-trait words (Anderson, 1968) were sorted from most to least likable (according to the mean values in Anderson, 1968), and the middle 200 words (word 178: hopeful to word 378: frustrated1 ) were translated into French by three native French speakers. If translators did not independently agree on the translation of an attribute, they were asked to reach a consensus. This middle section of Anderson's personality-trait words was selected in order to increase between-subject variability in self-attribution reports (see below) while also minimizing the within-word likability variance.

Forty-five participants completed, online and at their own time of convenience, a self-attribution questionnaire, which

1http://psy2.ucsd.edu/~pwinkiel/words.txt

was constituted of the translated 200 personality-trait words. Participants were asked to rate on a Likert Scale (1 = completely not-self to 7 = completely self) whether the attributes described themselves (self) or not (non-self), and whether they liked the word (also on a Likert Scale). No further instruction was given to participants completing the questionnaire in terms of what they should consider when judging whether a particular attribute described them or not. Word order was randomly shuffled between participants. Upon receiving participant's response to the self-attribution questionnaire, for each subject individually, the list of word was sorted from most to least "self," and the top and lower 50 words were compared by means of a paired t-test (alpha set at 0.05, two-tailed) for word-length, frequency in the French language (word frequency was obtained from http: //www.lexique.org/listes/liste\_mots.php), and likability. If for a particular subject the group of self and non-self words differed any of the variables stated above (length, frequency, or likability), the participant was not invited to partake in Experiments 1 and 2.

# Experiment 1

#### Materials and Apparatus

Visual stimuli consisted of high-contrast dynamic noise patches suppressors (Mondrians; Jiang and He, 2006; Hesselmann and Malach, 2011) and target stimuli. The target stimuli consisted of one of the personality-trait words (either "self " or "non-self ") in black Times New Roman, font 12 (example in **Figure 1**; "Hopeful"). Background was white. Mondrians (noise stimuli) were flashed at 10 Hz to the participants' dominant eye, and the targets were presented simultaneously to the other eye. A red fixation point was presented to both eyes. The target words we presented either above or below the fixation point in a randomized fashion. Stimuli were presented using ExpyVR2 , an in-house custom-built multimedia stimuli presentation software developed with Python 2.6 and the Open Graphics Library v.2.2. The stimuli were viewed via a Head-Mounted Display (HMD: VR1280, Immersion Inc, SXGA, 60◦ diagonal field of view, refresh rate 60 Hz). Participants' responses were gathered via button-press on a gamepad (XBOX 360 controller, Microsoft, Redmond, WA, 215 Hz sampling rate).

#### Procedure

Ten to 15 days after completion of the self-attribution report, subjects were brought into the lab for completion of the experimental phase (Experiments 1 and 2, which were ran in a counter-balanced order across participants). Participants were first tested for ocular dominance via the Miles test (Miles, 1930). Then, they were instructed that on each trial, in addition to the fixation point, which they were to gaze at, they would see a random pattern of colorful squares. They were equally told that, during some trials they would additionally see "a sequence of letters." They were instructed to press, during the trial, a button on a gamepad if they saw this additional stimulus. Those trials were indexed as having broke suppression, and removed from analyses for the stimuli categorization and location questions (see below), as we are interested in trials for which no subjective experience of the words was reported (i.e., unconscious self-processing). Then, as portrayed in **Figure 1** (upper), subjects were informed that upon completion of a trial, they would see the fixation point changing color: first blue, which indicated participants were to answer to the question "Was the additional stimuli self-related or not?", then green, which indicated that subjects were to answer to the question "Was the additional stimuli presented above or below fixation?" The questions were always asked in this order, and participants were reassured that if they had not seen anything (apart from the Mondrians and the fixation point) they were to simply guess. The question related to stimuli location (above or below fixation) was employed as a probe that some minimal form of processing was present when suppressing self-attributes. Participants responded via button-press, which mapping (self/non-self and above/below) was randomized across participants.

Trial presentation (Mondrians and personality-trait words) lasted for 3 s. Contrast of the target stimuli was ramped linearly from zero to full contrast over the 1 s. Participants were given unlimited time to answer both questions (Self-Attribution and Stimulus Location). Inter-trial interval was randomly shuffled between 1 and 1.5 s. The experiment consisted of 200 trials (presentation of each of the 100 participant-specific personalitytrait attributes – 50 self and 50 non-self – twice. Once above and once below fixation). Total experimental time was about 30 min.

#### Analyses

Participants' responses were first analyzed in terms of percentage of trials in which subjects broke suppression for self vs. non-self words, as well as for words presented above vs. below fixation. Then, subsequently to discarding the trials in which participants broke suppression (e.g., participants were conscious of the stimuli), Signal Detection Theory (SDT) analysis (sensitivity – d0 , and criterion – c) was applied to both the Self-Attribution and Stimulus Location questions. Null effects were assessed using JZS Bayes factor (BF) tests with default prior scales (Rouder et al., 2012) so that a BF < 0.33 implies substantial evidence for the null hypothesis, 0.33 < B < 3 suggests insensitivity of the data, and B > 3 implies substantial evidence for the alternative hypothesis (see Dienes, 2011).

#### Experiment 2

The material and apparatus, as well as the procedure and analyses used in and applied to Experiment 2 (which, again, was counterbalance in order with Experiment 1 across participants) followed largely those of Experiment 1, for the exception of the following.

In order to probe at whether an implicit association with the bodily self, by delivery of auditory stimuli within the PPS (Noel et al., 2015a; Serino et al., 2015; Salomon et al., 2016a), would boost non-conscious self-processing, acoustic stimuli (white noise) were presented both close to, and far from, the body. Specifically, as depicted in **Figure 1** (lower), a speaker (Near) was placed 20 cm in front of the participants in the anterior-posterior axis, at his or her midline and at sternum level, while a second speaker (Far) was placed 100 cm away (at the same horizontal and elevation level). The speakers emitted white noise for the last second of trial presentation. They were both regulated in

2http://lnco.epfl.ch/expyvr

fixation?". In Experiment 2, the same protocol as in Experiment 1 was repeated, but additionally a white noise sound was presented for a second (the last second of the attribute presentation) either from the near or from the far speaker.

order to produce 70 dB(A) of sound measured at the participants' ear. White noise was utilized (as opposed to more naturalistic stimuli) as this stimuli has been demonstrated to most effectively drive neurons encoding for PPS (Fogassi et al., 1996), and the onset of the acoustic stimuli was offset with respect of onset of visual stimuli in order to minimize the effect audio onset can have in thrusting a visual stimuli into awareness (Aller et al., 2015). Experiment 2, as Experiment 1, also consisted of 200 trials (presentation of each of the 100 participant-specific personalitytrait attributes twice. Once while a near sound was presented, and once while a far sound was presented (randomized on a trial by trial fashion). Total experimental time was about 30 min.

# RESULTS

# Experiment 1

#### Breaking Suppression

A repeated measures ANOVA was ran on the percentage of occasions in which self vs. non-self words broke suppression, as well as for words presented above vs. below fixation. Results revealed no significant difference between levels for neither Self-Attribution [F(1,21) = 0.241, p = 0.83] or Stimulus Location [F(1,21) = 0.832, p = 0.64], nor an interaction between these variables [F(1,21) = 0.828, p = 0.40]. Bayesian statistics demonstrated a BF of 0.23 in the case of the former and 1.15

in the case of the latter. Thus, in the case of the frequency with which self vs. non-self words broke suppression, data implies substantial evidence for the null hypothesis. Overall, stimuli broke suppression in about a fifth of the trials (self: M = 20.0%, SE = 6.4%; non-self: M = 19.7%, SE = 6.4%; above: M = 19.0%, SE = 6.5%; below: M = 20.7%, SE = 6.3%).

#### Self-Attribution

Sensitivity and criterion analysis on participants' response to the question; "Was the additional stimuli self-related or not?" was effectuated by means of purpose-made scripts in MATLAB (equations follow Macmillan and Creelman, 2005) and subsequent one-sample t-test comparison to zero (meaning no sensitivity and no bias, respectively for d' and c). The "Self " category was defined as "target," while the "Non-Self " category was used as "noise." Results revealed that participants were not able to discriminate between self and non-self words [d'; M = −0.49, SEM = 0.27; t(22) = −1.35, p = 0.32], nor did they show any bias [c; M = 0.03, SEM = 0.06; t(22) = −0.60, p = 0.55]. In the case of d', Bayesian statistical analysis suggests a lack of sensitivity data (BF = 0.82), nonetheless, in the case of the response bias Bayesian statistics provided substantial evidence for the null hypothesis (BF = 0.25).

#### Stimulus Location

In terms of subjects' responses to the question about Stimulus Location; "Was the additional stimuli presented above or below fixation?", and defining "Above" as "target" and "Below" as "noise," findings revealed that participants were indeed able to discriminate between the elevation of the personality traits presented [d'; M = 1.24, SEM = 0.19, t(22) = 5.82, p < 0.001] and did not show a criterion bias [c; M = 0.10, SEM = 0.05; t(22) = −1.24, p = 0.12]. In the case of the latter, the Bayes factor (BF = 1.1) seems to indicate a lack of sensitivity in the data.

# Experiment 2

#### Breaking Suppression

Repeated-measures ANOVAs were conducted on the percentage of occasions in which self vs. non-self words broke suppression, as well as for word location (Above vs. Below fixation), as a function of the location of sound presentation (Near or Far). Concerning the frequency in which self vs. non-Self words broke suppression as a function of the distance of sound presentation, results revealed no main effect, neither for Self-Attribution [F(1,21) = 0.83, p = 0.37], or Sound Location [F(1,21) = 0.04, p = 0.92], nor an interaction between these variables [F(1,21) = 0.99, p = 0.33]. In the case of the main effect Bayesian statistics demonstrated insensitive data (all BFs > 0.62), while in the case of the interaction these statistics provide substantial evidence for the null hypothesis (BF = 0.18).

A similar pattern of results was observed for the frequency with which words presented Above vs. Bellow fixation broke suppression, as a function of Sound Location. The repeated-measures ANOVA showed no main effect of Word Location [F(1,21) = 0.94, p = 0.35] or Sound Location [F(1,21) = 0.745, p = 0.41], nor an interaction between these variables [F(1,21) = 0.56, p = 0.57]. Bayesian Factors demonstrated substantial evidence for the null effect in the case of breaking suppression for attributes presented above vs. below fixation (BF = 0.22), while in the other cases Bayesian factors demonstrated the insensitivity of the data (all BF > 0.38).

#### Self-Attribution

As for Experiment 1, the analysis of the self-attribution question was performed by SDT. We compare sensitivity and criterion bias across Sound Locations by means of a paired-samples t-test, and subsequently, if Sound Location significantly modified either the sensitivity or criterion to Self or Non-self attributed words, we compared these parameters to the zero-baseline. As portrayed in **Figure 2A**, results revealed that regardless of the Sound Location, participants were not able to discriminate between Self and Non-Self words [t(21) = −0.58, p = 0.56; d 0 near, M = −0.14, SEM = 0.19; d 0 far, M = −0.08, SEM = 0.16], as in Experiment 1. Further, Bayesian statistics demonstrated substantial evidence for the null hypothesis, both for the case of close sounds (BF = 0.29) and far sounds (BF = 0.26).

In terms of the criterion, however, as shown in **Figure 2B**, findings did demonstrate a significant difference between Sound Location levels [t(21) = 3.76, p < 0.01], revealing a more conservative approach for identifying "Self " when a sound was presented Far (M = 0.07, SEM = 0.02), as opposed to when the sound was presented Near (M = −0.003, SEM = 0.05) to the participant. Comparisons to zero, indicated that the "far criterion," was indeed significantly different from no bias [t(21) = 1.80, p = 0.03], while the "near criterion" was not [t(21) = −0.05, p = 0.95]. These findings are further supported by the demonstration that in the case of the latter (near sounds), but not the former (far sounds), Bayesian statistics provided substantial evidence in favor of the null hypothesis (BF = 0.22).

#### Stimulus Location

With regard whether the target stimuli was presented above or below fixation, analysis demonstrated no significant difference between Sound Location levels [t(21) = −0.31, p = 0.75, BF = 0.32]. It must be noted, however, that as depicted in **Figure 2C**, and replicating the results from Experiment 1, participants did prove to be overall sensitive to the location of the target stimuli [d'; M = 1.05, SEM = 0.29, t(21) = 4.622, p < 0.001, two-tailed one-sample t-test vs. zero].

Lastly, contrarily to the case of the Self-Attribution reports, and as shown in **Figure 2D**, participants did not show a systematic bias in their response to the elevation of target stimuli as a consequence of Sound Location [t(21) = 0.338, p = 0.73]. BF was equal to 0.32, providing evidence for the postulation that participants were equally likely to categorize unseen visual stimuli as happening above or below fixation regardless of whether an acoustic stimulus was presented near or far from them.

## DISCUSSION

In the present study, we first posed the question whether self-relevant (vs. non-self relevant) semantic information is preferentially unconsciously processed. In addition, we tested

from them (D). Error bars represent ±1 SEM. ∗p < 0.05; ∗∗p < 0.01.

whether such self-effect could be modulated by priming a bodilyself representation via the presentation of a near sound within PPS.

In line with previous studies investigating semantic processing outside awareness (Zimba and Blake, 1983; Blake and Logothetis, 2002; Dehaene et al., 2006), our results indicate that masked words were not processed at the semantic level – at least inasmuch as to reveal a self-processing advantage (see Tacikowski and Ehrsson, 2016, as well as Stein, et al., 2016). This was true both in the case when no sounds were presented, and when sounds were presented either close to or far from the participant. It is important to highlight, however, that participants were able to correctly localize the elevation of the presented unseen visual stimuli, and thus, the lack of evidence for semantic processing is apparent even in light of clear evidence for lower-level subliminal processing. Blake (1988) also reported that even the first name of a specific observer, which represents a fundamental constituent of the semantic self (Joubert, 1991), proved insufficient to alter dominance of one percept (self) over another (non-self) under a condition of dichoptic stimulation. In contrast to the traditional view, however, an array of recent studies employing the b-CFS paradigm seem to demonstrate high-level processing of stimuli suppressed from conscious experience. Most notably, Jiang et al. (2007) showed that words that were visually familiar to a specific population's lexicon broke suppression more readily than foreign words. Costello et al. (2009), similarly demonstrated that semantic priming subsequently accelerated breaking suppression of related words, and finally Yang and Yeh (2011) reported that the affective connotation of an unseen word altered the duration for which

they could be maintained under suppression. It is under this context – one that has previously indicated the presence of semantic processing outside of awareness (Jiang et al., 2007; Costello et al., 2009) and even evidencing congruency priming under complete unawareness (Faivre et al., 2014; but see Noel et al., 2015b), that the current findings are important. It may be that the familiarity of words and their affective connotation can be processed and even integrated outside awareness, yet self-related and non-self related words are not differentially prioritized, at least inasmuch as this processing would entail a sensitive categorization of self and non-self words into their appropriate categories unconsciously. That is, although generally studies employing dichoptic presentation methods other than CFS have reported limited unconscious semantic processing (e.g., Blake, 1988), the recent surge in popularity of the CFS method and the numerous findings indicating unconscious processing under this paradigm has renewed interest in unveiling unconscious semantic processing. In the current study, we thus utilize the CFS method in order to mask self-attributes, and our results demonstrate no prioritized unconscious processing of the self vs. the non-self words.

On the other hand, our results revealed an interesting result relating to a shift in the criterion bias as a consequence of the location of sound presentation. That is, while in Experiment 1 no criterion bias was present, findings from Experiment 2 revealed a more conservative self-categorization approach (or a more liberal other-categorization) when sounds were presented outside, as opposed to inside, their PPS. That is, individuals required more evidence in order to judge that the unseen word presented was "self " when a sound was presented far as opposed to near. This differential response bias as a consequence of sound location was specific for the self dimension, as we did not observe a differential response bias among sound locations for the indication of whether subliminal visual stimuli were presented above or below fixation. That is, arguably, the self-other distinction process was modified when an auditory stimulus was provided far from the self. This effect is reminiscent of behavioral (Tajadura-Jiménez et al., 2013; Teneggi et al., 2013; Noel et al., 2015a; Salomon et al., 2016a) and neuroimaging (Brozzoli et al., 2012, 2013) multisensory studies demonstrating a self-other differentiation process grounded on the representation of PPS. PPS, indeed, might represent a multisensory-motor representation of the self in interaction with the environment (Noel et al., 2014; Galli et al., 2015; Serino et al., 2015), and the PPS boundaries might define a first level of selfother distinction (Noel et al., 2016). In fact, Blanke et al. (2015) hypothesized that changes in the size of PPS neurons receptive fields might underpin alterations in states of bodily self-consciousness, and we have recently demonstrated that subjective changes in self-location co-vary with changes (i.e., spatial translations) in the representation of PPS (Noel et al., 2015a) and even for unconscious manipulations of bodily self consciousness (Salomon et al., 2016a). Here, we show that priming the PPS boundary by presenting an auditory stimulus within the PPS, as contrasted to a stimulus outside the PPS, can modify the criterion with which individuals categorize stimuli as self-related or other-related. That is, the administration of spatially restricted exteroceptive stimuli either within or outside the PPS alters cognitive processes with regard the self. We find this observation particularly interesting as it seems to imply a continuous and dynamic re-assessment to the self as external objects and events come in contact with our sensory modules. Importantly, this change in criterion as a consequence of sound location, however, did not modulate the emergence of self-attributes into awareness, but changed the post-perceptual criterion participants used to determine whether the invisible word was self-related or not.

# CONCLUSION

It appears that though bodily representations of the self are able to modulate access to visual awareness (Lunghi et al., 2010; Lunghi and Alais, 2013; Salomon et al., 2015b), they do not enhance non-conscious processing of the narrative self (or at least association with the PPS does not). Importantly, self-attributes did not emerge into awareness in a prioritized way, even when presented in synchrony with exteroceptive signals that did alter self-bias. This lack of effect could putatively emanate from an inability of unconscious semantic processing, and not from a lack of enhanced self-processing. Nonetheless this option is in contrast to the results of recent studies demonstrating semantic processing of subliminal stimuli masked by means of CFS (Yang and Yeh, 2011; Prioli and Kahan, 2015). In conclusion, our results demonstrate no evidence for differential processing of masked semantic self-relevant stimuli, but do demonstrate a shift in criterion bias as a function of whether stimuli are presented in the peri- or extra-personal space.

# ETHICS STATEMENT

The study was approved by the Brain Mind Institute Ethics Committee for Human Behavioral Research of the EPFL, and conducted in line with the Declaration of Helsinki. All participants gave informed consent prior to participation and were remunerated with 20 Swiss Francs for their time.

# AUTHOR CONTRIBUTIONS

OB, AS, and RS conceived of the study, which was performed by J-PN. J-PN analyzed the data and wrote the manuscript, which was then edited by AS and RS. All authors approved the final version of the manuscript.

# ACKNOWLEDGMENTS

RS was supported by the National Center of Competence in Research (NCCR) "SYNAPSY – The Synaptic Bases of Mental Diseases" financed by the Swiss National Science Foundation (No. 51AU40\_125759). J-PN was supported by a Fulbright Scholarship from the US Institute of International Education.

# REFERENCES

fpsyg-08-00072 January 27, 2017 Time: 15:14 # 9

suppression? Front. Hum. Neurosci. 5:167. doi: 10.3389/fnhum.2011. 00167 doi: 10.3389/fnhum.2011.00167

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Noel, Blanke, Serino and Salomon. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Between-Subject Variability in the Breaking Continuous Flash Suppression Paradigm: Potential Causes, Consequences, and Solutions

#### Surya Gayet 1 \* and Timo Stein2, 3

1 Department of Experimental Psychology, Utrecht Univeristy, Utrecht, Netherlands, 2 Center for Mind/Brain Sciences, University of Trento, Rovereto, Italy, 3 Amsterdam Brain and Cognition, University of Amsterdam, Amsterdam, Netherlands

#### Edited by:

Guido Hesselmann, Charité Universitätsmedizin Berlin, Germany

#### Reviewed by:

Pieter Moors, KU Leuven, Belgium Zhicheng Lin, Ohio State University at Columbus, USA

> \*Correspondence: Surya Gayet<EMAIL_ADDRESS>
#### Specialty section:

This article was submitted to Consciousness Research, a section of the journal Frontiers in Psychology

Received: 17 January 2017 Accepted: 08 March 2017 Published: 27 March 2017

#### Citation:

Gayet S and Stein T (2017) Between-Subject Variability in the Breaking Continuous Flash Suppression Paradigm: Potential Causes, Consequences, and Solutions. Front. Psychol. 8:437. doi: 10.3389/fpsyg.2017.00437 A recent focus in the field of consciousness research involves investigating the propensity of initially non-conscious visual information to gain access to consciousness. A critical tool for measuring conscious access is the so-called breaking continuous flash suppression paradigm (b-CFS). In this paradigm, a high contrast dynamic pattern is presented to one eye, thereby temporarily suppressing a target stimulus that is presented to the other eye. The time it takes for observers to report (e.g., the location of) the initially suppressed stimulus provides a measure of conscious access. Typical observations in b-CFS studies include the finding that upright faces are released from suppression faster than inverted faces, and the finding that stimuli that match the current content of visual working memory are released from suppression faster than mismatching stimuli. Interestingly, the extent to which observers exhibit these effects varies extensively (in the range of hundreds of milliseconds). By re-analyzing existing datasets and a new dataset we establish that the difference in RTs between conditions in b-CFS tasks (i.e., the effect of interest) is highly correlated with participants' overall suppression durations, and with their trial-to-trial variability in RTs. We advocate the usage of a simple latency- normalization method, which (1) removes the between-subject variability in suppression duration from the effect of interest, while (2) providing distributions of RT differences that are better suited for parametric testing. We next compare this latency-normalization method to two other transformations that are widely applied on within-subject RT data (z-transformations and log-transformations). Finally, we tentatively discuss how trial-to-trial variability and overall suppression duration might relate to prolonged phases of shallow suppression that are more prone to modulations of conscious access.

Keywords: consciousness, visual awareness, continuous flash suppression, binocular rivalry, individual differences, response times, normalization, assumption of normality

# INTRODUCTION

Consciousness researchers have many psychophysical tools at their disposal to render visual input invisible to the observer (for reviews, see Kim and Blake, 2005; Breitmeyer, 2015; Overgaard, 2015). One method in particular, however, has been increasingly dominant in the recent consciousness literature. Continuous flash suppression (CFS; Tsuchiya and Koch, 2005) consists of presenting a high contrast dynamic masking stimulus to one eye, thereby causing prolonged suppression of a lower contrast static target stimulus presented to the other eye. In 2015 alone, more than 40 studies have been published that use CFS. In particular, many studies use the time it takes for an initially suppressed target stimulus to overcome CFS as a measure of conscious access. This method is referred to as "breaking continuous flash suppression" (b-CFS; Jiang et al., 2007; Stein et al., 2011a; for a review, see Gayet et al., 2014).

Recent findings using this method have sparked the debate about the extent to which visual input is processed in the absence of consciousness (e.g., Hassin, 2013; Gayet et al., 2014; Yang et al., 2014; Hesselmann and Moors, 2015). Response times (RTs) in b-CFS experiments are lower than in traditional detection tasks, where information is not interocularly suppressed. Simultaneously, raw RT differences between experimental conditions are typically larger (in absolute time) than those typically observed with traditional detection tasks (e.g., comparing Stein et al., 2011a, with Lewis and Ellis, 2003). Eyeballing individual participants' b-CFS data (e.g., in Jiang et al., 2007; Wang et al., 2012) reveals an interesting data pattern: participants with slower overall RTs across conditions typically exhibit a larger RT difference between conditions. A number of recent b-CFS studies explicitly reported this correlation between overall RT and raw RT difference between conditions (e.g., Gayet et al., 2016a,b). Similarly, it has been demonstrated that artificially lengthening the RTs (e.g., by reducing the contrast of the target or by using more potent masks) induces larger raw RT differences (Stein et al., 2011a). As such, the relatively slow RTs in the b-CFS paradigm might play a role in the paradigm's potency for uncovering (small) differences in processing strength between stimulus conditions, which would drown in noise using traditional methods. Another interesting observation is that the trial-totrial variability in RTs in b-CFS experiments is higher than in traditional detection tasks where information is not interocularly suppressed (i.e., within-participant variance is typically much larger in b-CFS conditions than in control conditions involving no interocular competition). Greater variability in detection times implies that observers have greater uncertainty regarding the appearance of a target stimulus. This might also play a role in the b-CFS paradigm's sensitivity for detecting differences between conditions (for a discussion, see Stein et al., 2011a). Considering that the b-CFS paradigm is widely employed to answer important theoretical and philosophical questions on consciousness, it is important to better understand the mechanisms that play a role in the potency of the b-CFS method in uncovering differences in visual processing between experimental conditions.

The purpose of this article is 3-fold. First, we empirically establish the abovementioned relationships between overall RTs, trial-to-trial variability and raw RT differences within the b-CFS paradigm. For this, we use two of the most replicated findings of the b-CFS literature: face inversion and VWM boost. For those effects, we demonstrate that a participant's raw RT difference between two experimental conditions is strongly correlated with (A) the participant's overall RT across conditions and (B) the participant's trial-to-trial variability in RTs. As it is well established that overall RTs correlate with RT variability (Wagenmakers and Brown, 2007), a correlation of both metrics with RT differences is expected. Second, we suggest that this observation is of particular relevance in the b-CFS paradigm, where between-subject variability in RTs stems from both interindividual differences in interocular suppression durations, as well as interindividual differences in response speed after the interocular conflict is resolved. Following this consideration, we provide a clear-cut solution for handling data in which such a correlation is observed. The proposed latency-normalization procedure, which removes between-subject variability of-nointerest, has been used in only a few previous b-CFS studies. Here, we show for the first time that (1) this latency-normalization procedure yields RT differences that more closely approximate a normal distribution, and (2) we make the case that it provides increased sensitivity for detecting differences between experimental conditions. Third, we tentatively propose that the correlation between overall RTs, trial-to-trial variability in RTs, and raw RT differences between conditions could reflect the role of perceptual uncertainty in modulating RTs in the b-CFS paradigm. Finally, we discuss how this putative role of perceptual uncertainty could allow for reconciling contradictory and surprising findings that have been observed in recent b-CFS studies.

For these purposes, we retrieved data from three existing b-CFS datasets that we deemed most representative of the b-CFS paradigm. Following Gayet et al. (2014) we distinguished between two types of manipulations in the b-CFS paradigm. The first type of manipulations in b-CFS paradigms consists of manipulations of stimulus content, in which two different stimuli are compared in their propensity to reach conscious access (e.g., faces vs. houses). Arguably, the most reliable manipulation of stimulus content in the b-CFS literature is the finding that upright faces are released from interocular suppression faster than inverted faces (e.g., Jiang et al., 2007; Stein et al., 2011a). This face inversion effect is of particular interest because it involves the comparison of stimuli with identical pixel values, differing only in their spatial orientation on the screen. This approach thus allows for ruling out many low-level stimulus confounds that could explain differences in suppression durations between image categories (Stein et al., 2012). The second type of manipulations in b-CFS paradigms consists of manipulations of stimulus context, in which conscious access of a single stimulus is assessed as a function of its relation with consciously accessible information (e.g., a target following a congruent vs. an incongruent cue). With this approach, by definition, stimuli in different conditions are identical, such that low-level stimulus contributions cannot account for any difference in suppression durations between conditions. The most reliable manipulation of stimulus context arguably consists of manipulating the concurrent content of visual working memory. Applying this manipulation revealed that visual input that matches concurrently memorized visual features (such as a color or shape) is released from interocular suppression faster than visual input that mismatches the concurrently memorized content (Gayet et al., 2013, 2016a; Pan et al., 2014; van Moorselaar et al., 2015, 2016; Gayet, 2016).

# METHODS

#### Datasets

For the present set of analyses, we used three b-CFS datasets, one from an experiment that has not yet been published, and two datasets were retrieved from published studies (Gayet et al., 2013; Gayet, 2016). The first dataset comprises of a b-CFS experiment in which RTs to upright faces were compared with RTs to inverted faces (see Appendix for a complete description of the Methods). For the sake of brevity, this dataset will be referred to as the "Face Inversion" experiment. It revealed that upright faces are released from interocular suppression faster than inverted faces. The second dataset comprises of a b-CFS experiment in which RTs to colored targets are compared between the case in which they match and the case in which they mismatch a color that is concurrently maintained in visual working memory for a subsequent recall task. Similarly, the third dataset comprises of a b-CFS experiment in which RTs to geometrical target shapes are compared between the case in which they match and the case in which they mismatch a geometrical shape that is concurrently maintained in visual working memory. These datasets will be referred to as the "Color" experiment and the "Shape" experiment respectively (see Gayet et al., 2013; Gayet, 2016, for a complete description of the Methods in these two experiments). In short, the Color and Shape Experiments revealed that identical visual stimuli are released from interocular suppression faster when they match a stimulus that is concurrently maintained in visual working memory (for a subsequent recall task). The left part of **Table 1** provides general information on these three experiments, including the number of participants, the overall mean RT and the trial-to-trial variability (see the "metrics" paragraph below).

# Metrics

In order to investigate the relation between overall RTs, trial-to-trial RT variability and the RT difference between experimental conditions, we established the following metrics. Each Participant's overall RT (RTOVERALL) was defined as the average of the median RTs in each experimental condition that elicited a RT difference (i.e., upright and inverted, or matching and mismatching conditions, but also left and right eye, different target locations, and differently colored or shaped targets). Medians were used to account for the skewness in the distributions of raw RTs, and for including trials that yielded no response within the time window (reflecting the longest suppression durations). The trial-to-trial variability was computed as the average of the SDs of each experimental condition that elicited a RT difference, hence yielding a withincondition SD (or SDWITHIN), which only reflects RT variability that is not caused by the experimental manipulations. The raw RT difference was computed by subtracting each participants' median RT to upright faces (or matching targets) from the median RT to inverted faces (or mismatching targets).

## Analysis

We computed standard Pearson correlations, which were regarded as significant in case the p-value was below the threshold of α = 0.05 divided by the number of parallel comparisons, following the Bonferroni correction. In addition, we ran onesided Bayesian correlation analyses, testing for the evidence or absence of a positive correlation (using JASP Team, 2016; Beta prior width of 1).

# RESULTS I: BREAKING CONTINUOUS FLASH SUPPRESSION

### Overall RTS

The results of the main correlational analyses are depicted in **Figure 1A**. In all three datasets reported here, participants' overall RTs correlated with the difference in raw RTs between experimental conditions. This was true for the Face Experiment, R = 0.912, p < 0.001, BF+0 = 7 ∗ 1071, for the Color Experiment, R = 0.726, p < 0.001, BF+0 = 2 ∗ 104 , and for the Shape Experiment, R = 0.687, p = 0.001, BF+0 = 38. This correlation has been reported in the b-CFS literature on a number of other occasions as well (e.g., Gayet et al., 2016a,b), and can also be observed when eyeballing individual participant's data in a number of other b-CFS studies (e.g., Jiang et al., 2007; Wang et al., 2012). Thus, this appears to be a common phenomenon in b-CFS data.

### A Simple Latency-Normalization Procedure

The present results show that, for different experimental manipulations in the b-CFS paradigm, participants with slower overall RTs also show a stronger RT difference between experimental conditions. In other words, it is possible that part of the between-subject variability in the RT difference between experimental conditions (i.e., the researcher's effect of interest) stems from between-subject variability in overall RTs (i.e., which is not related to the effect of interest). Consequently, when conducting statistical analyses to establish whether an effect of interest is present or absent, statistical power is reduced by between-subject variability that is unrelated to the effect of interest. A simple way to isolate the effect of interest from participants' overall response speed is to normalize the RT difference for each participant, such that it reflects a proportional (rather than absolute) difference in RTs caused by the experimental manipulation. This can be achieved by dividing the difference in RT between two conditions by the overall RT:

$$
\Delta RT\_{\text{NORMALIZED}} = 100 \ast \,\,\frac{RT\_A - RT\_B}{RT\_{OVERAL}}
$$

A similar approach has been proposed by Tsuchiya et al. (2009), by Stein (2012); Stein et al. (2011b, 2012), and by Gayet et al. (2016a); Gelbard-Sagiv et al. (2016).

In **Table 1**, frequentist and Bayesian test-statistics, as well as standardized effect sizes (Cohen's d) are provided for the raw RT difference and the normalized RT difference of all three datasets. This reveals that the latency-normalization procedure increased the sensitivity to detect differences between experimental conditions in all three datasets, as revealed by increased t-statistics and larger standardized effect sizes (Cohen's d) relative to the tests performed on the raw RT differences. This advantage of the normalization procedure was also observed in

FIGURE 1 | Results of the main correlational analyses in all three datasets. (A) Depicts the correlation between participants' overall RT (x-axis) and the RT difference between experimental conditions (y-axis) for all three datasets. (B) Depicts the correlation between the within-condition variability of RTs (x-axis) and the RT difference between experimental conditions (y-axis) for all three datasets. All correlations depicted here were significant after Bonferroni correction, with all p's < 0.005. Plots were adapted from JASP Team (2016) output.

\*Significant violation of normality according to the Shapiro-Wilk normality test.

Gayet et al. (2016b; Supplementary Materials). Thus, although we do not provide statistical confirmation of this improvement, the increase in test values is in line with the reasoning that removing a source of between subject variability of no interest (i.e., between subject variability in overall RTs) increases sensitivity for detecting an effect of interest (i.e., an RT difference between experimental conditions).

A second major asset of normalizing the RT differences in b-CFS experiments pertains to the normality of the distribution of RT differences. Indeed, the difference in raw RTs between experimental conditions tends to be skewed toward the tail end of the RT distribution (**Figure 2A**). For instance, in the Color experiment data, the assumption of normality is violated, according to the Shapiro-Wilk test, W = 0.848, p < 0.001.1 After the normalization procedure described above, however, this is no longer the case, W = 0.954, p = 0.155 (**Figure 2B**). Similarly, in the Shape experiment, the distribution of raw RT differences violated the assumption of normality, W = 0.774, p < 0.001, whereas the distribution of normalized RT differences did not, W = 0.951, p = 0.406. Again, in the Face Inversion experiment, the distribution of raw RT differences violated the assumption of normality W = 0.857, p < 0.001, whereas this was not (or much less) the case with the distribution of normalized RT differences, W = 0.966, p = 0.067. The distributions of the RT differences after the normalization procedure are depicted in **Figure 2B**. The reduction in the rightward tail-end of the RT difference by the latency-normalization procedure implies that this procedure reduces the amount of Type II errors (i.e., failing to detect an existing difference between conditions). The fact that the latencynormalized data more closely followed a normal distribution also implies greater control of Type I error rate compared to the skewed distribution of raw RT differences. Taken together, the latency-based normalization procedure offers an improved way to conduct parametric tests on b-CFS data. In addition, it reveals that the proportional (normalized) modulation of RTs in the b-CFS paradigm is approximately normally distributed (at least for the experimental manipulations provided here).

# A Comparison with Other Normalization Procedures

In order to evaluate the latency-normalization method, we also carried out two other transformations on our data sets that are widely applied to within-subject RT data: log-transformation and z-transformation (for a review, see Bush et al., 1993). Specifically, we aimed to investigate whether, within the b-CFS paradigm, RT differences between conditions would (1) yield larger effect sizes, and (2) more closely approximate a normal distribution, following these transformations as well. Ztransformations were obtained by dividing the raw RT difference between two conditions A and B (e.g., upright and inverted faces), by the standard deviation of the RT difference between these two conditions, within each participant:

$$
\Delta \text{RT}\_{\text{Z-TRANSFORMED}} = \frac{RT\_A - RT\_B}{SD\_{A-B}}
$$

The within-participant standard deviation for the difference in RTs between two conditions (SDA-B) was computed as follows:

SDA−B = s (NA − 1) ∗ SD2 A + (NB − 1) ∗ SD2 B (NA − 1) + (NB − 1)

where NA is the number of trials in condition A, and SDA is the standard deviation of the RTs in condition A. The logtransformed RT difference was obtained by first taking the logarithm with base ten of all RTs, and then computing the difference between the average2 log-transformed RTs of the two conditions:

$$
\Delta \text{RT}\_{\text{LOG-TRANSFORDED}} = \log\_{10}(RT\_A) - \log\_{10}(RT\_B)
$$

Across our three data sets, the three transformations applied to the RT difference increased the sensitivity to detect the difference between experimental conditions (see **Table 1**), except for the z-transformation in the Shape experiment. Overall, although we did not compare the methods statistically, the test statistics and effect sizes for the RT difference was numerically larger after applying the log- and the latencybased normalization procedures than after applying the ztransformation procedure. In addition, the RT difference more closely approximated a normal distribution after the logtransformation and latency-normalization procedure than after the z-transformation procedure. After z-transformation, the assumption of normality was still violated in 2 out of 3 data sets. On the basis of these observations, we encourage the usage of either the latency-based normalization procedure or log-transformations to b-CFS data before conducting statistical analyses, to increase sensitivity by reducing type II error-rate.

# RESULTS II: A MONOCULAR CONTROL CONDITION

#### Within-Condition RT Variability

The results of the correlation analyses between the withincondition variability in RTs (i.e., SDWITHIN) and the raw RT difference between experimental conditions is depicted in **Figure 1B**. These analyses revealed significant correlations between the within-condition variability in RTs and the raw

1 Individually, none of the three experiments of Gayet et al. (2013), which together constitute the data of the Color Experiment described here, violated the assumption of sphericity (all p's > 0.2). As such, the usage of parametric tests for these three separate experiments was allowed, although arguably suboptimal in retrospect.

2 In the Face experiment, the CFS masks were ramped down to zero contrast over the course of a trial, ensuring that observers would eventually perceive the target on each trial, allowing them to provide a response within the time limit. The color experiment had a relatively long time limit of ten seconds, so that targets were responded to within the time limit in virtually all trials in this experiment as well. Hence, for these two experiments a mean log-RT could be computed for each condition. In the Shape experiment, however, participants failed to respond within the time limit on an average of 19 trials (SD = 18) out of 144. These trials, which yielded 'infinite' suppression durations, could only be included in the analyses by using medians as an averaging metric (for an elaborate discussion on this approach, see Gayet et al., 2016b, Section 2.5).

FIGURE 2 | Distributions of the raw (A), latency-normalized (B), z-transformed (C), and Log10-transformed (D), difference in RTs between experimental conditions in all three experiments. These graphs reveal that the latency-normalization procedure had the largest impact on the RT differences of those participants that showed extremely large differences in RTs between conditions. In addition, these graphs demonstrate that after the normalization procedure, the log-transformation and (to a lesser extent) the z-transformation, the distribution of RT differences more closely followed a normal distribution. Plots were adapted from JASP Team (2016) output.

RT difference between experimental conditions in the Face Experiment, R = 0.693, p < 0.001, BF+0 = 1 ∗ 108 , in the Color Experiment, R = 0.465, p = 0.005, BF+0 = 9, and in the Shape Experiment, R = 0.625, p = 0.004, BF+0 = 13. Given that larger within-condition variability in RTs implies greater uncertainty, it is tentative to conclude that perceptual uncertainty positively correlates with the propensity of an experimental manipulation to affect RTs in a b-CFS paradigm. Arguably, the z-transformation described above increased effect sizes for RT differences by removing this source of between-subject variability from the RT difference between experimental conditions. We advocate caution in considering this interpretation, however, as it might be a spurious correlation emerging from the correlation between overall RTs and RT variability (e.g., Wagenmakers and Brown, 2007), which we also find in the Face Experiment, R = 0.797, p < 0.001, BF10 = 4 ∗ 1023, in the Color Experiment, R = 0.736, p < 0.001, BF10 = 4 ∗ 104 , and in the Shape Experiment, R = 0.659, p = 0.002, BF10 = 23.

#### Sources of RT Variability

It should be made explicit that we do not know what part of the between-subject variability in overall (i.e., median) RTs stems from individual differences in suppression durations, and what part stems from individual differences in response speed after the interocular conflict is resolved. Similarly, we do not know what part of the within-subject variability in RTs (i.e., the withincondition SD) stems from trial-to-trial differences in suppression durations, and what part stems from trial-to-trial differences in response speed after the interocular conflict is resolved. Finally, it is debated whether it is possible to unequivocally separate the contribution of these two processes that constitute b-CFS RTs (e.g., Stein et al., 2011a; Stein and Sterzer, 2014).

In order to separate the two, we capitalized on the fact that, in the Face Inversion experiment, the same participants also performed a control condition. In b-CFS control conditions, similar stimuli as in the CFS condition are used but no interocular suppression is induced. Hence, RTs in these control conditions offer insights into processes that take place after the interocular conflict is resolved. In these control conditions, both the CFS masks and the target stimuli are presented to both eyes (binocular control condition), or the CFS mask is presented to the same eye as the target stimulus, with the target stimulus displayed on top of the mask (monocular control condition). However, these control conditions typically yield null results, even when comparing stimuli that are well known to yield detection differences in other non-CFS detection paradigms, such as upright vs. inverted faces. The present Face Inversion experiment therefore adopted an improved control condition, in which face targets were blended into phase-scrambled face images (i.e., phase-scrambled noise, see Appendix for details). Face targets and phase-scrambled noise were presented to one eye, while the other eye was presented with the gray background only, such that no interocular suppression was induced.

In the Face Inversion experiment, the advantage for upright faces over inverted faces was also observed in the control condition but was smaller (9 vs. 26%) than in the CFS condition, t(64) = 8.999, p < 0.001, BF10 = 3 ∗ 1010. Overall RTs, in contrast, did not significantly differ between the CFS condition and the control condition, t(64) = 1.506, p = 0.137, BF01 = 3. These findings, summarized in **Table 2**, show that the upright face advantage is more pronounced in the CFS condition than in the control condition, even though overall RTs were successfully matched between the two (for a discussion on why this is important, see Stein et al., 2011a). Taken together, upright faces are detected faster than inverted faces when there is no interocular suppression (e.g., after the interocular competition is resolved) but, the larger RT difference in the CFS condition suggests that, on top of this, upright faces are also released from interocular suppression faster than inverted faces.

Similar to the CFS condition, the RT difference between upright and inverted faces in the control condition correlated positively with the overall response speed, R = 0.320, p = 0.009, BF+0 = 9, and with the within-condition SD, R = 0.306, p = 0.013, BF+0 = 6. These correlations (depicted in **Figures 3A,B** respectively) were less reliable and smaller in magnitude, however, than those observed in the CFS condition.

The within-condition SD (reflecting trial-to-trial variability) was much smaller in the control condition (M = 292 ms, SD = 143) than in the CFS condition (M = 1,298 ms, SD = 835), t(64) = 10.373, p < 0.001, BF10 = 8 ∗ 1013. This observation is not surprising, considering that, in the CFS condition, our metric of within-condition variability captured both variability in suppression duration and variability in response speed after resolution of the interocular conflict, whereas in the control condition (variability in) suppression durations did not contribute to within-condition variability. The large difference in magnitude between the within-condition variability observed with and without interocular competition, however, indicates that the within-condition SD observed in the CFS condition mostly reflected trial-to-trial variability in suppression durations. In line with this view, the raw difference in RT between upright and inverted faces in the CFS condition did not correlate with

\*Significant violation of normality according to the Shapiro-Wilk normality test (none observed).

the within-condition variability in the control condition, R = 0.016, p = 0.902, BF01 = 6 (see **Figure 3C**). This observation provides additional grounds for the notion that the relation between RT differences and trial-to-trial variability in RTs in the CFS condition (reported in the previous section) is primarily driven by trial-to-trial variability in suppression durations.

# GENERAL DISCUSSION

#### Brief Summary

Using three representative b-CFS datasets, we established that longer RTs relate to larger RTs differences between experimental conditions in the b-CFS paradigm. This may be considered a nuisance, because part of the between-subject variance that will affect statistical testing for a difference between experimental conditions (i.e., the effect of interest), does not reflect individual differences in the effect of interest, but rather reflects individual differences in overall RTs. Usually, these individual differences in overall RTs that are not specific to the experimental conditions, are of no interest to the research question. This relation between overall RTs and RT differences is probably inherent to RT data, and therefore not specific to the b-CFS paradigm (i.e., it might be a general law, following from the increase in RT variability that accompanies an increase in RT; Wagenmakers and Brown, 2007). It is of particular importance in the b-CFS paradigm, however, for two main reasons. First, in the b-CFS paradigm, overall RTs reflect both (1) the duration of interocular suppression and (2) the response speed to the stimulus after the interocular conflict is resolved. Both processes could independently elicit longer overall RT latencies and thus larger RT differences between experimental conditions than in experiments not involving CFS. Second, b-CFS is believed to be a highly potent measure to discern differences between experimental conditions because it yields large raw RT differences between conditions, with relatively few trials (e.g., compare the face inversion effect of Stein et al., 2011a, using b-CFS, with that observed in traditional detection paradigms reviewed in Lewis and Edmonds, 2003). At the same time, however, artificially stretching out RT differences between conditions, for example by presenting target stimuli at low contrast or by increasing depth of suppression through high mask contrast, also causes noise to be stretched out, allowing for more Type I errors (i.e., false positives) of a larger magnitude. For these reasons, it should become standard procedure in b-CFS experiments that examine differences between experimental conditions (the effect of interest) to investigate whether the effect of interest correlates with participants' mean RTs, and remove this between-subject variability of non-interest from the effect of interest.

# A Simple Solution

We encourage the usage of a simple normalization method (for similar approaches, see Tsuchiya et al., 2009; Stein et al., 2011b, 2012; Stein, 2012; Gayet et al., 2016a,b) to remove the influence of participants' overall RTs from the effect of interest. Using this method provides three clear benefits. First, removal of uninteresting between-subject variability provides a more precise measure of the effect of interest. Indeed, across all three data sets, test-statistics and effect sizes were improved after applying this normalization method to the RT differences. Second, the distribution of normalized RT difference approximates normality to a greater extent than the distribution of raw RT differences. This allows for conducting parametric tests, by transforming the data in a way that is tailored to the specifics of a participant's RT distribution, rather than being a generic approximation (e.g., such as with log-transformations). Third, normalized RT differences provide an intuitive measure of how an experimental manipulation affected participants' behavior, in the form of a proportional difference (reduction) in RTs caused by the experimental manipulation. This is an advantage over, for instance, log-transformed or z-transformed RT differences.

Log-transformed RT differences yielded increases in effect size, and approximations of normality, that were comparable to that of normalized RT differences. Therefore, log-transformations are an equally good solution for dealing with non-normal RT data with large between subject variability in the b-CFS paradigm. Z-transformations, on the other hand, were predominantly less successful in generating normally distributed RT differences.

# Suppression Durations

As we argued above, RTs in a b-CFS paradigm comprise both the duration of interocular suppression, and the response speed to the stimulus after the interocular conflict is resolved. Similarly, the individual differences in (trial-to-trial variability in) RTs that we analyzed in the present study reflect variability in suppression durations, as well as variability in response speed after the interocular conflict is resolved. We suggest that the relation between individual differences in RTs and individual differences in the effect of interest, which we report here, are largely specific to CFS, reflecting individual differences in suppression durations. This notion is supported by the finding that RT differences in conditions that include interocular suppression (the CFS condition) do not correlate with individual differences in RTs drawn from conditions that do not include interocular suppression (i.e., a control condition). Another argument stems from the magnitude of the effects of interest and the magnitude of individual differences in RTs (i.e., in interocular suppression conditions). These are much larger than those typically observed in non-suppression conditions, suggesting that suppression durations constitute a large portion of the eventual RTs and RT differences. Taken together, these observations suggest that individual differences in overall RTs primarily reflect individual differences in suppression durations.

# The Role of Response Bias

In this study, we observed that trial-to-trial variability was compellingly larger in the b-CFS suppression condition compared to the b-CFS control condition. This is potentially problematic, as the absence (or reduction) of an experimental effect in b-CFS control conditions is typically interpreted as evidence that an experimental effect in the CFS condition was not caused by anything happening after the interocular conflict was resolved, and thus reflects a difference in suppression duration. The difference in trial-to-trial variability between suppression and control conditions, however, could be associated with differences in perceptual uncertainty, such that response criteria might differ between suppression and control conditions. From this perspective, even the absence of an effect in a b-CFS control condition, does not necessarily preclude that an effect of interest in the b-CFS suppression condition was caused by a difference in response criteria. For example, more familiar stimuli, such as upright faces, might be more readily responded to in a state of perceptual uncertainty than less familiar stimuli, such as inverted faces (Stein et al., 2011a). In the control conditions, however, the lower levels of uncertainty could still be insufficient for the different response criteria between these conditions to modulate RTs. Future b-CFS studies could rule out the potential influence of such response biases by using non-speeded, signal detection-based bias-free protocols in which presentation times are fixed (e.g., Kaunitz et al., 2013; Lupyan and Ward, 2013; Hedger et al., 2015).

# The Role of Perceptual Transitions

In b-CFS, an initially fully invisible stimulus eventually becomes fully visible. However, this emergence into conscious awareness is not an abrupt, all-or-none phenomenon, but a gradual one (Stein et al., 2011a), and some stimulus properties become available to consciousness earlier than others. For example, recent observations have shown that, under conditions of prolonged CFS, periods of partial awareness might arise, in which some stimulus properties (such as color) are available to consciousness, whereas other are not (such as orientations; Zadbood et al., 2011; Yang and Blake, 2012). Thus, prolonged periods of continuous flash suppression might enable distinct stages of suppression strength that vary in the degree of susceptibility to experimental manipulations. This is in line with the idea that interocular competition is modulated at different levels throughout the visual processing hierarchy (Blake and Logothetis, 2002). A recent study showed that stimuli under CFS elicited high-level behavioral priming effects only under conditions of partial awareness, but not when fully suppressed (Gelbard-Sagiv et al., 2016). Similarly, a recent study demonstrated that manipulations of attention only affected dominance durations in a binocular rivalry paradigm around the time of perceptual transitions (Dieter et al., 2015). Another binocular rivalry study showed that the detection performance of monocular probes followed gradual changes in consciousness, rather than being dichotomous (Alais et al., 2010). Thus, variations in suppression strength, as are likely to occur over the course of a b-CFS (or binocular rivalry) trial, might enable different processes to modulate suppression durations of initially fully suppressed visual input.

Both fluctuations in suppression strength during individual trials, as well as the duration of perceptual transitions may vary between observers. It is thus conceivable that our current metrics (the trial-to-trial variability in RTs, and the overall RTs) relate to the duration of transitory percepts—with limited depth of suppression—in individual participants. Accordingly, certain experimental manipulations should exert more influence on the RTs of participants with longer transitory percepts (or longer periods of shallow suppression) than on the RTs of participants with shorter transitory percepts (or shorter periods of shallow suppression). Thus, longer periods of shallow suppression may allow for a greater extent of non-conscious processing, such that larger differences in detection times can be expected for participants with longer periods of shallow suppression. The correlation between longer overall RTs (and larger trial-totrial variability) and the RT difference between experimental conditions observed in this study fits well with this idea.

From a neural perspective, different levels of suppression might allow for neural responses to non-conscious stimuli to travel up to different levels of the (visual) processing hierarchy, thereby enabling more cognitive functions (e.g., Grill-Spector et al., 2000; Bar et al., 2001; Kanwisher, 2001; Supèr et al., 2001; Sergent and Dehaene, 2004). This leads to the expectation that, with manipulations requiring increasing cognitive demands, the relation between our current metrics and the modulation of RTs by the experimental manipulations should become more pronounced. The general consensus was, traditionally, that interocularly suppressed information cannot be processed at a semantic or conceptual level (e.g., Zimba and Blake, 1983; for a review, see Lin and He, 2009). This view stemmed from experiments using such variants of interocular suppression as binocular rivalry (Wheatstone, 1838; Alais and Blake, 2005) and flash suppression (Wolfe, 1984). In contrast, b-CFS studies have yielded contradictory findings, with some studies demonstrating that semantic or conceptual information can drive conscious access of initially suppressed visual input (e.g., Mudrik et al., 2011; Yang and Yeh, 2011; Sklar et al., 2012) while other studies have challenged this view (e.g., Heyman and Moors, 2014; Moors et al., 2016; Rabovsky et al., 2016; Stein et al., in press). Investigating the point in time (or the suppressive strength) at which a manipulation impacts conscious access in a b-CFS paradigm might be a valuable tool to resolve this apparent conflict.

#### ETHICS STATEMENT

All data analyzed in this study was retrieved from previously published or unpublished studies. Hence, despite human participant data was used in this study, it was already collected before this study was initiated. Hence, ethical statements

#### REFERENCES

are to be found in the articles from which the data was retrieved.

## AUTHOR CONTRIBUTIONS

The study idea was developed by SG and worked out in more detail by SG and TS. Data collection and analyses were conducted by SG and TS. The manuscript was initially written by SG, and was later improved by comments and suggestions provided by TS. All authors approved the final version of the manuscript.

#### ACKNOWLEDGMENTS

The authors want to thank Pieter Moors for useful comments and insights provided during review of an earlier version of this manuscript.

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fpsyg. 2017.00437/full#supplementary-material

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Gayet and Stein. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Neural Correlates of Subjective Awareness for Natural Scene Categorization of Color Photographs and Line-Drawings

Qiufang Fu1,2 \*, Yong-Jin Liu3 , Zoltan Dienes4 , Jianhui Wu1,5, Wenfeng Chen1 and Xiaolan Fu1,2

1 State Key Laboratory of Brain and Cognitive Science, Institute of Psychology, Chinese Academy of Sciences, Beijing, China, 2 University of Chinese Academy of Sciences, Beijing, China, 3 Tsinghua National Laboratory for Information Science and Technology, Department of Computer Science and Technology, Tsinghua University, Beijing, China, 4 Sackler Centre for Consciousness Science, School of Psychology, University of Sussex, Brighton, UK, 5 College of Psychology and Sociology, Shenzhen University, Shenzhen, China

It remains controversial whether visual awareness is correlated with early activation indicated by VAN (visual awareness negativity), as the recurrent process hypothesis theory proposes, or with later activation indicated by P3 or LP (late positive), as suggested by global workspace theories. To address this issue, a backward masking task was adopted, in which participants were first asked to categorize natural scenes of color photographs and line-drawings and then to rate the clarity of their visual experience on a Perceptual Awareness Scale (PAS). The interstimulus interval between the scene and the mask was manipulated. The behavioral results showed that categorization accuracy increased with PAS ratings for both color photographs and line-drawings, with no difference in accuracy between the two types of images for each rating, indicating that the experience rating reflected visibility. Importantly, the event-related potential (ERP) results revealed that for correct trials, the early posterior N1 and anterior P2 components changed with the PAS ratings for color photographs, but did not vary with the PAS ratings for line-drawings, indicating that the N1 and P2 do not always correlate with subjective visual awareness. Moreover, for both types of images, the anterior N2 and posterior VAN changed with the PAS ratings in a linear way, while the LP changed with the PAS ratings in a non-linear way, suggesting that these components relate to different types of subjective awareness. The results reconcile the apparently contradictory predictions of different theories and help to resolve the current debate on neural correlates of visual awareness.

Keywords: visual awareness, subjective measures, objective measures, natural scene categorization, Perceptual Awareness Scale (PAS)

# INTRODUCTION

Recently, an increasing amount of attention has been given to neural correlates of visual awareness, i.e., the subjective experience of seeing. Neural imaging studies have revealed that a distributed system, including the primary visual cortex (V1), V2, temporal, parietal, and frontal areas, plays a crucial role in the generation of visual awareness (see Koch et al., 2016 for a review; Rees, 2001; Vuilleumier et al., 2002; Salminen-Vaparanta et al., 2012). Electrophysiological studies have shown

#### Edited by:

Guido Hesselmann, Charité – Universitätsmedizin Berlin, Germany

#### Reviewed by:

Zhicheng Lin, Ohio State University, USA Liad Mudrik, Tel Aviv University, Israel

> \*Correspondence: Qiufang Fu<EMAIL_ADDRESS>
#### Specialty section:

This article was submitted to Consciousness Research, a section of the journal Frontiers in Psychology

Received: 11 August 2016 Accepted: 01 February 2017 Published: 15 February 2017

#### Citation:

Fu Q, Liu Y - J, Dienes Z, Wu J, Chen W and Fu X (2017) Neural Correlates of Subjective Awareness for Natural Scene Categorization of Color Photographs and Line-Drawings. Front. Psychol. 8:210. doi: 10.3389/fpsyg.2017.00210

that an early positive enhancement (P1) over occipital regions that were approximately 100 ms from the stimulus onset, a negative difference wave (visual awareness negativity, VAN) at the posterior regions typically occurring 150–250 ms from the stimulus onset, and a P3 or a late positive (LP) wave occurring 300 ms from the stimulus onset were probably correlated with the emergence of visual awareness (see Koivisto and Revonsuo, 2010; Revonsuo and Koivisto, 2010; Naccache et al., 2016; contrast Silverstein et al., 2015). However, there is no agreement on what the identity of the neural indicator is for the emergence of visual awareness. For example, the N1 component (e.g., Jiang et al., 2013), the P2 component (e.g., Melloni et al., 2011), and the N2 component (e.g., Koivisto and Revonsuo, 2003) have been also argued to be related to visual awareness. The identity of the neural indicator for the emergence of visual awareness is still unclear.

One of the challenges of addressing this issue is the measurement of the status of visual awareness. There are two measures of awareness or consciousness in the literature: objective and subjective measures. With objective measures, the ability to discriminate features of the world is taken to indicate whether knowledge is conscious (Fu et al., 2008; Koivisto et al., 2008). An inability to discriminate reflects the absence of conscious knowledge. Using objective measures, P1 and VAN might be found to be related to visual awareness. For example, in a two-alternative matching to sample task, participants were asked to determine whether a briefly presented "sample word" was the same or different from a second "test word" by pressing one of two keys. Melloni et al. (2007) found that a P1 like component was significantly different between visible and invisible words, indicating that the P1 component was the earliest event-related potential (ERP) component that correlated with visual awareness. Pins and Ffytche (2003) also found a P1 effect for visual awareness, but this P1 correlate was not been observed in other studies (e.g., Sergent et al., 2005; Koivisto et al., 2008; Melloni et al., 2011). Moreover, Koivisto et al. (2008) compared the neural activities for hit trials (aware) with those for miss trials (unaware) and found that the posterior VAN was an earlier ERP correlate of visual awareness. The posterior VAN correlate has been observed in other studies (e.g., Koivisto and Revonsuo, 2003; Genetti et al., 2009).

With subjective measures, the ability to report or discriminate mental states is taken to indicate whether the knowledge is conscious (Fu et al., 2008; Lamy et al., 2009). If a person says he does not see or he has no experience of the stimulus, the knowledge revealed in the behavioral forced-choice response is unconscious by these measures. Using subjective measures, P3 or LP might be found to be related to visual awareness. For example, Lamy et al. (2009) compared the neural activities for correct trials in which participants reported they had seen the stimulus (aware-correct) and correct trials in which participants reported that they merely guessed (unawarecorrect). They found that only an enhanced P3 effect with a widely distributed topography was related to subjective awareness. Similarly, Del Cul et al. (2007) also found that only the P3 amplitude distinguished visible and invisible stimuli in subjective reports. Moreover, although the studies using objective measures found VAN to be related to visual awareness, most of them also found P3 or LP (later positive) effects in visual awareness (e.g., Koivisto et al., 2008; Genetti et al., 2009).

Obviously, different measures lead to different findings in the neural correlates of visual awareness, but more importantly, they are based on different theories of consciousness. For example, subjective measures are consistent with the assumption of the higher-order thought (HOT) theory that states that a distinguishable characteristic of a conscious state is that an individual can report the state (Rosenthal, 2005; Lau and Rosenthal, 2011). According to the HOT theory, the conscious status of a mental state is most naturally measured by subjective measures. Meanwhile, the global workspace (GWS) theory takes the dynamic long-distance synchronization as a key feature for conscious condition (Dehaene and Naccache, 2001). According to the GWS theory, objective measures could be satisfied by local processors without information entering the workspace. Information in the workspace would be available to higher order thoughts, so the GWS theory naturally motivates subjective measures. In this respect, the HOT and GWS theory are similar. In contrast, a recent recurrent processing hypothesis states that localized recurrent processing is sufficient for consciousness, and while not entirely obvious theoretically, this approach has been used to motivate objective measures (Lamme, 2010). But most generally, the local recurrent processing approach attempts to move away from behavioral or introspective methods in the measure of consciousness.

The widely distributed P3 or LP correlate of visual awareness is in line with the prediction of the GWS theory, which assumes that visual awareness correlates with later activation in the frontoparietal cortices; the more local posterior VAN correlate of visual awareness is in line with the recurrent processing hypothesis, which proposes that visual awareness correlates with early activation in the visual cortex (Windey et al., 2013; Andersen et al., 2015; Koivisto and Grassini, 2016). In addition, the GWS theory assumes that visual awareness is dichotomous, while the recurrent processing hypothesis assumes that visual awareness is graded (Windey et al., 2013; Andersen et al., 2015; Koivisto and Grassini, 2016). It remains controversial whether visual awareness is correlated with early activation indicated by VAN, as the recurrent process hypothesis theory proposes, or with later activation indicated by P3 or LP, as suggested by the GWS and HOT theories, and whether the neural indicator is correlated with visual awareness in a dichotomous or graded way.

Interestingly, Timmermans et al. (2010) argued that studies supporting the GWS theory generally use complex stimuli (e.g., Del Cul et al., 2007, 2009), whereas studies supporting the recurrent processing hypothesis mostly use simple stimuli (e.g., Fahrenfort et al., 2007). Moreover, Windey et al. (2013) found that simple task produced evidence for graded visual awareness, whereas more complex task produced more dichotomous visual awareness. Thus, the findings about the neural correlates of visual awareness could be influenced by stimulus or task complexity. To the best of our knowledge, no study has adopted complex stimuli such as natural scenes to explore neural correlates of visual awareness. Natural scenes are rather complex, but people

can categorize them quickly and accurately, even with little or no attention (Li et al., 2002; Rousselet et al., 2002; Feifei et al., 2005; Otsuka and Kawaguchi, 2007), and it has been claimed that the integration of scene elements can be achieved without awareness (Mudrik et al., 2011; Mudrik and Koch, 2013), although there is inconsistent evidence for this (Faivre and Koch, 2014; Moors et al., 2016). Moreover, recently, a fMRI study found that natural scene categorization of line-drawings generated similar neural activation as color photographs, although color photographs includes more features such as color and contexts than linedrawings (Walther et al., 2011).

Therefore, to further examine what are the neural correlates of visual awareness, we adopted natural scenes of both color photographs and line-drawings in a backward masking task. In the backward masking task, the stimulus duration was identical, i.e., the physical stimulus was identical, and the interstimulus interval (ISI) between the stimulus and the mask was manipulated. It has been demonstrated that masking has little effect on the feedforward processing of a stimulus from low to high visual areas, but interferes with the recurrent processing of the stimulus from high to low visual areas that is crucial for the emergence of visual awareness (Rolls et al., 1999; Lamme et al., 2002; Lamme, 2010). To measure the status of visual awareness, we used subjective measures because they are motivated by two major psychological approaches to consciousness, that is, higher order and GWS approaches. Specifically, we used the Perceptual Awareness Scale (PAS), in which participants were asked to rate the clarity of their visual experience using a 4-point scale with the elements: no experience, brief glimpse, almost clear image, and clear image (Ramsøy and Overgaard, 2004; Sandberg et al., 2010; see also Dienes and Seth, 2010). The interpretation of PAS depends on its theoretical relation to awareness of the relevant stimulus characteristics. On one view, each PAS rating represents an increase in visual awareness in a graded way (e.g., Ramsøy and Overgaard, 2004). On another view, the distinction between, on the one hand, brief glimpse, in which the subject claims not to have seen anything of relevance, and almost clear image, on the other hand, marks a distinct change in the conscious awareness of information relevant to the task discrimination (Dienes and Seth, 2010). On the first view, a correlate of awareness should vary linearly with the PAS ratings; on the second view, relevant components should vary non-linearly with the PAS ratings. Further, if subjective visual awareness correlates with early activation in the visual cortex, the posterior VAN should vary with the PAS ratings in an appropriate way; or else, if long-distance activation among brain areas is additionally required for the emergence of subjective awareness, a widely distributed P3 or LP would vary with the PAS ratings in an appropriate way.

# MATERIALS AND METHODS

# Participants

Twenty-two college students (11 males and 11 females; mean age = 21.82 years, SD = 2.34 years) participated in this study. All of them had normal or corrected-to-normal vision, and none of them had any history of neurological or psychiatric diseases. This experiment was approved by the committee for the protection of subjects at the Institute of Psychology, Chinese Academy of Sciences. All students gave written informed consent and were paid for their attendance.

# Stimuli and Apparatus

The stimuli were images of six natural scene categories: beaches, city streets, forests, highways, mountains, and offices (see **Figure 1**). Each image had a resolution of 320 pixels × 240 pixels in two versions: color photographs and line-drawings. The linedrawings were produced by trained artists by tracing contours in the color photographs (see Walther et al., 2011). Each category had 76 to 80 different images, for a total of 475. The masks were two white noise images at two different spatial scales: one was generated at the resolutions of 320/240, and the other was generated at the resolutions of 20/15 and then resized to 320/240 pixels. Each mask was also resized to 320 pixels × 240 pixels in two versions: one was in color and the other was in grayscale. The stimulus was presented against a silver gray background on a CRT monitor with a resolution of 1280 pixels × 768 pixels and a refresh rate of 75 Hz. The distance from the participants to the screen was approximately 60 cm, and the horizontal and vertical visual angles were approximately 8.7 and 8.2 degrees, respectively.

# Procedure

Each trial began with a black fixation cross at the center of the screen for 500–950 ms at random. Then, an image was presented for 13 ms, and followed by two masks for 100 ms. Each mask was shown for 50 ms and the sequence of the two masks was randomly assigned. For color photographs, the masks were in color, and for line-drawings, the masks were in grayscale. The ISI between the image and the mask was 0, 13, 26, or 200 ms at random. After the masks, a blank was presented for 500 ms and then six category names appeared on the screen from left to right. Participants were asked to report the category of the presented image by pressing a corresponding key on the keyboard. The keys D, F, G, H, J, and K corresponded to the locations of the six category names. To prevent participants from preparing for their response before the appearance of the category names, the locations of the six category names were randomly assigned for each trial. There was no feedback about the response. After the response, they were asked to rate their experience of the stimulus, i.e., how clearly did they see the image, with four possible options from left to right on the PAS scale [(1) no experience; (2) brief glimpse; (3) almost clear experience; (4) clear experience] by pressing a corresponding key. Participants were told to select the option according to their true feelings and then to press the space key to start the next trial when they were ready. There were approximately 95 trials in each block, in which type of image (color photographs vs. line-drawings), natural scene category, and ISI were counterbalanced. There were at least 30 s of rest between any two blocks. The experiment included 10 blocks, for a total of 950 trials.

# EEG Recording and Analysis

Electroencephalograph (EEG) was recorded with a NeuroScan Synamps amplifier using a custom 64-channel cap. The left mastoid was used as an on-line reference and the algebraic average of left and right mastoids was used as off-line rereference. Two pairs of electrodes placed 1 cm above and below one eye and 1 cm lateral from the outer canthi of both eyes were used to monitor blinks and other eye movements. The EEG signals were amplified by using a band pass of 0.05 to100 Hz, with a sampling rate of 500 Hz. Electrode impedances were kept below 5 k. For the analysis of ERPs, the continuous EEG signals were low pass filtered, with a cutoff frequency at 30 Hz, and then segmented in series of epochs of 800 ms. Each epoch started 100 ms before the stimulus onset. Baseline correction was performed over the 100 ms window before the stimulus presentation. Trials containing voltages exceeding ± 80 µV were rejected. The average rejection rate was only 2.17%.

ERPs for correct trials were averaged separately for the different experience ratings of color photographs and linedrawings. Based on previous studies (for review, see Koivisto and Revonsuo, 2010; Melloni et al., 2011), the mean amplitudes of the ERPs were analyzed for P1 (100–130 ms), N1 (140–200 ms), P2 (150–190 ms), N2 (210–310 ms), VAN (240–300 ms), P3a (370– 470 ms), and LP or P3 (420–570 ms). The time windows for the statistical analysis of the ERPs were determined on the basis of the grand average waves. The mean amplitudes were computed over groups of electrodes representative of the topography of each component. For P1, N1, and VAN, a group of occipital electrodes (CB1, O1, Oz, O2, and CB2) was selected; for P2, N2, and P3a, a group of fronto-central electrodes (F3, Fz, F4, FC3, FCz, FC4, C3, Cz, and C4) was selected; for LP, a group of occipito-parietal electrodes (P3, Pz, P4, PO3, POz, PO4, O1, Oz, and O2) was selected. Each was subjected to repeated two-way ANOVA with the type of image (color photographs vs. line-drawings) and experience ratings (no experience vs. brief glimpse vs. almost clear experience vs. clear experience) as within-subject variables.

#### RESULTS

#### Inferential Strategy

fpsyg-08-00210 February 13, 2017 Time: 14:18 # 5

For all tests p-values are reported; in addition, for all one degree of freedom tests, Bayes factors, B, are reported. Bayes factors (B) were used to assess strength of evidence for H1 relative to H0 (Wagenmakers et al., 2015). Unlike null-hypothesis significance testing, Bayes factors have the advantage of distinguishing sensitive evidence for H0 from not much evidence at all (relative to the range of effects that could plausibly be obtained on H1). A B of above 3 indicates evidence for the alternative hypothesis and below 1/3 evidence for the null hypothesis. Bs between 3 and 1/3 indicate data insensitivity in distinguishing H0 and H1 (see Dienes, 2014; cf Jeffreys, 1939). Thus we will report that there was no effect only when B < 1/3, that is, when there is evidence for H0 over H1. Here, BN(0,x) refers to a Bayes factor in which the predictions of H1 were modeled as normal distribution with an SD of x (see Dienes, 2014), where x scales the size of effect that could be expected. If a rough maximum effect can be determined, x can be set as half that value (Dienes, 2014).

For proportions of different PAS ratings, with four ratings the average proportion was 0.25, so we report BN(0,0.25) . For accuracy of classification, as the range from baseline (i.e., chance level 0.17) to 1 was 0.83, a difference of 0.83 is the maximum difference that could be expected between any two conditions. Thus, we took half of 0.83 as the SD and report BN(0,0.42) . For ERP results, we took half of the roughly average difference between maximal and minimal amplitude of the ERP component over color photographs and line-drawings as x for each component. Note that the difference between maximal and minimal amplitude was never itself tested, so there is no double counting (Jeffreys, 1939). As positive trend coefficients summed to 1 and negative to −1, the expected size of linear trends is the same as a simple difference. Thus, for trend contrasts, H1 was modeled as a normal with an SD equal to half of the roughly average difference between maximal and minimal amplitudes of the ERP component, as just mentioned. With these models of H1, it so happened that whenever p < 0.05, then B > 3, and vice versa. There is no guarantee of this correspondence (Lindley, 1957), but in the current case significance testing and Bayesian analyses agreed when indicating that there was an effect.

#### Behavioral Results

As in previous studies (e.g., Ramsøy and Overgaard, 2004; Del Cul et al., 2007; Railo et al., 2015), we treated the experience rating as a within-subject factor. **Figure 2** shows the proportion and accuracy for each experience rating for both types of images. Repeated ANOVA on proportions, with experience ratings and type of image as within-subject variables, revealed a significant effect of experience ratings, F(1.62,33.97) = 4.34, p < 0.01, η 2 p = 0.17, and a significant experience ratings by type of image interaction, F(3,63) = 22.55, p < 0.001, η 2 p = 0.52. People reported having a "brief glimpse" less often for color photographs than for line-drawings, t(21) = −7.37, p < 0.001, dz = 1.61, BN(0,0.25) = 7.13 × 1010; but reported having a "clear experience" more often for color photographs than for line-drawings, t(21) = 6.69, p < 0.001, dz = 1.46, BN(0,0.25) = 9.07 × 108 . There were no differences between color photographs and linedrawings in the proportion of time they reported having "no experience," t(21) = −0.03, p = 0.98, BN(0,0.25) = 0.08, nor in the proportion of "almost clear experience" reports, t(21) = −1.43, p = 0.17, BN(0,0.25) = 0.32. Overall, the above results provided substantial evidence that people had subjective experiences with more felt clarity of color photographs than of line-drawings.

Repeated ANOVA on accuracy, with experience ratings and type of image as within-subject variables, revealed only a significant effect of experience ratings, F(1.83,38.86) = 299.49, p < 0.001, η 2 p = 0.93. For both types of images, the accuracy was higher for a "clear image" than for an "almost clear image," t(21) = 4.81, p < 0.001, dz = 1.05, BN(0,0.42) = 3.66 × 1020 , higher for an "almost clear image" than for a "brief glimpse," t(21) = 13.05, p < 0.001, dz = 2.85, BN(0,0.42) = 1.58 × 1035 , and higher for a "brief glimpse" than for "no experience," t(21) = 9.90, p < 0.001, dz = 2.16, BN(0,0.42) = 1.47 × 104 . There was no effect of type of image, F(1,21) = 0.09, p = 0.77, η 2 p = 0.004, BN(0,0.42) = 0.04. The two-way interaction did not reach significance, F(3,63) = 2.17, p = 0.10, η 2 p = 0.09. For both color photographs and line-drawings, the accuracy was above chance (0.17) for "no experience," t(21) = 4.11, p < 0.001, dz = 0.88, BN(0,0.42) = 2.85 × 1058 , t(21) = 3.56, p < 0.01, dz = 0.76, BN(0,0.42) = 3.51 × 1026. That is, people could correctly classify natural scenes even when they reported that they did not see them at all.

#### ERP Results

Inspection of ERPs revealed seven major components: P1, N1, P2, N2, VAN, P3a, and LP (or P3). **Figure 3** shows the ERP data for each condition. **Figure 4** shows the mean amplitudes of each component for each condition. To evaluate which components were correlated with subjective awareness, repeated two-way ANOVA with experience ratings (no experience vs. brief glimpse vs. almost clear experience vs. clear experience) and type of images (color photographs vs. line-drawings) as within-subject variables was conducted for the mean amplitude of each component. Moreover, as "brief glimpse" involves some visual experience, but not of anything relevant to making the task discrimination, the difference between "brief glimpse" and "almost clear image" is when perceptual contents go from largely unconscious and to largely conscious. Even on a graded consciousness view (Ramsøy and Overgaard, 2004), this particular contrast represents a distinction in amount of conscious awareness. Thus, to further examine whether each component was a real marker of the conscious status of knowledge of a stimulus, the planned comparison between "brief glimpse" and "almost clear image" were conducted for the mean amplitude of each component.

**Table 1** summarizes significant results of the ANOVAs. As observed from **Table 1**, for posterior P1 amplitudes, the effect of

the experience ratings and interaction did not reach significance (ps > 0.31). The planned comparison revealed that there was evidence for no difference between "brief glimpse" and "almost clear experience" for color photographs, t(21) = −1.24, p = 0.23, dz = 0.26, BN(0,3) = 0.26, and marginal evidence that there was no difference for line-drawings, t(21) = 1.46, p = 0.16, dz = 0.31, BN(0,3) = 0.35. That is, the posterior P1 amplitude did not always relate to subjective awareness.

For posterior N1 and anterior P2 amplitudes, the twoway ANOVAs revealed significant experience rating effects which were modulated by the type of image. Further analysis revealed that for color photographs only, the amplitudes of posterior N1 and anterior P2 significantly varied with visual awareness, F(1.86,39.08) = 23.02, p < 0.001, η 2 p = 0.52, F(2.09,43.81) = 10.54, p < 0.001, η 2 p = 0.33, but did not for line-drawings (ps > 0.12). For N1, the planned comparison revealed that "brief glimpse" differed from "almost clear experience" for color photographs, t(21) = −5.85, p < 0.001, dz = 1.25, BN(0,3) = 4.46 × 106 , but not for linedrawings, t(21) = −0.81, p = 0.43, dz = 0.17, BN(0,3) = 0.18. For P2, the planned comparison revealed that there was no evidence for whether or not "brief glimpse" differed from "almost clear experience" for color photographs, t(21) = 1.14, p = 0.27, dz = 0.24, BN(0,4) = 0.47, but there was substantial evidence for no difference for line-drawings, t(21) = −0.07, p = 0.94, dz = 0.01, BN(0,4) = 0.08. Thus, the results indicated that the posterior N1 and anterior P2 amplitudes did not always correlate with subjective awareness.

For the later components N2, VAN, P3a, and LP, the twoway ANOVAs revealed significant experience rating effects (all ps < 0.01), which were not significantly influenced by the type of images (all ps > 0.19). For N2, the planned comparison revealed that "brief glimpse" differed from "almost clear experience" for both color photographs, t(21) = −4.03, p = 0.001, dz = 0.86, BN(0,4) = 8.20 × 102 , and line-drawings, t(21) = −3.42, p < 0.01, dz = 0.73, BN(0,4) = 69.39. For VAN, "brief glimpse" differed from "almost clear experience" for both color photographs, t(21) = 3.12, p < 0.01, dz = 0.67, BN(0,4) = 23.89, and line-drawings, t(21) = 4.06, p < 0.001, dz = 0.87, BN(0,4) = 2.35 × 103 . However, for P3a, there was not much evidence for the difference between "brief glimpse" and "almost clear experience" either way for either color photographs, t(21) = 0.84, p = 0.41, dz = 0.18, BN(0,4) = 0.35, or linedrawings, t(21) = 1.08, p = 0.29, dz = 0.23, BN(0,4) = 2.59. Finally, for LP, there was a difference between "brief glimpse" and "almost clear experience" for both color photographs, t(21) = 3.95, p = 0.001, dz = 0.84, BN(0,3) = 6.60 × 102 , and line-drawings, t(21) = 2.11, p < 0.05, dz = 0.45, BN(0,3) = 3.24. That is, the results suggested that the N2, VAN, and LP amplitudes could be related to the emergence of subjective awareness, while the evidence for P3a amplitude marking a change in subjective awareness was insensitive.

To further examine how the later components correlated with the subjective awareness, we used "no experience," "brief glimpse," and "almost clear experience" to calculate the linear trend coefficient for each of the later component (i.e., the difference between the first and last ratings). For N2, the linear trend coefficients were above zero for both color photographs and line-drawings, t(21) = 5.81, p < 0.001, dz = 1.24, BN(0,2) = 3.84 × 105 , t(21) = 3.16, p < 0.01, dz = 0.67, BN(0,2) = 27.03. For VAN, the linear trend coefficients were below zero for both types of images, t(21) = −4.16, p < 0.001, dz = 0.89, BN(0,2) = 1.13 × 103 , t(21) = −3.70, p = 0.001, dz = 0.79, BN(0,2) = 3.815 × 102 . For LP, there was not much evidence either way for whether or not the linear trend coefficients were different from zero for both types of images, t(21) = 1.64, p = 0.12, dz = 0.35, BN(0,2) = 2.56, t(21) = 1.31, p = 0.20, dz = 0.28, BN(0,2) = 1.58. Moreover, for N2, "no experience" differed from "brief glimpse" for both color photographs, t(21) = −2.80, p < 0.05, dz = 0.60, BN(0,4) = 15.88, and line-drawings, t(21) = −2.13, p < 0.05, dz = 0.45, BN(0,4) = 5.14. For VAN, "no experience" also differed from "brief glimpse" for both color photographs, t(21) = 3.90, p = 0.001, dz = 0.83, BN(0,3) = 5.84 × 102 , and line-drawings: t(21) = 2.43, p < 0.05, dz = 0.52, BN(0,3) = 6.08. But for LP, there was marginal evidence for no difference between "no experience" and "brief glimpse" for color photographs, t(21) = 0.80, p = 0.43, dz = 0.17, BN(0,3) = 0.37, and evidence for no difference between "no experience" and "brief glimpse" for line-drawings,

FIGURE 3 | Grand-average ERPs for each experience rating for color photographs and line-drawings. (A) Grand-average ERPs of correct trials at electrodes FCz, Pz, and Oz for each experience rating for color photographs and line-drawings. (B) ERP differences between every two adjacent experience ratings for color photographs and line-drawings. (C) The scalp topography of the P1, N1, P2, N2, VAN, P3a, and LP, trials of the relatively higher experience ratings minus trials of the adjacent lower experience ratings separately for color photographs and line-drawings.

TABLE 1 | Significant results of the two-way repeated ANOVAs performed over the amplitude of each component, considering experience ratings and type of images.

In each ANOVA, we report F-values with significance and η 2 p . ∗p < 0.05, ∗∗p < 0.01, ∗∗∗p < 0.001.

t(21) = 0.26, p = 0.80, dz = 0.06, BN(0,3) = 0.22. The results suggested that the N2 and VAN could be related to PAS in a linear way, whereas the LP correlated with PAS in a non-linear fashion.

To explore the relationship between experience ratings and ERPs controlling for ISIs, we firstly described the distribution of ISIs cross correct trials for each experience rating for color photographs and line-drawings (see **Figure 5**). As can be seen from **Figure 5**, the number of correct trials for some ISI and experience rating combinations was rather small. As ERP effects usually require a large number of trials to measure them accurately in each condition (Luck, 2005, p. 23), "no experience" and "brief glimpse" were collapsed to form an "unconscious" category, while "almost clear experience" and "clear experience" were collapsed together for a "conscious" category, as per Melloni et al. (2011). The data from three participants were excluded because the number of trials for at least one condition was zero for them. **Figure 6** shows the ERP data for conscious and unconscious categories of each ISI for color photographs and line-drawings. An ANOVA with type of image (color photographs vs. line-drawings), experience ratings (unconscious vs. conscious), and ISI (0, 13, 26, and 200 ms) was conducted separately for amplitudes of N2, VAN, and LP. For N2, there was an experience rating effect, F(1,18) = 28.91, p < 0.001, η 2 p = 0.62,

fpsyg-08-00210 February 13, 2017 Time: 14:18 # 8

BN(0,4) = 2.15 × 105 , a significant ISI effect, F(3,54) = 20.39, p < 0.001, η 2 p = 0.53. For VAN, there was a type of image effect, F(1,18) = 26.89, p < 0.001, η 2 p = 0.60, BN(0,4) = 8.68 × 104 , a significant ISI effect, F(1.65,29.63) = 37.53, p < 0.001, η 2 p = 0.68, and a significant type of image by ISI interaction, F(2.34,42.20) = 8.71, p < 0.001, η 2 p = 0.33. For LP, there was a type of image effect, F(1,18) = 7.50, p < 0.05, η 2 p = 0.29, BN(0,3) = 8.30, an experience rating effect, F(1,18) = 17.67, p = 0.001, η 2 p = 0.50, BN(0,3) = 9.03 × 102 , and a significant ISI effect, F(2.21,39.69) = 26.12, p < 0.001, η 2 p = 0.59. The main effect of experience ratings on amplitudes indicated that the N2 and LP were each related to subjective awareness even controlling for ISI.

# DISCUSSION

Our behavioral results showed that people reported having more "clear experience" but less "brief glimpse" for color photographs than for line-drawings, suggesting that people had more experienced clarity for color photographs than for line-drawings. Importantly, there was no difference for accuracy between color photographs and line-drawings for each experience rating and the accuracy for both types of image gradually increased with rating, consistent with the experience ratings reflecting visibility. However, the accuracy for both types of images was above chance for "no experience," providing evidence for subliminal perception with an elaborated subjective scale. This may seem inconsistent with the finding of Ramsøy and Overgaard (2004), in that they found the accuracy for the stimulus form was not significantly better than chance for "no experience". However, a non-significant result is not in itself evidence for the null hypothesis.