All relevant data are within the paper and its Supporting Information files.

Introduction {#sec001}
============

The proliferation of online social media services such as Twitter is widely recognized as signifying a revolution in how we utilize information and understand the way our society communicates. It also presents an open opportunity for any interested party to utilize the massive data accrued from such services for understanding the world as well as ourselves \[[@pone.0124722.ref001], [@pone.0124722.ref002]\]. Among many attempts to harness the potential of online social media, a prominent one is to mine the opinions and sentiments of the public for a variety of purposes, either academic or commercial \[[@pone.0124722.ref003]--[@pone.0124722.ref006]\]. The use of social media for such purposes poses a potentially critical problem, however: Since online social media users constitute an uncontrolled, thus likely unfair, sample of the population unlike in well-designed opinion polls, it is unclear to what extent the online sphere accurately represents the offline world. User profiles collected by online social media and made available to researchers frequently fall short of giving a sufficiently accurate demographic information about the users, and the traditional polling techniques using questionnaires have to be used to augment them \[[@pone.0124722.ref007]\]. The ideal case would be that the demographics of online users were identical to that of the offline public at large, guaranteeing the agreement of the opinions mined from the online sphere and with traditional polls. A less ideal, but a manageable case would be when the online demographics are precisely known, allowing us to employ straightforward calibrations to find accurate results. But for a limited number of cases the demographic differences between online service users and non-users are not precisely known \[[@pone.0124722.ref008]\], and given the breadth of topics discussed online and the difficulty of accurately estimating the number of users it does not seem realistic that such a state can ever be reached.

To those looking to mine for information from social media as a proxy for conducting polling, the potential disparity between the online and the offline world due to the unknowable demographics raises serious doubts on the usefulness of online social media, particularly where the offline reality is essential such as politics. In such cases the lack of a systematic method by which to compare the online and the offline would make finding relevant, useful information from online data infeasible. When researchers study such problems, they could circumvent this issue by restricting the scope of their study to the online sphere so that by design the real-world public at large is of little concern. This may be appropriate when the issue being studied is pertinent only to the online world; for instance, when the issue affects the online sphere only, or when one declares that their only interest is in the opinions of the online sphere. While certain scientific discoveries can still be made this way since the data themselves are nonetheless novel and interesting, the inability to draw conclusions regarding many issues pertinent to the offline potentially cripples the potential of online social media.

In order to overcome such a problem, it is important to identify the discrepancies between the online and the offline quantitatively. One way of achieving that is to apply a common analytical and modeling framework to comparable data sets representing the online and the offline so that we can perform a cross-analysis of the patterns found therein. In this paper we demonstrate the process using as an example the landscape of political spectra (signifying the political leanings or positions) constructed from distinct online and offline data of the U.S. and South Korea, two nations with modern representative democracies. Specifically we study the political spectra of the legislators in each country, constructed in two ways. First, we construct a spectrum from the Twitter followership network, which we take as representing the online political landscape. There are some direct evidence that connection on social networks are biased towards those with similar political viewpoints. In a well-known study, Adamic and Glance showed a clear partisan divide on the blogosphere \[[@pone.0124722.ref009]\]. Regarding Twitter, Parmalee and Bichard showed that over 70% of those espousing strong conservative or liberal ideology completely avoided following political leaders who challenge their beliefs \[[@pone.0124722.ref010]\]. In the case of Korea (one of the two countries studied in this work), Chang found that both politicians and ordinary Twitter users exhibit a systematic relationship between their political choices and their position in the Twitter network \[[@pone.0124722.ref011]\]. Second, we construct a spectrum from the legislators' roll call (voting) records, which we take as representing the offline political landscape that more closely reflects the true public one, since the legislators' voting is likely to be influenced by the offline public. The cross-examination of the two spectra should then reveal the differences between the online and the offline.

Results {#sec002}
=======

Data {#sec003}
----

We utilized the following four data sets: (i) The Twitter followership network of the members of the 111th U.S. Senate; (ii) the roll call of the Senators on legislative bills; (iii) The Twitter followership network of the members of the 18th Korean National Assembly; and (iv) the roll call of the Assembly members between the years 2008 to 2010 (covering three quarters of the term). The Twitter data are current as of June 2011, and Twitter users who followed two or more legislators were included in the data. The data contain all legislators who owned an official Twitter account at the time of data collection and maintained their seat during the entire term considered. This resulted in 67 senators (out of 100), 698 roll calls, and 139 806 Twitter users for the U.S., and 194 assembly members (our of 299), 1 119 roll calls, and 124 341 Twitter users for Korea. Both countries have two-party systems, the Republican Party (GOP) and the Democratic Party (DEM) that account for 99% of the U.S. Senate, and the Grand National Party (GNP) and the Korean Democratic Party (DP) that account for 86% of the Korean National Assembly. \[[@pone.0124722.ref012]\]

The data were modeled as bipartite networks. See [Fig 1](#pone.0124722.g001){ref-type="fig"} for the schematic illustrations of the data.

![Schematic illustrations for the data employed in our study.\
(A) The legislators are included in two distinct bipartite networks. On the left is the legislator--Twitter user network, and on the right the legislator--legislative bill network. Of the two types of edges---a 'nay' vote and a 'yea' vote---we consider the 'nay', since they are believed to carry more information in determining the legislators' political spectra (see the [Method](#sec007){ref-type="sec"} section for more detail). The structure of each bipartite network can reveal differences in political positions of the legislators, which is the origin of the online-offline discrepancy. Here, for example, the upper two legislators occupy similar positions that are different from that of the the lower one since their follower sets are disjoint. The two groups' voting patterns may show less clear differences. In this study we use Multidimensional Scaling (MDS) and Kendall's ranking correlation coefficient to quantify the spectra and their discrepancies. (B) A sample of the U.S. senator--Twitter followership network consisting of ten legislators and their Twitter followers. The red squares are Republican (GOP) senators, and the blue squares are the Democratic (DEM) senators. All other nodes are Twitter users.](pone.0124722.g001){#pone.0124722.g001}

Online and offline political spectra {#sec004}
------------------------------------

The data introduced above can be modeled as bipartite networks on which the political spectra are constructed using matrix decomposition methods \[[@pone.0124722.ref013], [@pone.0124722.ref014]\]. In this study we use Classical Multidimensional Scaling (CMDS) \[[@pone.0124722.ref015], [@pone.0124722.ref016]\]. The method requires a dissimilarity function that act as the distance between two legislators. We use the Kulczynski dissimilarity \[[@pone.0124722.ref017]\]. In essence, CMDS regards the dissimilarity values as the Euclidean distances between legislators in 1-dimensional space and determines the coordinates of the legislators that most closely matches the given distances. We then interpret the coordinates as the political positions of the legislators. Once the spectra of the legislators are determined this way using the Twitter network, we can construct the spectra of the Twitter users by averaging the positions of the legislators that each user follows (see the [Methods](#sec007){ref-type="sec"} section for more details).

We consider the offline, roll call-based spectra as revealing the true political positions of the legislators as they results from their actual actions of voting. Furthermore, we accept the roll call-based spectra as more accurately reflecting true political positions of the offline public and the society, as the legislators' actions are likely to be heavily influenced by public interests.

The political spectra of the legislators thus constructed are shown in Fig [2A](#pone.0124722.g002){ref-type="fig"} and [2B](#pone.0124722.g002){ref-type="fig"} for the U.S., and in Fig [2C](#pone.0124722.g002){ref-type="fig"} and [2D](#pone.0124722.g002){ref-type="fig"} for Korea. The spectra of the Twitter users (the average of the positions of the legislators that they follow) are shown as the solid lines in Fig [2A](#pone.0124722.g002){ref-type="fig"} and [2C](#pone.0124722.g002){ref-type="fig"}.

![Political spectra determined from Twitter and roll call data.\
(A) The Twitter-based spectrum of the legislators, and the spectrum of Twitter users in the U.S.; (B) The roll call-based spectrum of the legislators in the U.S.; (C) The Twitter-based spectrum of the legislators, and the spectrum of Twitter users in Korea; (D) The roll call-based spectrum of the legislators in South Korea. The legislators are indicated by the shapes (one shape corresponds to one legislator) and the spectra of the Twitter users are indicated by solid lines. The party-line splits of the legislators are evident in both the Twitter- and the roll call-based spectra for both countries. The general Twitter public (solid lines in Fig 2A and C) show a majority-minority reversal, more people aligning themselves with the minority parties.](pone.0124722.g002){#pone.0124722.g002}

A close examination of these spectra can reveals the disparities between the online and the offline. First, we find a stark indication of the significance of the disparity between Twitter and the offline in the majority-minority reversal in both countries: The majority of Twitter users align themselves with the minority parties. The mean and the median positions, and the overall distribution of the spectra are shown in [Fig 2A](#pone.0124722.g002){ref-type="fig"} for U.S. and [Fig 2C](#pone.0124722.g002){ref-type="fig"} for Korea. We find that in the U.S., Republican Senators John McCain (the presidential candidate in 2008) and Jim DeMint (of the populist Tea Party movement) boasting a disproportionately large number of followers are primarily responsible for this behavior. In Korea, the DP (green) overwhelms the GNP (blue) in the Twitter sphere despite being outnumbered by a nearly two-to-one margin in the number of seats in the Assembly.

Second, the comparison between the roll call-based spectra of the legislators as the proxy for the offline public at large and the spectra of the Twitter users also reveals a similar disparity. Although we cannot directly compare the online and offline spectra on the same plot because of the differences in scale, the comparison of relative biases in the distribution is still valid. For the U.S., the spectrum of the Twitter users is heavily skewed towards GOP in sharp contrast to the DEMs who are favored in the spectrum of the offline public (Fig [2A](#pone.0124722.g002){ref-type="fig"} and [2B](#pone.0124722.g002){ref-type="fig"}). The trend is similar in the Korean spectra: the offline public favors GNP in contrast to the Twitter user who favor DP (Fig [2C](#pone.0124722.g002){ref-type="fig"} and [2D](#pone.0124722.g002){ref-type="fig"}).

Third, there also exists a discrepancy between the real political positions of the legislators and their positions constructed from the Twitter data. The party-line split in the positions of the legislators (indicated as solid shapes in the figures) is immediately noticeable for both nations, and accordingly the roll call-based spectra and the Twitter-based spectra of the legislators exhibit some degree of overall agreement (similarity) as measured by Kendall's *τ*-coefficient (1 means identity; 0 means independence), with 0.68 ± 0.02 for the U.S. and 0.48 ± 0.01 for Korea (see [Methods](#sec007){ref-type="sec"} section for more detail). When we consider each party separately, however, we find a weaker agreement---for the U.S. parties, the *τ* is 0.463 ± 0.003 for GOP and 0.291 ± 0.022 for DEM. For the Korean parties that is 0.029 ± 0.005 for GNP and 0.062 ± 0.051 for DP.

This indicates that Korean Twitter population is more noticeably impervious to the political positions of the legislators, resulting in the very low level of intra-party correlations between the Twitter based spectrum and the roll call based spectrum of the legislators. In U.S. the correlation is more sizable.

These disparities between the Twitter users and the offline public show the limits of Twitter as a source for fair and accurate opinion mining, especially pertaining to political or social issues. In fact, the majority-minority reversal shows us that Twitter may be functioning as a de facto "alternative media."

Quantifying partisanship of twitter users {#sec005}
-----------------------------------------

The spread structures of the Twitter users' spectra in Fig [2A](#pone.0124722.g002){ref-type="fig"} and [2C](#pone.0124722.g002){ref-type="fig"} imply that Twitter users are biased (show partisanship) in choosing which legislators to follow. We can quantify the strength of a user's bias by measuring the probability that a user follows certain parties, and comparing it with the random probability that a user becomes connected to a party if the connections were made at random. If a user is unbiased they must be identical, and deviations between the two probabilities signify the strength of the bias.

A widely-used measure for the discrepancy between two probability distributions is the Kullback-Leibler (KL) divergence *D*. It is non-negative, i.e. *D* ≥ 0, 0 when two distributions being compared are identical, and positive otherwise. To account for random fluctuations we calculate the *Z*-score of KL divergences (see [Methods](#sec007){ref-type="sec"} section for detail) *Z* ~*D*~ as a function of the number of politicians (degree) *k* that a user follows, shown in [Fig 3](#pone.0124722.g003){ref-type="fig"}. *Z* ~*D*~ ≳ 1 implies partisanship, while *Z* ~*D*~ ≲ −1 implies the opposite. The radius of the disc around each plot point is proportional to the the log of the number of users with the given degree *k*. The figure shows the widespread nature of partisanship on Twitter \[[@pone.0124722.ref009]\], with the exception of high-degree Twitter users who are more balanced in following each party (see [Methods](#sec007){ref-type="sec"} section for more detail).

![Z-score of the Kullback-Leibler (KL) divergence as a function of the number of legislators *k* that a Twitter user follows in the U.S. (A), and in Korea (B).\
The radius of each circle is proportional to the log of the number of Twitter users with given *k*. *Z* ~*D*~ ≳ 1 implies a statistically significant level of partisanship (political bias). We see that partisanship is widespread in both countries, except for high-degree Twitter users for whom *D* → 0.](pone.0124722.g003){#pone.0124722.g003}

Discussion {#sec006}
==========

In this paper we proposed a method to compare the political spectra of the online and offline public. We found significant discrepancies between the online and the offline, demanding caution when one tries to use online social media such as Twitter as source for fair and accurate opinion mining. A quantitative measurement of political biases found widespread partisanship among Twitter users.

While our work shows the potential limits of the correctness of online social media-based findings, it is still meaningful that were able to obtain the political spectra of Twitter users based on their behavior observed objectively. This enables us to overcome uncertainty in methods based on self-assessment of political positions \[[@pone.0124722.ref018]--[@pone.0124722.ref020]\] caused by the lack of objective, common measure of political positions agreed upon by people. In our method, however, a person's political position is determined from the objective behavior of all other people in the data, allowing us to construct global and objective political spectra.

Methods {#sec007}
=======

Classical multi-Dimensional scaling {#sec008}
-----------------------------------

CMDS is a method to embed a set of objects in a pre-defined *N*-dimensional space such that the distance between objects match given dissimilarities as much as possible. Specifically, CMDS aims to produce the coordinates of the objects that minimize a loss function called STRAIN \[[@pone.0124722.ref015]\]. In this paper we set *N* = 1 to produce a linear political spectrum that agrees with the common way of thinking of political polarization (e.g. left wing versus right wing).

CMDS is performed by the eigendecomposition of the matrix made from the dissimilarities. The quality of the resulting coordinates in reproducing the given dissimilarities is defined as the fraction of the first eigenvalue over the sum of all positive eigenvalues. The values are 0.297 for Korea and 0.872 for the U.S. with the roll call data, and 0.113 for Korea and 0.093 for the U.S. with the Twitter data. The noticeably high value for the U.S. roll call data may be due to the stable two-party system of the U.S.

While one could theoretically gain more information from higher-dimension CMDS, we could not observe meaningful patterns allowing straightforward political interpretation beyond the first. Our 1-dimensional solutions show clear separations of the legislators along party lines, thus justifying the political interpretation given.

We used the Kulczynski dissimilarity \[[@pone.0124722.ref017]\]. In Twitter data it is given between two legislators *i* and *j* as $$\begin{array}{r}
\left. \delta_{T}\left( i,j \right) = 1 - \frac{1}{2} \middle| \frac{F_{i} \cap F_{j}}{|F_{i}|} + \frac{F_{i} \cap F_{j}}{|F_{j}|} \middle| , \right. \\
\end{array}$$ where *F* ~*x*~ denotes the set of the followers of legislator *x*. The Kulczynski dissimilarity is recommended as an alternative to the commonly used Jaccard measure when the size of *F* ~*x*~ exhibits a wide range as in our Twitter data ([Fig 4](#pone.0124722.g004){ref-type="fig"}) \[[@pone.0124722.ref021]\].

![Degree distributions in the data.\
(A) The distribution of the number of Twitter followers for each legislator. (B) The distribution of the number of legislators that Twitter users follow. All distribtuions are heavy-tailed.](pone.0124722.g004){#pone.0124722.g004}

The dissimilarity between legislators *i* and *j* in the roll call data is, similarly, $$\begin{array}{r}
\left. \delta_{R}\left( i,j \right) = 1 - \frac{1}{2} \middle| \frac{V_{i} \cap V_{j}}{|V_{i}|} + \frac{V_{i} \cap V_{j}}{|V_{j}|} \middle| , \right. \\
\end{array}$$ where *V* ~*x*~ denotes the set of *nay* (opposition to a bill's passing) votes of legislator *x*. We ignore the *yea* votes here. The motivation for this is that most bills that pass are bipartisan, potentially limiting the discriminating power of the *yea* votes.

Notes on the spectrum of Twitter users {#sec009}
--------------------------------------

From the legislator-Twitter data we estimated the spectrum of the legislators first on their Twitter followership, and then used it to obtain the spectrum of Twitter users. A dual approach could have been employed where we perform CMDS on the Twitter users by defining the dissimilarities between the users based on the set of legislators that they follow, and determining the positions of the legislators based on the results. While there is no reason to object this approach in principle, its computational cost was prohibitive, requiring several tens of gigabytes of memory to store the dissimilarity matrix alone.

Kendall's *τ* rank correlation coefficient {#sec010}
------------------------------------------

Kendall's rank correlation *τ* quantifies the similarity between two orderings of objects. Letting (*x* ~1~, *x* ~2~, ⋯, *x* ~*n*~) and (*y* ~1~, *y* ~2~, ⋯, *y* ~*n*~) be the orderings we wish to compare, Kendall's *τ* rank correlation coefficient is defined as $$\begin{array}{r}
{\tau = \frac{2\left( C - D \right)}{n\left( n - 1 \right)}.} \\
\end{array}$$

In this definition *C* is the number of (*i*, *j*) pairs such that the (*i*, *j*) satisfies *x* ~*i*~ \> *x* ~*j*~ and *y* ~*i*~ \> *y* ~*j*~ or it satisfies *x* ~*i*~ \< *x* ~*j*~ and *y* ~*i*~ \< *y* ~*j*~. *D* is the number of (*i*, *j*) pairs such that the (*i*, *j*) satisfies *x* ~*i*~ \> *x* ~*j*~ and *y* ~*i*~ \< *y* ~*j*~ or it satisfies *x* ~*i*~ \< *x* ~*j*~ and *y* ~*i*~ \> *y* ~*j*~.

Errors in Kendall's *τ* were estimated through the jackknife method \[[@pone.0124722.ref022]\].

Z-score of the Kullback-Leibler divergence {#sec011}
------------------------------------------

Kullback-Leibler divergence (KL divergence) measures deviation of a probability distribution from a reference probability distribution. Here the reference probability distribution is the fractions of seats occupied by each party, i.e. the probability that a random user would follow a legislator from a given party.

Let *P* ~*i*~ be the fraction of the number of legislators from party *i* among the number of the legislators, and *p* ~*ij*~ be the fraction of the number of legislators from party *i* followed by Twitter user *j* among all legislators followed by *j*. Then the KL divergence *D* ~*j*~ of Twitter user *j* is $$\begin{array}{r}
{D_{j} = \sum\limits_{i = 1}^{N}p_{ij}\log\frac{p_{ij}}{P_{i}},} \\
\end{array}$$ where *N* is the number of parties. KL divergence is non-negative. A large *D* ~*j*~ means that user *j*'s followership pattern does not agree with the reference probability, i.e. favors one party disproportionately when in a two-party system.

While the average KL divergence ⟨*D*⟩~*k*~ of Twitter users who follow *k* legislators can be obtained easily from the empirical data, for statistical significance we use the *Z*-score $$\begin{array}{r}
{Z_{D}\left( k \right) = \frac{\left\langle D \right\rangle_{k} - \mu_{k}}{\sigma_{k}},} \\
\end{array}$$ where *μ* ~*k*~ and *σ* ~*k*~ are the expectation and standard deviation of the KL divergence when a user choose to follow a party according to the reference probability *P* ~*i*~. If we assume a Twitter user follows *k* legislators randomly with uniform distribution, then the probability of $\overrightarrow{n} = \left( n_{1},\cdots,n_{N} \right)$, the vector of the number of legislators in each party that the user follows, is given by the multinomial distribution $f\left( \overrightarrow{n};\overrightarrow{P} \right)$ where *n* ~1~+⋯+*n* ~*N*~ = *k* and $\overrightarrow{P} = \left( P_{1},\cdots,P_{N} \right)$. Then we have $$\begin{array}{r}
{\mu_{k} = \sum\limits_{n_{1} + \cdots + n_{N} = k}f\left( \overset{\rightarrow}{n};\overset{\rightarrow}{P} \right)\sum\limits_{i = 1}^{N}\frac{n_{i}}{k}\log\frac{n_{i}}{kP_{i}}} \\
\end{array}$$ and $$\begin{array}{r}
{\sigma_{k}^{2} = \sum\limits_{n_{1} + \cdots + n_{N} = k}f\left( \overset{\rightarrow}{n};\overset{\rightarrow}{P} \right)\left( \sum\limits_{i = 1}^{N}\frac{n_{i}}{k}\log\frac{n_{i}}{kP_{i}} \right)^{2} - \mu_{k}^{2}.} \\
\end{array}$$

*Z* ~*D*~(*k*) ≳ 1 indicates that Twitter users follow a biased set of legislators, and that the bias is larger than the typical random fluctuation. Similarly, *Z* ~*D*~(*k*) ≲ −1 means that Twitter users follow the parties in close agreement with the reference probability *P* ~*i*~. As *k* approaches the actual number of legislators *p* ~*i*,\ *j*~ converges to *P* ~*i*~, *Z* ~*D*~(*k*) becomes more negative (since *D* → 0).

Supporting Information {#sec012}
======================

###### List of the Korean legislators.

Each line is a record for a legislator. The first column is the Twitter ID, and the second column is the name. The last column is the party of the legislator. The file is encoded with UTF-8 for Korean characters.

(TXT)

###### 

Click here for additional data file.

###### Followership data for the Korean legislators.

Each line is a followee-follower relation. The first column is the Twitter ID of a legislator, and the second column is the Twitter ID of a follower of the legislator.

(ZIP)

###### 

Click here for additional data file.

###### Roll call votes data of the Korean legislators.

Each line is a record of a legislator. The first column is the name of the legislator, and the rest columns are votes. A vote is encoded as 1 for yea, 2 for nay, 3 for abstention, and 4 for absence. The file is encoded with UTF-8 for Korean characters.

(TXT)

###### 

Click here for additional data file.

###### List of the US legislators.

The format is the same with [S1 Dataset](#pone.0124722.s001){ref-type="supplementary-material"}.

(TXT)

###### 

Click here for additional data file.

###### Followership data for the US legislators.

The format is the same with [S2 Dataset](#pone.0124722.s002){ref-type="supplementary-material"}.

(ZIP)

###### 

Click here for additional data file.

###### Roll call votes data of the US legislators.

The format is the same with [S3 Dataset](#pone.0124722.s003){ref-type="supplementary-material"}.

(TXT)

###### 

Click here for additional data file.

We thank Z. Hyung, C. Han, and B. Kahng.

[^1]: **Competing Interests:**The authors have declared that no competing interests exist.

[^2]: Conceived and designed the experiments: DL KH SY JP. Analyzed the data: DL KH JP. Contributed reagents/materials/analysis tools: DL KH JP. Wrote the paper: DL KH SY JP.
