user-1: In my own games I have seen fewer advance Caro-Kann games (1. e4 c6 2. d4 d5 3. e5) than the semi-open games (3. Nc3 or 3. Nd2) or the Botvinnik systems (3. exd5 cxd5 4. c4 or 2. c4).
My questions are:

Has the advance Caro-Kann become less popular at club or master level?
How do we know? E.g. how can we mine a database to show opening popularity over time?


user-1: Summary
From the plot of games played under the Caro-Kann ECO codes (B10-19), the advance variation (in B12, but B12 does hold some other miscellaneous variations) has been consistently the most popular line since the 1990's and peaked at 1,803 games in 2008.

Methodology

I extracted the games from the Fritz for Fun 13 into PGN format
I ran the following R code, which extracts the year (mandatory) and the ECO (optional). The games are filtered to ECO codes B10-19 (Caro-Kann) and for games from 1927. Why 1927? That is when it debuted at the top:


He [Capablanca] used it [the Caro-Kann] in his crushing strategic destruction of Aaron Nimzowitsch in the great New York tournament of 1927.

Source: The Caro-Kann: Move-by-Move by IM Lakdawala

The raw number of games are then plotted over time. Unfortunately my data only goes to 2009


library(ggplot2) # For making the charts look nice

dat <- readLines("C:/Users/Peter/SkyDrive/Chess/Fritz for fun 13 database.pgn") # Read in the data from a local file

eco <- dat[grep("\\[ECO ",dat)] # Extract the ECO, if there is one 
eco <- sub("\\[ECO \"","",eco) 
eco <- sub("\"\\]","",eco)

year <- dat[grep("\\[Date ",dat)]
year <- sub("\\[Date [\"]","",year)
year <- sub("\\..*","",year)
year <- as.numeric(year)

ecoindex <- grep("\\[ECO ",dat) # Find which lines have an eco
yearindex <- grep("\\[Date ",dat) 
yearindex <- yearindex + 5 # The date is always the 3rd pgn tag and is mandatory. The ECO, if present, is the 8th tag
matchindex <- yearindex %in% ecoindex  # This code finds which games have both a year and eco tag

year <- year[matchindex] # Filter out games without an ECO

countgame <- table(eco,year)
countgame <- as.data.frame(countgame) # Convert to a data frame

Carocode <- paste("B",10:19,sep="") # ECO codes for the Caro-Kann are B10-B19

Carogame <- countgame[which(countgame$eco %in% Carocode),] # Keep the data for the Caro-Kann ECO codes

Carogame$year <- as.numeric(as.character(Carogame$year))
Carogame <- Carogame[Carogame$year>=1927,]

p1 <- ggplot(Carogame,aes(x=year,y=Freq,group=eco))+geom_line(aes(colour=eco)) # Plot count of games

p1 <- p1 + ggtitle("Number of Caro-Kann games over time") + ylab("Number of games")

print(p1)


-1: <p>In my own games I have seen fewer advance Caro-Kann games (1. e4 c6 2. d4 d5 3. e5) than the semi-open games (3. Nc3 or 3. Nd2) or the Botvinnik systems (3. exd5 cxd5 4. c4 or 2. c4).</p>

<p>My questions are:</p>

<ol>
<li>Has the advance Caro-Kann become less popular at club or master level?</li>
<li>How do we know? E.g. how can we mine a database to show opening popularity over time?</li>
</ol>
 -1: <h2>Summary</h2>

<p>From the plot of games played under the Caro-Kann ECO codes (<a href="https://en.wikipedia.org/wiki/Caro%E2%80%93Kann_Defence#ECO_codes" rel="nofollow noreferrer">B10-19</a>), the advance variation (in B12, but B12 does hold some other miscellaneous variations) <strong>has been consistently the most popular line since the 1990's and peaked at 1,803 games in 2008</strong>.</p>

<p><a href="https://i.stack.imgur.com/rsWdr.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/rsWdr.png" alt="Number of Caro-Kann games over time"></a></p>

<h2>Methodology</h2>

<ul>
<li>I extracted the games from the <em>Fritz for Fun 13</em> into PGN format</li>
<li>I ran the following R code, which extracts the year (mandatory) and the ECO (optional). The games are filtered to ECO codes B10-19 (Caro-Kann) and for games from 1927. Why 1927? That is when it debuted at the top:</li>
</ul>

<blockquote>
  <p>He [Capablanca] used it [the Caro-Kann] in his crushing strategic destruction of Aaron Nimzowitsch in the great New York tournament of 1927.</p>
</blockquote>

<p>Source: <em>The Caro-Kann: Move-by-Move</em> by IM Lakdawala</p>

<ul>
<li>The raw number of games are then plotted over time. Unfortunately my data only goes to 2009</li>
</ul>

<hr>

<pre><code>library(ggplot2) # For making the charts look nice

dat &lt;- readLines("C:/Users/Peter/SkyDrive/Chess/Fritz for fun 13 database.pgn") # Read in the data from a local file

eco &lt;- dat[grep("\\[ECO ",dat)] # Extract the ECO, if there is one 
eco &lt;- sub("\\[ECO \"","",eco) 
eco &lt;- sub("\"\\]","",eco)

year &lt;- dat[grep("\\[Date ",dat)]
year &lt;- sub("\\[Date [\"]","",year)
year &lt;- sub("\\..*","",year)
year &lt;- as.numeric(year)

ecoindex &lt;- grep("\\[ECO ",dat) # Find which lines have an eco
yearindex &lt;- grep("\\[Date ",dat) 
yearindex &lt;- yearindex + 5 # The date is always the 3rd pgn tag and is mandatory. The ECO, if present, is the 8th tag
matchindex &lt;- yearindex %in% ecoindex  # This code finds which games have both a year and eco tag

year &lt;- year[matchindex] # Filter out games without an ECO

countgame &lt;- table(eco,year)
countgame &lt;- as.data.frame(countgame) # Convert to a data frame

Carocode &lt;- paste("B",10:19,sep="") # ECO codes for the Caro-Kann are B10-B19

Carogame &lt;- countgame[which(countgame$eco %in% Carocode),] # Keep the data for the Caro-Kann ECO codes

Carogame$year &lt;- as.numeric(as.character(Carogame$year))
Carogame &lt;- Carogame[Carogame$year&gt;=1927,]

p1 &lt;- ggplot(Carogame,aes(x=year,y=Freq,group=eco))+geom_line(aes(colour=eco)) # Plot count of games

p1 &lt;- p1 + ggtitle("Number of Caro-Kann games over time") + ylab("Number of games")

print(p1)
</code></pre>
