user11629: I understand that AlphaZero has to use a different kind of hardware than regular Stockfish. I would expect that the hardware has a large effect on engine strength. That's why I wonder whether there have been any attempts made to provide comparable hardware to both. Also what would "comparable" mean here?
Specifically I read that people complain about:

Stockfish being given only 1 GB of cache, and
the time limit of 1 min/move (How would this disadvantage Stockfish?)


user14893: Visit Talkchess Forum to know more, there is where you will find some 3000 programmers. 
This was all a scam. 
Alpha played on 30 times bigger hardware than SF, 4TPUs vs 64 cores. 
4TPUs is around 1000 cores or even more. 
Alpha had simulated opening book, trained on countless top GM winning games. 
SF had very little hash. 
TC was fixed at 1 minute per move, which is again detrimental to SF, which has advanced time management. 
TPUs lack the SMP inefficiencies with more cores, so the hardware advantage was even bigger. 
Etc, etc., so basically, this was just a huge publicity stunt on the part of Google. 
Currently, Alpha is around 2800 on single core, so 400 elos below SF, and will not advanced much in the future, as, from now on, it will need advanced evaluation it will not be able to discover. 
Concerning the 4-hours issue, well, LOL, this was 48 hours ago, so now Alpha is at 5000 elo? 
Come on.

user15219: Running on comparable hardware would be required if Google's end goal was to build a superior chess engine, but this exercise wasn't really about chess. Chess is just a convenient way to demonstrate the AI's ability to learn complicated tasks from scratch. If it can perform well against some vaguely reasonable configuration of Stockfish, it's checked the box. 
I predict the Google team will not spend much more effort on chess; instead, they will move on to other problems that AI has never been able to accomplish.

user3950: One of the original authors of Stockfish answers the specific complaints you mentioned here:

Meanwhile Chess.com also received a lengthy comment from one of the
  original Stockfish authors, Tord Romstad, which we'll give in full:

The match results by themselves are not particularly meaningful
    because of the rather strange choice of time controls and Stockfish
    parameter settings: The games were played at a fixed time of 1
    minute/move, which means that Stockfish has no use of its time
    management heuristics (lot of effort has been put into making
    Stockfish identify critical points in the game and decide when to
    spend some extra time on a move; at a fixed time per move, the
    strength will suffer significantly). The version of Stockfish used is
    one year old, was playing with far more search threads than has ever
    received any significant amount of testing, and had way too small hash
    tables for the number of threads. I believe the percentage of draws
    would have been much higher in a match with more normal conditions.
On the other hand, there is no doubt that AlphaZero could have played
    better if more work had been put into the project (although the "4
    hours of learning" mentioned in the paper is highly misleading when
    you take into account the massive hardware resources used during those
    4 hours). But in any case, Stockfish vs AlphaZero is very much a
    comparison of apples to orangutans. One is a conventional chess
    program running on ordinary computers, the other uses fundamentally
    different techniques and is running on custom designed hardware that
    is not available for purchase (and would be way out of the budget of
    ordinary users if it were).
From another perspective, the apples vs orangutans angle is the most
    exciting thing about this: We now have two extremely different (both
    on the hardware and the software side) man-made entities that both
    display super-human chess playing abilities. That's much more
    interesting than yet another chess program that does the same thing as
    existing chess programs, just a little better. Furthermore, the
    adaptability of the AlphaZero approach to new domains opens exciting
    possibilities for the future.
For chess players using computer chess programs as a tool, this
    breakthrough is unlikely to have a great impact, at least in the short
    term, because of the lack of suitable hardware for affordable prices.
For chess engine programmers -- and for programmers in many other
    interesting domains -- the emergence of machine learning techniques
    that require massive hardware resources in order to be effective is a
    little disheartening. In a few years, it is quite possible that an
    AlphaZero like chess program can be made to run on ordinary computers,
    but the hardware resources required to create them will still be way
    beyond the budget of hobbyists or average sized companies. It is
    possible that an open source project with a large distributed network
    of computers run by volunteers could work, but the days of hundreds of
    unique chess engines, each with their own individual quirks and
    personalities, will be gone.


Source: https://www.chess.com/news/view/alphazero-reactions-from-top-gms-stockfish-author

user15115: I think it's best if I elaborate on your second point with an example move in the game 1 between AlphaZero and Stockfish which also served to satisfy my curiosity today.

the time limit of 1 min/move (How would this disadvantage Stockfish?)

Stockfish's performance is dependent upon both the time limit and the hardware configuration, so just think of when someone doubles the CPU threads, then Stockfish needs less time (not necessarily half) to find the solution than it would with the first configuration.
On the first report that was posted on Chess.com someone claimed that Stockfish was not playing optimally because he couldn't reproduce the same results using the same Stockfish on his computer. He said that on the position below (game 1 - move 11) Stockfish played Kg1-h1 (moved its king) which made no sense at all. On the other hand, stockfish on his computer showed a more developing move like Be3 (move the dark square bishop), lets look at the position:

Yes, it was a passive move and it seems that Stockfish should have played a more developing move. But he was wrong. Why? Because he ran Stockfish for 15 seconds, and if he had run it for an hour he would have gotten Kg1-h1 as the best move in that position. Stockfish changes it's decision when it analyses all the possible moves in more depth. Here's what I originally said in my reply:
I ran the latest stockfish on the position (at move 11):

At first, It gives b4 as the optimal move when the engine is running for about a minute. After that, it decides Be3 is better.
But after 5 minutes on my hardware that runs on 1,400k nodes/s it will decide to go with Kh1 as the optimal move.
In the paper, it is said that stockfish calculates 70,000k positions per second and is run for 1 minute per move, that's about 50 times my hardware, so I'll let mine run for 50 minutes... Kg1-h1 is still the choice for Stockfish.

Time limit is the key
In the above case, it probably didn't matter much if Stockfish ran for twice the time because the decision would have been the same, but on the next move it definitely would:

In this position, Stockfish chose to move the pawn on the left side (a4-a5). Let's say I have a computer that runs the Stockfish engine at a speed of 1,400k nodes per second, that's about 50 times lower than the Stockfish in the real game (In the paper, it says 70,000k n/s). So I can simulate the game if I run it for 50 minutes at each move. Okay.
I ran Stockfish analysis on the above position and I got the following results:

Stockfish started out suggesting some moves, but after 6 minutes on my computer (corresponds to 7.2 seconds on the Stockfish in the real game) it preferred a4-a5 just as the game went.

That's good, but I kept it running for a complete 50 minutes in order to reach the computations of the Stockfish in the game that was allowed 1 minute:

After 18 minutes the line changed back to Qc3.
Again, after 24 minutes the best move was Ne3.
After 27 minutes, it suggested Ng1 all the way up to 56 minutes. After that, the optimal move was Nf3xe5 but it's past the 50 minutes mark I mentioned before (you can see the engine analysis for an hour and 22 minutes in this screenshot). 

The sad truth is that I believe Stockfish lost all its games because of the time limit. Stockfish gets a more in-depth search and evaluation as the time passes and in the game it wasn't allowed to use an opening book which makes it consider many moves in shallow depths. Note that in the actual game a4-a5 was played which shows that (assuming it could evaluate 70 million positions per second) the Stockfish in the game didn't spend more than 21.6 seconds on the move. Otherwise, it would have changed its decision to those three other moves in the actual game. The reason for this is still unclear to me since my Stockfish was also consuming less memory (about ~130MB of RAM compared to the 1GB mentioned in the original paper, assuming all of it goes to hash tables). 

Conclusion
The hardware that ran Stockfish, as I pointed out, was at the very best 18 times faster than mine (Update: on a single core) based on the move I analyzed. I'm not sure if AlphaZero could really make use of such hardware to train its networks in 4 hours, I can only assume it's too low for a game like chess. Besides, AlphaZero spent those hours on learning which also includes building solid openings (and as the paper points out, preferences over certain openings). On the other hand, Stockfish was handicapped on openings, and it did not evaluate 70 million positions per second for 60 seconds on each move. 
As a final note, all the things I said were based on my assumptions. Of course, the outcome of AlphaZero and the games were super interesting to me. However, I would have loved to see a game where the Stockfish play was just like what I get on my computer, too. That is, more time and an opening book allowed. It's also easy to get the outputs of Stockfish analysis on every move, and I wish they release it in order to show how well it performed.

user4418: 
That's why I wonder whether there have been any attempts made to
  provide comparable hardware to both.

This is Google you're talking about! So the answer is obviously "No".
From the original paper hardware used for initialising and training -

Training proceeded for 700,000 steps (mini-batches of size 4,096)
  starting from randomly initialised parameters, using 5,000
  first-generation TPUs (15) to generate self-play games and 64
  second-generation TPUs to train the neural networks

and hardware used for the games -

AlphaZero and the previous AlphaGo Zero used a single machine with 4
  TPUs Stockfish and Elmo played at their strongest skill level using 64
  threads and a hash size of 1GB.

So, AlphaZero used special hardware developed by Google. It used specialized Tensor Processor Units (TPUs) rather than general Central Processing Units (CPUs) as are available commercially.
This is how Wikipedia describes the second generation TPUs they used -

The second generation TPU was announced in May 2017. Google stated the
  first generation TPU design was memory bandwidth limited, and using 16
  GB of High Bandwidth Memory in the second generation design increased
  bandwidth to 600 GB/s and performance to 45 TFLOPS. The TPUs are then
  arranged into 4-chip 180 TFLOPS modules

They used 4 TPUs for the games, so a processing power of 180 TFLOPS.
Note TFLOPS = 1000 billion floating point operations per second.
For comparison Intel's latest most powerful chip is the Core i9 Extreme Edition processor which clocks in at 1 TFLOP. A top of the line I7 that you would find in a gaming machine would typically be about 100 GFLOPs (i.e. one tenth of a TFLOP).
I think it's fair to say that AlphaZero was using an 800 pound gorilla of a hardware configuration compared to Stockfishes mouse.

11629: <p>I understand that AlphaZero has to use a different kind of hardware than regular Stockfish. I would expect that the hardware has a large effect on engine strength. That's why I wonder whether there have been any attempts made to provide comparable hardware to both. Also what would "comparable" mean here?</p>

<p>Specifically I read that people complain about:</p>

<ul>
<li>Stockfish being given only 1 GB of cache, and</li>
<li>the time limit of 1 min/move (How would this disadvantage Stockfish?)</li>
</ul>
 14893: <p>Visit Talkchess Forum to know more, there is where you will find some 3000 programmers. 
This was all a scam. 
Alpha played on 30 times bigger hardware than SF, 4TPUs vs 64 cores. 
4TPUs is around 1000 cores or even more. 
Alpha had simulated opening book, trained on countless top GM winning games. 
SF had very little hash. 
TC was fixed at 1 minute per move, which is again detrimental to SF, which has advanced time management. 
TPUs lack the SMP inefficiencies with more cores, so the hardware advantage was even bigger. 
Etc, etc., so basically, this was just a huge publicity stunt on the part of Google. 
Currently, Alpha is around 2800 on single core, so 400 elos below SF, and will not advanced much in the future, as, from now on, it will need advanced evaluation it will not be able to discover. 
Concerning the 4-hours issue, well, LOL, this was 48 hours ago, so now Alpha is at 5000 elo? 
Come on.</p>
 15219: <p>Running on comparable hardware would be required if Google's end goal was to build a superior chess engine, but this exercise wasn't really about chess. Chess is just a convenient way to demonstrate the AI's ability to learn complicated tasks from scratch. If it can perform well against some vaguely reasonable configuration of Stockfish, it's checked the box. </p>

<p>I predict the Google team will not spend much more effort on chess; instead, they will move on to other problems that AI has never been able to accomplish.</p>
 3950: <p>One of the original authors of Stockfish answers the specific complaints you mentioned here:</p>

<blockquote>
  <p>Meanwhile Chess.com also received a lengthy comment from one of the
  original Stockfish authors, Tord Romstad, which we'll give in full:</p>
  
  <blockquote>
    <p>The match results by themselves are not particularly meaningful
    because of the rather strange choice of time controls and Stockfish
    parameter settings: The games were played at a fixed time of 1
    minute/move, which means that Stockfish has no use of its time
    management heuristics (lot of effort has been put into making
    Stockfish identify critical points in the game and decide when to
    spend some extra time on a move; at a fixed time per move, the
    strength will suffer significantly). The version of Stockfish used is
    one year old, was playing with far more search threads than has ever
    received any significant amount of testing, and had way too small hash
    tables for the number of threads. I believe the percentage of draws
    would have been much higher in a match with more normal conditions.</p>
    
    <p>On the other hand, there is no doubt that AlphaZero could have played
    better if more work had been put into the project (although the "4
    hours of learning" mentioned in the paper is highly misleading when
    you take into account the massive hardware resources used during those
    4 hours). But in any case, Stockfish vs AlphaZero is very much a
    comparison of apples to orangutans. One is a conventional chess
    program running on ordinary computers, the other uses fundamentally
    different techniques and is running on custom designed hardware that
    is not available for purchase (and would be way out of the budget of
    ordinary users if it were).</p>
    
    <p>From another perspective, the apples vs orangutans angle is the most
    exciting thing about this: We now have two extremely different (both
    on the hardware and the software side) man-made entities that both
    display super-human chess playing abilities. That's much more
    interesting than yet another chess program that does the same thing as
    existing chess programs, just a little better. Furthermore, the
    adaptability of the AlphaZero approach to new domains opens exciting
    possibilities for the future.</p>
    
    <p>For chess players using computer chess programs as a tool, this
    breakthrough is unlikely to have a great impact, at least in the short
    term, because of the lack of suitable hardware for affordable prices.</p>
    
    <p>For chess engine programmers -- and for programmers in many other
    interesting domains -- the emergence of machine learning techniques
    that require massive hardware resources in order to be effective is a
    little disheartening. In a few years, it is quite possible that an
    AlphaZero like chess program can be made to run on ordinary computers,
    but the hardware resources required to <em>create</em> them will still be way
    beyond the budget of hobbyists or average sized companies. It is
    possible that an open source project with a large distributed network
    of computers run by volunteers could work, but the days of hundreds of
    unique chess engines, each with their own individual quirks and
    personalities, will be gone.</p>
  </blockquote>
</blockquote>

<p>Source: <a href="https://www.chess.com/news/view/alphazero-reactions-from-top-gms-stockfish-author" rel="noreferrer">https://www.chess.com/news/view/alphazero-reactions-from-top-gms-stockfish-author</a></p>
 15115: <p>I think it's best if I elaborate on your second point with an example move in the game 1 between AlphaZero and Stockfish which also served to satisfy my curiosity today.</p>

<blockquote>
  <p>the time limit of 1 min/move (How would this disadvantage Stockfish?)</p>
</blockquote>

<p>Stockfish's performance is dependent upon both the time limit <em>and</em> the hardware configuration, so just think of when someone doubles the CPU threads, then Stockfish needs less time (not necessarily half) to find the solution than it would with the first configuration.</p>

<p><a href="https://www.chess.com/news/view/google-s-alphazero-destroys-stockfish-in-100-game-match" rel="noreferrer">On the first report that was posted on Chess.com</a> someone claimed that Stockfish was not playing optimally because he couldn't reproduce the same results using the same Stockfish on his computer. He said that on the position below (game 1 - move 11) Stockfish played Kg1-h1 (moved its king) which made no sense at all. On the other hand, stockfish on his computer showed a more developing move like Be3 (move the dark square bishop), lets look at the position:</p>

<p><a href="https://raw.githubusercontent.com/abdolrezat/Stockfish/Figures/figures/move11.png" rel="noreferrer"><img src="https://raw.githubusercontent.com/abdolrezat/Stockfish/Figures/figures/move11.png" alt="Game 1 on move 11"></a></p>

<p>Yes, it was a passive move and it seems that Stockfish should have played a more developing move. But he was wrong. Why? Because he ran Stockfish for 15 seconds, and if he had run it for an hour he would have gotten Kg1-h1 as the best move in that position. Stockfish changes it's decision when it analyses all the possible moves in more depth. Here's <a href="https://www.chess.com/news/view/google-s-alphazero-destroys-stockfish-in-100-game-match" rel="noreferrer">what I originally said in my reply</a>:</p>

<p>I ran the latest stockfish on the position (at move 11):</p>

<ul>
<li>At first, It gives b4 as the optimal move when the engine is running for about a minute. After that, it decides Be3 is better.</li>
<li><p>But after 5 minutes on my hardware that runs on 1,400k nodes/s it will decide to go with Kh1 as the optimal move.</p></li>
<li><p>In the paper, it is said that stockfish calculates 70,000k positions per second and is run for 1 minute per move, that's about 50 times my hardware, so I'll let mine run for 50 minutes... Kg1-h1 is still the choice for Stockfish.</p></li>
</ul>

<h2>Time limit is the key</h2>

<p>In the above case, it probably didn't matter much if Stockfish ran for twice the time because the decision would have been the same, but on the next move it <strong>definitely would</strong>:</p>

<p><a href="https://raw.githubusercontent.com/abdolrezat/Stockfish/Figures/figures/move12.png" rel="noreferrer"><img src="https://raw.githubusercontent.com/abdolrezat/Stockfish/Figures/figures/move12.png" alt="enter image description here"></a></p>

<p>In this position, Stockfish chose to move the pawn on the left side (<strong>a4-a5</strong>). Let's say I have a computer that runs the Stockfish engine at a speed of 1,400k nodes per second, that's about 50 times lower than the Stockfish in the real game (<a href="https://arxiv.org/pdf/1712.01815.pdf" rel="noreferrer">In the paper</a>, it says 70,000k n/s). So I can simulate the game if I run it for 50 minutes at each move. Okay.</p>

<p>I ran Stockfish analysis on the above position and I got the following results:</p>

<ul>
<li>Stockfish started out suggesting some moves, but after 6 minutes on my computer (corresponds to <strong>7.2 seconds</strong> on the Stockfish in the real game) <strong>it preferred a4-a5 just as the game went</strong>.</li>
</ul>

<p>That's good, but I kept it running for a complete 50 minutes in order to reach the computations of the Stockfish in the game that was allowed 1 minute:</p>

<ul>
<li>After 18 minutes the line changed back to <strong>Qc3</strong>.</li>
<li>Again, after 24 minutes the best move was <strong>Ne3</strong>.</li>
<li>After 27 minutes, it suggested <strong>Ng1</strong> all the way up to 56 minutes. After that, the optimal move was <strong>Nf3xe5</strong> but it's past the 50 minutes mark I mentioned before (<a href="https://raw.githubusercontent.com/abdolrezat/Stockfish/Figures/figures/120engineanalysis_LI.jpg" rel="noreferrer">you can see the engine analysis for an hour and 22 minutes in this screenshot</a>). </li>
</ul>

<p>The sad truth is that I believe Stockfish lost all its games because of the time limit. Stockfish gets a more in-depth search and evaluation as the time passes and in the game it wasn't allowed to use an opening book which makes it consider many moves in shallow depths. Note that in the actual game <em>a4-a5</em> was played which shows that (assuming it could evaluate 70 million positions per second) the Stockfish in the game didn't spend more than 21.6 seconds on the move. Otherwise, it would have changed its decision to those three other moves in the actual game. The reason for this is still unclear to me since my Stockfish was also consuming less memory (about ~130MB of RAM compared to the 1GB mentioned <a href="https://arxiv.org/pdf/1712.01815.pdf" rel="noreferrer">in the original paper</a>, assuming all of it goes to hash tables). </p>

<hr>

<h2>Conclusion</h2>

<p>The hardware that ran Stockfish, as I pointed out, was at the very best 18 times faster than mine (Update: on a single core) based on the move I analyzed. I'm not sure if AlphaZero could really make use of such hardware to train its networks in 4 hours, I can only assume it's too low for a game like chess. Besides, AlphaZero spent those hours on learning which also includes building solid openings (and as the paper points out, preferences over certain openings). On the other hand, Stockfish was handicapped on openings, and it did not evaluate 70 million positions per second for 60 seconds on each move. </p>

<p>As a final note, all the things I said were based on my assumptions. Of course, the outcome of AlphaZero and the games were super interesting to me. However, I would have loved to see a game where the Stockfish play was just like what I get on my computer, too. That is, more time and an opening book allowed. It's also easy to get the outputs of Stockfish analysis on every move, and I wish they release it in order to show how well it performed.</p>
 4418: <blockquote>
  <p>That's why I wonder whether there have been any attempts made to
  provide comparable hardware to both.</p>
</blockquote>

<p>This is Google you're talking about! So the answer is obviously "No".</p>

<p>From <a href="https://arxiv.org/pdf/1712.01815.pdf" rel="noreferrer">the original paper</a> hardware used for initialising and training -</p>

<blockquote>
  <p>Training proceeded for 700,000 steps (mini-batches of size 4,096)
  starting from randomly initialised parameters, using 5,000
  first-generation TPUs (15) to generate self-play games and 64
  second-generation TPUs to train the neural networks</p>
</blockquote>

<p>and hardware used for the games -</p>

<blockquote>
  <p>AlphaZero and the previous AlphaGo Zero used a single machine with 4
  TPUs Stockfish and Elmo played at their strongest skill level using 64
  threads and a hash size of 1GB.</p>
</blockquote>

<p>So, AlphaZero used special hardware developed by Google. It used specialized Tensor Processor Units (TPUs) rather than general Central Processing Units (CPUs) as are available commercially.</p>

<p>This is how Wikipedia describes the second generation TPUs they used -</p>

<blockquote>
  <p>The second generation TPU was announced in May 2017. Google stated the
  first generation TPU design was memory bandwidth limited, and using 16
  GB of High Bandwidth Memory in the second generation design increased
  bandwidth to 600 GB/s and performance to 45 TFLOPS. The TPUs are then
  arranged into 4-chip 180 TFLOPS modules</p>
</blockquote>

<p>They used 4 TPUs for the games, so a processing power of 180 TFLOPS.
Note TFLOPS = 1000 billion floating point operations per second.</p>

<p>For comparison Intel's latest most powerful chip is the Core i9 Extreme Edition processor which clocks in at 1 TFLOP. A top of the line I7 that you would find in a gaming machine would typically be about 100 GFLOPs (i.e. one tenth of a TFLOP).</p>

<p>I think it's fair to say that AlphaZero was using an 800 pound gorilla of a hardware configuration compared to Stockfishes mouse.</p>
