Rybka 4 vs. Stockfish 1.8

For fans of computer chess matches, here's an email from Martin Thoresen:

I'm currently hosting a high-level, long time control computer match between the newest incarnation of Rybka and one of its closest rivals, Stockfish.

All information about hardware etc. can be seen on the website:

The broadcast is updated each 3 minutes and the engines are playing from 24 fixed positions with the most normal GM openings. They play each position as black and white, so a total of 48 games.

I was the one hosting the tournament where GM Kavalek annotated the game between Stockfish 1.7.1 and Rybka 4 as seen here:

Unfortunately I was unsuccessful of contacting him to say thanks.

Best Regards,

Martin Thoresen

Comments

[Glenn Snow]

One wonders how long they will continue to develop Rybka and other commercial programs when apparently free programs like Stockfish can compete on even terms (I suppose this isn't confirmed yet.) with them.

[Daniel]

I wonder when people will be more interested in software as a tool for learning, training, and analyzing and relatively less interested in which program is the "best."

[RuralRob]

Nowadays, comparing chess programs is like comparing Jupiter and Saturn to see which one will cause more damage after colliding it with Earth.

[Dennis Monokroussos]

D and RR: While people do like trying to be or have the best, there's a perfectly legitimate reason to be curious about such things: the better the program, the better one's opening prep can be.

[Larry L]

Dennis I want to thank you for letiting us know aout the match. It amazes me how the software programs with almost perfect tactical knowledge can stil lose games.It says that chess is still too complex to be completely expressed by a set number of rules. Watching the programs push their rating over the 3200 mark is just an example of the progress they are making. My first chess program was only a 1900+ level with a very limited opeing book.
As for comparing them against humans, that is as useless as comparing a modern automobiles speed to a human runner. Note however, under the right circumstances a human can out perfrom a car (eg.very bad terrain) using his or her versitility. On the other hand humans have used machines to improve human speed perfromance in a number of ways.

[cheVelle]

Where can you get Stockfish for free?

[Dennis Monokroussos]

Have you tried Googling it?

[ami]

http://lmgtfy.com/?q=Stockfish

[Dennis Monokroussos]

Very nice, ami!

[Larry L]

I having following the match and find the play very interesting. First I was surprised to see the programs allowing several "locked" pawn positions, with lots of manuvering before a final pawn break. Even very strong programs use to avoid that sort of thing wanting piece play at the expense of broken pawn structures. The other good feature of the match is the forced use of 24 different openings, with each side getting white and black with the same opening. The third interesting feature of the match, how close they are in playing strenght. Rybka 4 was three games up after only four rounds but as the openings changed Stockfish 1.8 was able to win two in a row on the white and black side of the same opening. Since then the match has been even with very every contested games over a variety of openings. So what is not to like, no Grandmaster draws, a large variety of openings and 7 out of 12 games have been decisive.

[JC]

I agree. I'm not too bothered to know which engine is best, but most of the signs seem very positive so far. I'd always imagined that chess would tend towards much more drawish play as the players became progressively stronger, but it seems that any such situation is a long way off. Together with the fairly high proportion of decisive games so far, it's encouraging to note that at this point White has won 4 games to Black's 3. As an indication of the future of high level chess play, the results are positive so far.

On quite a few occasions it's also interesting to see how differently the two 'players' evaluate the same position. For example, throughout the middlegame of game 13, Stockfish has consistently rated its advantage from 0.4 to around 0.9, while Rybka considers the position almost equal, with a very slight advantage for Rybka. Of course we don't know why the evaluations are so different, but again, I think it's heartening for the future of chess: it's an indication that players at this level can still 'think' differently about the game, and be successful.

That's not to say that the match has been without its dull computerish moments: the final 30+ moves of game 11 were utterly pointless (although even here it's a little interesting that Stockfish seemed much more aware of the drawn position).

[Dennis Monokroussos]

I've been following the match with some interest as well. One slightly negative comment: can Mr. Thorsen really claim these are positions "with the most normal GM openings"? There were a couple of normal-ish Caro-Kanns with the Short System in the Advance Variation, but most everything else is at best in the second-tier range (e.g. the Blumenfeld Gambit, 2.Bg5 vs. the Dutch, the Snake Benoni, the Alekhine, etc.). No Slav, Semi-Slav, Najdorf or other Open Sicilian, no Ruys, no Nimzos, etc. On the one hand, I can see that the goal is to avoid issues with each computer's book getting in the way. On the other hand, it detracts from the interest in and value of the match if the computers keep getting into positions that are of little to no interest and relevance to human chess.

I realize it's early in the match and those openings are likely to arise. Hopefully when they do, they'll at least go far enough into theory that even if they depart before getting into trendy contemporary lines, the positions will at least have something to do with what human beings play.

[Bno]

@Dennis Monokroussos, I think the "2.Bg5 vs. the Dutch" (game 11) was the choice of Stockfish. The players were already out of 'book' after the first move. You can tell that by the evaluation being shown after the second move. The book is also available for download.

[JC]

@DM
Agreed on the openings - I'd expected the games to start after maybe 5 to 10 moves, rather than 2 to 5. Not only would this make the games more relevant to high level human play, but it'd also tend to give objectively higher level openings: it's not as though the lines the top GMs play are played for quirky/psychological reasons (mostly) - they'll all have been examined by Rybka/Stockfish/... for much longer than five minutes per move.

Still, I'm sure there'll be another such match soon enough, even if the opening theory depth isn't ideal in this one from some points of view - it's not as though Rybka/Stockfish will have other commitments.

Any ideas on what positions would make it into the 24 you'd ideally like to see in such a setup? Perhaps Martin Thoresen would be happy to have some suggestions for next time. (personally, I don't really know enough to have any strong/informed/relevant opinion)

[Martin Thoresen]

Thanks for the comments everyone and thanks for making a blog entry about this match, Dennis.

As for the openings; you're all correct when you say that my comment of "most normal GM openings" might be a bit misleading.

Here's all the openings that are used in this match, the first openings is game 1-2, the second is game 3-4 and so on. After these moves presented here, there is NO book of ANY kind provided for the engines. They do the rest themselves.

I have choses short opening positions in order to test the engines in all aspects of chess - opening, middle and endgame play.

Thank you for the interest, everyone!

Best Regards,
Martin

[Event "?"]
[Site "?"]
[Date "2010.06.21"]
[Round "?"]
[White "Alekhine's Defense"]
[Black "?"]
[Result "0-1"]
[ECO "B03"]
[PlyCount "7"]
[EventDate "2010.??.??"]

1. e4 Nf6 2. e5 Nd5 3. d4 d6 4. Nf3 0-1

[Event "?"]
[Site "?"]
[Date "2010.06.22"]
[Round "?"]
[White "Benkö's Gambit"]
[Black "?"]
[Result "0-1"]
[ECO "A57"]
[PlyCount "6"]
[EventDate "2010.??.??"]

1. d4 Nf6 2. c4 c5 3. d5 b5 {
Moves after the blue 'critical opening position' are my repertoire} 0-1

[Event "?"]
[Site "?"]
[Date "2010.06.22"]
[Round "?"]
[White "Blumenfeld Gambit"]
[Black "?"]
[Result "0-1"]
[ECO "E10"]
[PlyCount "7"]
[EventDate "2010.??.??"]

1. d4 Nf6 2. c4 e6 3. Nf3 c5 4. d5 0-1

[Event "?"]
[Site "?"]
[Date "2010.06.22"]
[Round "?"]
[White "Bogo-Indian"]
[Black "?"]
[Result "0-1"]
[ECO "E11"]
[PlyCount "6"]
[EventDate "2010.??.??"]

1. d4 Nf6 2. c4 e6 3. Nf3 Bb4+ 0-1

[Event "?"]
[Site "?"]
[Date "2010.06.21"]
[Round "?"]
[White "Caro-Kann"]
[Black "Advance Variation"]
[Result "0-1"]
[ECO "B12"]
[PlyCount "6"]
[EventDate "2010.??.??"]

1. e4 c6 2. d4 d5 3. e5 Bf5 0-1

[Event "?"]
[Site "?"]
[Date "2010.06.22"]
[Round "?"]
[White "Dutch Defense"]
[Black "?"]
[Result "0-1"]
[ECO "A80"]
[PlyCount "2"]
[EventDate "2010.??.??"]

1. d4 f5 0-1

[Event "?"]
[Site "?"]
[Date "2010.06.22"]
[Round "?"]
[White "English Opening"]
[Black "?"]
[Result "0-1"]
[ECO "A20"]
[PlyCount "3"]
[EventDate "2010.??.??"]

1. c4 e5 2. g3 0-1

[Event "?"]
[Site "?"]
[Date "2010.06.21"]
[Round "?"]
[White "French Tarrasch"]
[Black "with 3...c5"]
[Result "0-1"]
[ECO "C07"]
[PlyCount "6"]
[EventDate "2010.??.??"]

1. e4 e6 2. d4 d5 3. Nd2 c5 0-1

[Event "?"]
[Site "?"]
[Date "2010.06.22"]
[Round "?"]
[White "Grünfeld"]
[Black "?"]
[Result "0-1"]
[ECO "D80"]
[PlyCount "6"]
[EventDate "2010.??.??"]

1. d4 Nf6 2. c4 g6 3. Nc3 d5 0-1

[Event "?"]
[Site "?"]
[Date "2010.06.21"]
[Round "?"]
[White "Hungarian Defense"]
[Black "?"]
[Result "0-1"]
[ECO "C50"]
[PlyCount "8"]
[EventDate "2010.??.??"]

1. e4 e5 2. Nf3 Nc6 3. Bc4 Be7 4. d4 d6 0-1

[Event "?"]
[Site "?"]
[Date "2010.06.21"]
[Round "?"]
[White "King's Gambit Accepted"]
[Black "Knight Gambit"]
[Result "0-1"]
[ECO "C34"]
[PlyCount "5"]
[EventDate "2010.??.??"]

1. e4 e5 2. f4 exf4 3. Nf3 0-1

[Event "?"]
[Site "?"]
[Date "2010.06.22"]
[Round "?"]
[White "King's Indian"]
[Black "?"]
[Result "0-1"]
[ECO "E70"]
[PlyCount "8"]
[EventDate "2010.??.??"]

1. d4 Nf6 2. c4 g6 3. Nc3 Bg7 4. e4 d6 0-1

[Event "?"]
[Site "?"]
[Date "2010.06.22"]
[Round "?"]
[White "Nimzo-Indian"]
[Black "?"]
[Result "0-1"]
[ECO "E20"]
[PlyCount "6"]
[EventDate "2010.??.??"]

1. d4 Nf6 2. c4 e6 3. Nc3 Bb4 0-1

[Event "Simetrical"]
[Site "?"]
[Date "2010.06.21"]
[Round "?"]
[White "Petroff Defense"]
[Black "Modern Attack"]
[Result "0-1"]
[ECO "C43"]
[PlyCount "10"]
[EventDate "2010.??.??"]

1. e4 e5 2. Nf3 Nf6 3. d4 Nxe4 4. Bd3 d5 5. Nxe5 Nd7 0-1

[Event "?"]
[Site "?"]
[Date "2010.06.21"]
[Round "?"]
[White "Pirc Defense"]
[Black "?"]
[Result "0-1"]
[ECO "B07"]
[PlyCount "6"]
[EventDate "2010.??.??"]

1. e4 d6 2. d4 Nf6 3. Nc3 g6 0-1

[Event "?"]
[Site "?"]
[Date "2010.06.22"]
[Round "?"]
[White "Queen's Gambit Accepted 3.e4"]
[Black "?"]
[Result "0-1"]
[ECO "D20"]
[PlyCount "5"]
[EventDate "2010.??.??"]

1. d4 d5 2. c4 dxc4 3. e4 0-1

[Event "?"]
[Site "?"]
[Date "2010.06.22"]
[Round "?"]
[White "Queen's Gambit Declined"]
[Black "?"]
[Result "0-1"]
[ECO "D31"]
[PlyCount "5"]
[EventDate "2010.??.??"]

1. d4 d5 2. c4 e6 3. Nc3 0-1

[Event "?"]
[Site "?"]
[Date "2010.06.23"]
[Round "?"]
[White "Ruy Lopez"]
[Black "Steinitz's Defense"]
[Result "0-1"]
[ECO "C74"]
[PlyCount "9"]
[EventDate "2010.??.??"]

1. e4 e5 2. Nf3 Nc6 3. Bb5 a6 4. Ba4 d6 5. c3 0-1

[Event "?"]
[Site "?"]
[Date "2010.06.22"]
[Round "?"]
[White "Réti Opening"]
[Black "?"]
[Result "0-1"]
[ECO "A09"]
[PlyCount "3"]
[EventDate "2010.??.??"]

1. Nf3 d5 2. c4 0-1

[Event "?"]
[Site "?"]
[Date "2010.06.23"]
[Round "?"]
[White "Scotch Opening"]
[Black "?"]
[Result "0-1"]
[ECO "C45"]
[PlyCount "7"]
[EventDate "2010.??.??"]

1. e4 e5 2. Nf3 Nc6 3. d4 exd4 4. Nxd4 {
Moves after the blue 'critical opening position' are my repertoire} 0-1

[Event "?"]
[Site "?"]
[Date "2010.06.22"]
[Round "?"]
[White "Sicilian Defense"]
[Black "Rossolimo Variation"]
[Result "0-1"]
[ECO "B31"]
[PlyCount "6"]
[EventDate "2010.??.??"]

1. e4 c5 2. Nf3 Nc6 3. Bb5 g6 0-1

[Event "?"]
[Site "?"]
[Date "2010.06.22"]
[Round "?"]
[White "Slav Defense"]
[Black "?"]
[Result "0-1"]
[ECO "D10"]
[PlyCount "4"]
[EventDate "2010.??.??"]

1. d4 d5 2. c4 c6 0-1

[Event "?"]
[Site "?"]
[Date "2010.06.22"]
[Round "?"]
[White "Trompowsky Attack"]
[Black "?"]
[Result "0-1"]
[ECO "A45"]
[PlyCount "3"]
[EventDate "2010.??.??"]

1. d4 Nf6 2. Bg5 0-1

[Event "?"]
[Site "?"]
[Date "2010.06.21"]
[Round "?"]
[White "Two Knights"]
[Black "Bishop Modern Opening"]
[Result "0-1"]
[ECO "C55"]
[PlyCount "7"]
[EventDate "2010.??.??"]

1. e4 e5 2. Nf3 Nc6 3. Bc4 Nf6 4. d3 0-1

[JC]

In case anyone hasn't checked the opening database being used, (or didn't notice anyway), the openings are being played in alphabetical order. So we've had:
Alekhine's Defence
Benko Gambit
Blumenfeld Gambit
Bogo-Indian
Caro-Kann Advance
Dutch
English

The next games will feature the French in some form, and it'll be a while before the Ruy or Sicilian. I'm not sure which of the remaining openings will get used (only 24 out of 48 in the database), but only one goes to move 7 (French - Classical system), and a few others to move 6. The coverage seems to be pretty reasonable to my inexpert eyes, but I guess some people would find positions after more book moves to be more interesting.

[JC]

It seems my typing is a little slow to remain relevant :). Thanks for the info Martin.

[Larry L]

I am not disappointed in the openings, hundreds of human games are played with the latest version of the Slav, Berlin, or Najdorf etc. nearly every week. Do we really want to see the computer version of Alekhine - Capblanca match playing the Queen's Gambit Declined into the ground? While it maybe very educational it becomes rather boring to someone not interested in technical nuances.
What I do find enjoyable is seeing how these super strong programs take a second rate opening like the Benko Gambit and play a very interesting game, both as black or white. It's like watching super GM's playiing club openings with the desire to fight to the end. I believe that has given the games a fresh look about them. Yes some have become boring when they were allowed to play on when a draw is the proper outcome. On the other hand how many grandmaster games end in a draw when it would have been nice to see the game played out to a more definitive conclusion?
I definitely like the idea of both programs having the same hardware, no 128 core machine vs a program on an iphone. These programs seem well matched for if Rybka 4 is 3261 as noted on the CCRL website then Stockfish 1.8 looks to be around 3250+. Insane rating numbers.
One last comment does anyone know who Martin Thoresen has lined up for the next match?

[Dennis Monokroussos]

LarryL: I didn't say the Benko Gambit was a second-rate opening. The issue was how common the openings being used were, and so when I called the Benko and the other lines second-tier the point was that they were rarer than openings like the Najdorf, Slav, Nimzo-Indian and so on. And how you're interpreting that as something like a desire to see something like 33 out of 34 QGDs as in the Alekhine-Capablanca match is completely beyond me. I didn't advocate either repetition or boring lines. (Just the opposite: among the openings mentioned I explicitly suggested open Sicilians - the Najdorf in particular - and the Semi-Slav.)

In fact, I didn't even complain about the openings; what I said was that what we're getting isn't what Mr. Thoresen advertised; to wit, we're not getting the most common GM opening positions, but something else altogether. That said, I think the match would be more interesting if we went a bit further into the openings before shutting the books off. That could be done very easily without turning it into a replica of some trendy theoretical battles (though such a battle in a Poisoned Pawn Najdorf or the Anti-Moscow Gambit in the Semi-Slav could be exciting).

[Martin Thoresen]

Dennis,

I assume you checked the openings I listed in an earlier post? There are many of them which are not yet played like the Slav, Sicilian, Ryu Lopez, QGA, QGD, Pirc and Nimzo-Indian.

For those interested, the match is now also relayed over at www.chessbomb.com which offers a live chat feature.

Best Regards,
Martin

[Larry L]

Dennis your points are well taken. Some of the games have had the look of Schlecter vs Tarrasch. I agee with you that it was misleading to say the openings would be GM fare. The reference to the Alekine - Capablance match was a bit of hyperbole (they did play a few other openings beside the QGD), but meant to illustrate the point that whether is was one or the same 10 or 12 openings being theoreticalyl "discussed" ab absurdum in many of the super GM Tournaments we forget just how many interesting openings there are . For partical players of your level and above the need to know the latest theory is just as important as the entertainment of the games. But how many Petroff's and Berlin Defenses does a club player need to see before there is more theory on these few openings than he will ever learn in a life time devoted to chess? The Najdorf's been around since the fifies and most lines are at least 15 moves deep. So while there is a treasury of great games with that opening most, if you know the theory, are now endgames. Besides if the GM's are using computers now constantly testing the latest lines all the time, what could this match contribute to that?.

[Dennis Monokroussos]

Martin: Yes, I did, but I was responding on the topic of what I had written earlier.

LarryL: Actually, it was all and only QGDs in Alekhine-Capablanca, except for the French in game 1!

About the Berlin & Petroff comment, it seems you're repeating a point I've already addressed. There are "dull" openings in the match as-is, too. You're also correct that the Najdorf is deep and getting tested on a regular basis, but this doesn't affect what I was suggesting either. Let me clarify on two points.

First, I'm all for the general format, where 24 different lines get tried out twice apiece, with each program having White once and Black once. I never objected to that, and indeed, I think the variety is good.

Second, I'm not suggesting that the lines go so far in that it becomes theoretically relevant. But I think if they go further in than they do it becomes more useful - not to professional per se but to amateurs and fans. The reason why it would be more useful to those groups is that the openings will go far enough in that it looks like something they recognize. If you look at many of the games so far, they end up looking like nothing anyone would bother looking at a second time. Very often there's no logical thread we can follow: in part because programs don't really work that way anyway, and in part because if there is a thread, the positions so quickly become foreign to us that we can't discern it. By forcing the engines to go more deeply into the kinds of structures humans get in their own games, we're more likely to see something we - not just professionals, not just masters, but ordinary amateurs too - have some grasp of.

[Dennis Monokroussos]

Btw, Larry, I don't get the "Schlechter vs. Tarrasch" quip. I guess the reference to Schlechter alludes to his being called the "drawing master" of his day and thus presumably a dry player, but that wasn't Tarrasch's reputation at all.

[Kenneth W. Regan]

I think it would be interesting to start the programs from each opening of the Anand-Topalov match, beginning with the position after the recognized novelty. That would make 24 games, alternating each side.

[Larry L]

Dennis the Schlechter - Tarrasch quick was refering to the style of play of the Classical era (1890-1920) where games often featured locked pawn formations and there was often twenty moves of endless manuevering before something happened. Game 9 of the computer match brought that to mind. However, I did a look up of all the Tarrasch - Schlechter games and 69% were drawn!
I also want to say that your points are well taken. One of the problems I am seeing as the match moves on it the uneven quality of play. Some of the openings have lead to very lively games. Others looked dull and boring and your recommendations of a better book with more sharp openings could have helped. When I get a new engine and want to see how it plays I use Nunn's test position database.
Thanks again for the great work you do to bring interesting chess to all of us.