Sunday
Oct172021
Chess Tools: ChessCoach?!
Sunday, October 17, 2021 at 1:03AM
I saw this linked a week or so ago, but haven't tried it out myself. Looking at the commentary on the page, it doesn't seem that the natural language commentary is particularly useful, but (a) that may not be representative of what the software is capable of, and (b) the implementation of the concept is bound to improve. Has anyone tried it?
Reader Comments (2)
Interesting; my friends at Lichess (where I volunteer) had similar questions while being amused by some of its quirky commentary in endgame positions. I find (b) more interesting to discuss than (a)... thanks for asking!
Over the years, I tried a few products with "English analysis" so I think eventually a neural network trained for long enough on powerful enough hardware with high-quality feedback from real players could catch up with and maybe surpass existing technology:
* Kasparov's Gambit (EA, 1993) includes a "coach" and sometimes after your move, one of a few canned video clips run with Garry either commenting on the accuracy of your play or warning you to be careful. Charming, but perhaps of greater interest was the constantly animating analysis board where you could see what the opponent is calculating.
* Power Chess (Sierra, 1996) featured a "coach" that after each game against a rival (which gradually increased in strength) would tell you which moves are bad, and struggle to explain (stringing together English sound bytes) why they are bad.
* ChessMaster 10th Edition (Ubisoft, 2004) allows a player to request analysis for a game, and the English written commentary to my surprise seems useful for beginner and maybe some intermediate players. After this product I assumed that there isn't enough consumer demand for significantly better products tailored to advanced amateur players.
* Lichess post-game "Learn From Your Mistakes" button doesn't provide English commentary, but years ago was surprisingly insightful. Many chess products grade severity of mistakes based upon changes in evaluation, for example dropping a rook in an equal position would be considered a blunder, but dropping a rook in a losing position would equally be considered a blunder. Based upon others' research with Stockfish, we use a sigmoid function to convert changes in evaluation to changes in "winning chances"... I think a neural network somehow motivated by player feedback could eventually make** this same discovery about which moves are mistakes, and maybe someday offer insightful English commentary.
I'm not expecting (b) to improve anytime soon, but then again I didn't expect DeepMind's AlphaGo (training cost: $35 million according to Wired) to be feasible either! I am curious what other players think.
**or the developer could transform the input data so the neural network motivated by player feedback doesn't need to learn to do this
it's cool but from a technical perspective, it's just fluff and implementation of what's already been done.
I don't even think the purpose of the English description is to be helpful. I think it's to sound natural. For example, there isn't a need for the conversational style of, "I'm not sure what white was thinking here, maybe nd7, blocking the file".
Making these kinds of plain English commentaries relies on understanding motifs like open files, doubled rooks and passed pawns. So maybe you collate your known features with simultaneous shifts in evaluation and those features and you take note of when someone misses a mate in x, and you note big swings in evaluation etc. Now you apply your template language to it and you have some kind of natural language translation.
The point is, it relies on understanding motifs and shifting evaluations. Well, good luck with understanding features and motifs in the middle of a neural network. it's generating it's own internal motifs inside a black box so I actually think it's more likely the natural language generator is independent of the playing engine. If it really is internal to the NN then those motifs have to be prebuilt in. It won't invent language for new concepts for you.
Anyway, that's all to say, this is cool, but it is still just an amusement.
[DM: That seems like a plausible diagnosis of what's going on, but it would be good to have confirmation from someone who has tried it out. If it is more sophisticated than the visible excerpt suggests, however, it's not very good advertising on their part to give such an insubstantial example of what the program can do. All the more reason to think our suspicions are correct.]