In August 2019, Microsoft reported that a program had reached 10 on Tenhou. Suphx, short for Super Phoenix, came out of Microsoft Research Asia and got there in four months and more than 5,000 games, by Microsoft's own account. About 180 people had ever held that rank.
Seven years later the headline is the least useful part. The strength reached ordinary players as two review engines, NAGA and Mortal.
What Suphx did on Tenhou
The paper, "Suphx: Mastering Mahjong with Deep Reinforcement Learning", came out in March 2020. Tenhou's room is open to humans only, 7 dan and up, so Suphx played in , the room for 4 dan and above. There it played 5,760 games, reached 10 dan, and held a stable rank of 8.74 dan.
Stable rank is the number to look at. The dan on a badge is a peak, what the paper calls your record rank. Stable rank is Tenhou's long-run average, built from placements alone: five times your 1sts plus twice your 2nds, divided by your 4ths, minus two. The Tenhou rank calculator runs it on your own placement rates.
For the human row, the paper pooled every 10-dan player's Tokujou games after reaching 10 dan into one player, since most of them live in Houou.
| Player | Games | Record rank | Stable rank |
|---|---|---|---|
| Bakuuchi (University of Tokyo) | 30,516 | 9 dan | 6.59 |
| NAGA (Dwango Media Village) | 9,649 | 8 dan | 6.64 |
| humans with 10 dan, pooled | 8,031 | 10 dan | 7.46 |
| Suphx | 5,760 | 10 dan | 8.74 |
About two dan clear of the earlier bots, more than a dan clear of the pooled humans, and above 99.99% of Tenhou's officially ranked players.
How it learned to play like that
Suphx started by copying humans. Trained on Tenhou logs, its discard model matched the human discard 76.7% of the time and its model 85.7%. Then it played itself, with reinforcement learning, for about 2.5 million games. Three ideas from the paper made that work.
Global reward prediction. One hand's score is a bad teacher. Suphx trained a predictor for the final game reward and judged each hand by that. A 1,000-point win that locks in 2nd place in South 4 is a great hand. A that leaves you in 4th isn't.
Oracle guiding. During training, a version of the agent that could see every hidden tile guided the learner, and the peek was withdrawn step by step until the agent played on what a human can see.
Run-time policy adaptation. When a round starts, the agent simulates the hidden tiles from its own and tunes its policy for this round: harder with a good hand, tighter with a poor one. Tenhou's time limit per action kept this out of the online test, so the 8.74 came without it.
Two figures in the paper show the result. In one, Suphx keeps a safe tile most humans would discard at once, taking a slower hand against a riichi it can't see yet, and drops that tile when the riichi comes instead of breaking its hand. In the other, needing more than 12,000 to leave 4th, it passes up the easy cheap win for the tile that wins less often but wins enough. One kept through the middle turns, and the score you need over the hand you have: the Tokujou-to-Houou curriculum in two pictures.
NAGA and Mortal: the strength you can run on your own logs
NAGA was already in the table above. Dwango Media Village built it on convolutional neural networks and later turned it into a paid review service: from ¥550 a month, a Niconico account, a Japanese-only site. Its summary gives three numbers, 類似度 (similarity), 悪手率 (bad-move rate), and 一致率 (match rate), and all three cover discards only, so every riichi, call, and decision is left out.
Mortal is free and open source, written by Equim and built for Tenhou's standard ranked four-player rule. You run it through mjai.ekyu.moe: paste a Tenhou or Mahjong Soul log link, keep the balanced 4.1b network, and the review comes back in seconds. only; results stay up for 15 days.
What the engines settled
Run a few hundred hands through either engine and three answers keep coming back. The Houou human data agrees with each.
Riichi the . A declared at turn 8 wins 52 to 57% of the time in Houou logs, a 65 to 66%, and each turn you wait costs a two-sided wait 3 to 4 points. The stick, the , and buy back more than the few points a quiet hand gains. The exception is the one the data marks out: honor waits gain the most from riichi, and a big two-sided hand late in the game can afford to stay .
Push a real , fold a fake one. Against a riichi at turn 8, a clean fold costs a non-dealer about 1,120 points, and one deal-in costs about five clean folds. A two-sided tenpai chase beats folding by about 3,300 points; a narrow wait needs a hand worth about 2,000. The engines don't fold a tenpai because someone said riichi, and they don't push a 1,000-point into it either.
Call when the hand is cheap anyway. A fast 1,000 that ends the dealer's turn or locks a placement is a good hand, and the engines call for it far more often than closed-hand purists do.
What they haven't settled
The person across from you. Mortal's FAQ lists what the engine ignores: each player's level, historical stats, the thinking time of each move, and the lobby type. It plays the table, not the player.
Live-table tells. Timing, tile placement, the pause before a comes out: none of it exists in a log. The reads that separate a live-table pro from a 7-dan never reach the log; that's the argument of mahjong statistics vs Japanese riichi players.
The rules. Mortal is built for Tenhou's standard ranked rule, and its FAQ's example of the point split is 90, 45, 0, and -135: the 7-dan Houou hanchan table. Tenhou's 4th-place penalty grows with every dan, -120 at 6 dan and -180 at 10 dan by the paper's table, while the rewards for 1st and 2nd stay flat. Every push and fold the engine makes is priced on that table. A tournament with a flat , a parlor with chips, a league without red fives: the right play moves, and the engine doesn't know it moved. That's why Tenhou habits don't transfer to a live table.
Use Mortal as a mirror, not an oracle
Tick "Show rating" on mjai.ekyu.moe and the page prints one score for the whole game. The note beside the box is the most honest line on any mahjong site.
Rating is a commonly misused value with high variance. Use with caution.
It goes on: the rating was never used in training, it has no calibration or confidence intervals, and the author worries that it "is diverting everyone's attention from more important matters - those individual wwyd scenes."
Match rate has the same problem in quieter form. Every decision counts once, so a turn-2 honor cut weighs the same as the South 4 push that decided your placement. You can raise the number by agreeing on easy tiles and still lose the game in the two spots that mattered.
Here's the routine. After each hanchan, paste the log, leave the rating off, and open two hands: the one you dealt in on, and the one that decided your placement. Decide your tile again before you reveal the bars, and write one line on why. Three decisions a game, not 200. When the engine's tile still makes no sense after a minute, put that hand on a list for a human review.
Suphx needed 5,760 games to show an 8.74. Put your own placement rates into the Tenhou rank calculator and see what dan they hold. Then fix tonight's three hands. Slower than 2.5 million games of self-play, and the only curriculum open to you.
Was this article helpful?
