Making real games virtual: Tracking board game pieces

The same game is often played in real and virtual worlds. We integrate in-person and on-line playing of board games such as Go, bringing the real world into the virtual world. A player may record an in-person game by placing their camera on the table next to the game board, taking photos of the game. After automatically detecting the board and playing pieces, we perform inference on the time series of detections to eliminate errors and accurately estimate long sequences of moves. The game transcript may be studied afterwards, shared with friends and teachers, or added to online compilations, bringing the attendant benefits of online game play to an in-person game.