Chess Robot

A robot that plays physical chess — moving real pieces on a real board — built from scratch as a BSc thesis project at KTH with a classmate. The robot can play against a human or run Stockfish vs. Stockfish autonomously.

How it works

Every move goes through a full pipeline:

1. Camera capture — a Raspberry Pi Camera Module V2 mounted above the board takes an image after each turn.
2. Vision pipeline — OpenCV detects the board’s inner corners, extrapolates to outer corners, and applies a perspective warp to correct for the camera angle. Each square is then sampled at its centre pixel and classified by RGB values to determine occupancy and piece colour.
3. FEN generation — the board state is encoded into Forsyth-Edwards Notation and handed to the Stockfish engine.
4. Move selection — Stockfish evaluates all legal moves, scores each one, and returns the best. The code handles mate-in-N detection, pawn promotion, and captures.
5. Inverse kinematics — the target square’s XY coordinates are converted into joint angles (α and β) for the two-link SCARA arm using the standard IK equations.
6. Motor control — stepper motor step counts are calculated from the joint angles and driven via GPIO. The Z-axis is a servo-driven rack-and-pinion. An electromagnet (toggled through an NPN transistor) picks up and drops pieces, which have steel screws embedded in their bases.

Hardware

The arm is a SCARA (Selective Compliance Articulated Robot Arm) designed in Solid Edge and 3D-printed on an ANET A8. Two NEMA 17 stepper motors drive the XY plane via timing belts; a continuous servo drives the vertical gear rack. Everything runs off a Raspberry Pi 4 with an external 400W PSU for the motors and electromagnet.

Chess pieces were custom 3D-printed in high-contrast colours (red and blue) and made uniform in height to keep the vision pipeline simple.

Results

Arm positioning achieved 100% accuracy in isolated Z and XY tests. Full-game reliability improved significantly when motor speed was reduced — at the lower setting, 8 out of 10 games completed without failure, averaging 12 moves before any issue at the higher speed.

Visual recognition worked reliably down to ~15 lux (roughly a lit city street at night), where red pieces remained detectable but dark-blue pieces became hard to distinguish from black squares — a known limitation of pure RGB detection without machine learning.

What I’d do differently

• Adaptive motor speed (slow for short moves where shaking is worst, fast for long ones)
• Stiffer arm material to eliminate the flex that caused occasional Z-axis inconsistencies
• ML-based piece recognition to handle pawn promotion and lower-light conditions
• A physical button or automatic image diff to detect the human’s move, removing the need for SSH input