How a Grade 11 Student Built an Autonomous Chess Robot.

‍

What happens when a student decides to build an autonomous chess robot from scratch?

For Nathaniel Mitri, a Grade 11 student at Sunny Hills High School in Fullerton, California, the answer became a three-and-a-half-month engineering challenge that combined robotics, artificial intelligence, mathematics, computer vision, and software engineering into a single project.

The result is the Autonomous Chess Robot. A system capable of detecting moves, validating game states, calculating responses, and physically moving chess pieces across a board without human intervention.

‍

Built using Jetson Infinity robotic arms, a Jetson Nano, custom inverse kinematics, and a powerful chess engine, the robot allows users to play a complete game of chess against a machine capable of analyzing the board and responding with its own moves.

It serves as an exciting application of Physical AI, representing an intelligent system that perceives its environment, reasons, and executes physical actions in the real world.

What makes the project particularly impressive isn't simply that it works. It's how it came together.

The Autonomous Chess Robot wasn't built from a step-by-step tutorial. There was no blueprint to follow. Instead, it required Nathaniel to break down a complex challenge into smaller problems, solve them one at a time, and gradually assemble them into a complete system.

In many ways, it mirrors the same engineering process used in industry.

‍

Looking for the Next Challenge

Before the Autonomous Chess Robot existed, Nathaniel had already completed another robotics project involving a robotic arm mounted to a vehicle. The project was successful, but it left him wanting more.

I like building things and making things move and work together.
‍
—Nathaniel Mitri

At the same time, he had been playing chess with his brother. One day, the two interests collided.

What if a robot could watch a game of chess, understand the board, and physically play against a human opponent?

The idea sounded difficult. That was exactly what made it appealing.

"I knew it would be hard," Nathaniel said. "But that just added to the value of the challenge."

Before committing to the project, Nathaniel asked his older brother, a computer scientist, a simple question: Was it actually possible?

The answer was yes. Difficult, but possible. That was all the encouragement he needed.

I knew it would be hard, but that just added to the value of the challenge.
‍
—Nathaniel Mitri

At the time, Nathaniel was also considering other robotics projects, including building a drone. Ultimately, he chose chess. Part of the decision came down to practicality. A drone could crash. A mistake in the software could damage expensive hardware.

A chess robot offered a safer environment to experiment, troubleshoot, and improve. More importantly, it presented a fascinating engineering problem. The technology already existed. The challenge was figuring out how to make it all work together.

‍

Five Systems. One Robot.

One of the first things Nathaniel realized was that the project wasn't really one project. It was five projects. Each system needed to function independently before they could operate together.

‍

The first component was the physical structure.

Everything - from the camera position to the placement of the robotic arms - had to be carefully arranged relative to the chess board.

The second component was the vision system.

Using a camera mounted directly above the board, the robot captures images and identifies changes between moves. By analyzing characteristics such as color, brightness, variance, and edge detection, the system determines which pieces have moved and where they have moved to.

Once a move is detected, the third component takes over.

A chess engine validates whether the move is legal and calculates the strongest possible response.

The fourth component involves inverse kinematics - —the mathematics required to determine how the robotic arm should move through physical space.

Finally, the fifth component handles communication.

A Jetson Nano processes vision, game logic, and movement calculations before communicating instructions to an Arduino Mega, which controls the robotic arms. Together, they translate software decisions into physical actions.

Each system had to work reliably on its own.

Only then could they be combined into a fully autonomous platform.

‍

The Problem that Refused to Cooperate

Of all the challenges Nathaniel encountered, one stood above the rest.

Inverse kinematics.

The goal sounds simple.

Move a robotic arm from one square to another.

In reality, it's much more complicated.

The arm has multiple joints, multiple possible movement paths, and numerous opportunities to collide with surrounding pieces.

Nathaniel initially explored several approaches.

One involved creating predefined positions for every square on the board.

Another relied on existing inverse kinematics libraries.

Neither felt right.

The predefined positions became difficult to maintain whenever hardware changed.

The existing libraries occasionally produced movements that risked colliding with pieces on the board.

Weeks passed.

Solutions were tested.

Abandoned.

Rebuilt.

Then came a breakthrough.

Instead of treating the problem as a fully complex six-dimensional robotics challenge, Nathaniel realized much of the movement could be simplified using geometry.

Suddenly, the impossible became manageable.

This problem that seemed super hard is literally just basic geometry in its roots, and a little bit of algebra. I thought that was so cool.
‍
—Nathaniel Mitri

By breaking the challenge into smaller pieces, Nathaniel was able to simplify much of the system while maintaining precise control over the robot's movements.

The solution wasn't just effective.

It was elegant.

And it revealed one of the most important lessons in engineering:

Sometimes the best solution isn't the most complicated one.

‍

Building Like an Engineer

One of the most interesting aspects of the project is how closely Nathaniel's process mirrored professional engineering teams.

Rather than attempting to build everything simultaneously, he worked on individual subsystems independently.

The vision system was developed first.

Then the inverse kinematics.

Then the communication systems.

Only after each module functioned reliably did he begin integrating them together.

This approach allowed problems to be isolated and solved before they affected other parts of the project.

The process also involved frequent redesigns.

Nathaniel estimates he adjusted the physical layout numerous times throughout development.

Moving a robotic arm a few centimeters often meant recalibrating portions of the system.

"It was so annoying," he joked. "Every time I had to take apart the whole build and move everything."

Yet these iterations ultimately made the final system stronger.

Each adjustment provided new information.

Each failure revealed something worth improving.

‍

When Theory Meets Reality

One of the biggest transitions for many students entering engineering is learning how to apply theoretical concepts to real-world systems.

The Autonomous Chess Robot required exactly that.

Concepts from geometry, trigonometry, software engineering, computer vision, electronics, and robotics all had to work together.

In the classroom, these topics are often taught separately.

In the real world, they rarely stay separate.

A camera affects software.

Software affects hardware.

Hardware affects mechanics.

Mechanics affect reliability.

Every decision influences something else.

For Nathaniel, the project became an opportunity to bridge that gap between theory and application.

What started as a chess robot gradually became a lesson in systems engineering.

‍

Testing the Competition

Once the robot was complete, friends and family naturally wanted to play against it.

The results were not encouraging.

For the humans.

Nathaniel integrated a highly aggressive chess engine capable of evaluating positions and selecting strong responses.

Despite numerous attempts, nobody has managed to defeat it.

"None of us have ever even come close to beating it," Nathaniel laughed.

To be fair, the robot has an advantage.

Unlike most human players, it never gets tired.

It never loses focus.

And it doesn't panic under pressure.

‍

Unexpected Attention

When Nathaniel first began the project, he wasn't thinking about interviews, articles, or recognition.

He simply wanted to build something interesting.

"I thought I'd finish the project, put it on my portfolio, and learn a couple of cool things."

Instead, the project attracted attention from educators, robotics enthusiasts, and industry professionals.

The response surprised him.

"I wasn't expecting any of this."

For Nathaniel, however, the attention isn't really the point.

The learning is.

The project provided an opportunity to tackle difficult problems, develop new technical skills, and gain experience building a complex system from the ground up.

Those lessons will remain valuable long after the final chess match is played.

‍

What's Next?

Although the Autonomous Chess Robot is complete, Nathaniel has already started working on his next challenge…

Chess engines.

Nathaniel is currently developing a custom chess engine in Python and experimenting with advanced optimization techniques such as alpha-beta pruning and transposition tables to improve performance. Eventually, he plans to migrate the project to C++ for additional speed and efficiency.

The project allows him to continue exploring many of the same concepts that made the Autonomous Chess Robot so interesting in the first place.

Complex systems.

Difficult problems.

Creative solutions.

Even if you don't finish it, you're going to learn so much from trying.
‍
—Nathaniel Mitri

‍

Advice for Future Roboticists

When asked what advice he would give other students interested in robotics and artificial intelligence, Nathaniel's answer was surprisingly simple.

Choose something you genuinely care about.

The difficult moments are inevitable.

The motivation to work through them comes from building something that excites you.

If it's something you love doing and something you think is really cool, you're going to get over all of the hurdles eventually.

—Nathaniel Mitri

That mindset helped transform an ambitious idea into a working robot.

And while the Autonomous Chess Robot may be capable of winning games, its greatest achievement might be something else entirely.

It demonstrates what can happen when curiosity, persistence, and a willingness to tackle difficult problems come together.

For one Grade 11 student in California, that journey started with a game of chess.

Three and a half months later, it ended with a robot that could play the game back.

‍

Watch the Autonomous Chess Robot in Action

Timelapse video of the Autonomous Chess Robot playing a complete game of chess.

‍

Technology Highlights

• ♟️ Computer vision for chess piece detection
• 🤖 Custom inverse kinematics for Jetson Infinity robotic arm control
• 🧠 Chess engine integration and move validation
• ⚙️ Real-time communication between a Jetson Nano and Arduino Mega
• 📷 Automated board state analysis and move tracking
• 🏫 Developed by Nathaniel Mitri, Grade 11 Student, Sunny Hills High School (Fullerton, California) - partnered school with Jetson Infinity