Chess data pipeline

Some of my code from my chess data pipeline.

Back when I was working at MLS, my boss and I used to play chess against each other a lot after hours. He quickly got into online chess, just like me. This sparked the curiosity in me to write a data pipeline to gather data from so that I could examine it.

My project can be found on my Github. Essentially, it just collects data from an API endpoint, cleans it up some, then puts the data into a PostgreSQL server. I used to have the architecture running on my Raspberry Pi Zero W, but I've now moved it to my VPS, where it runs nightly. There are a couple of reasons to move it to the VPS: better uptime, better processing power, and more local space.

I also made it able to write "newsletters". They're pretty primitive in that they only include two graphs, but at least they're dynamic. These newsletters are sent weekly through the SendGrid API. Honestly, they're pretty fun.

I've also started doing some simple analyses: mainly, I've looked at game outcomes across weekdays (since my old boss was interested in that) and I've built a simple win probability model based on Stockfish evaluation of the board position.

This is still a big work in progress, especially since I'd like to expand the analyses available through this data.