Introduction
Hello, I am Tobias Dean, a senior computer science student at Earlham College. This page serves as a collection of the final deliverables created by my capstone project. Over the course of a year of research, split by a semester in Germany, I researched and attempted to implement techniques for an algorithm to automatically parse the most important data for an AI from a UI intended originally for human use. Thank you for your interest in my work, I hope you can learn as much from it as I did.
Abstract
There is no denying that AI, Artificial intelligence, is a versatile technology with countless applications, but this versatility is far from its full potential. One of the greatest limitations to modern AI is its need for carefully formatted data, both to train and make decisions.
The below document covers my research and work on developing an algorithm to parse data from the Tetris UI, user interface, into AI digestible data. This project aimed to give insight into techniques for automatically extracting data from human digestible UIs into one consistent format for AI use.
The python algorithm produced, Packing Coordinator, uses a combination of ImageMagick, scikit-image, and Keras to produce an abstract representation of the current game state. This algorithm uses images of the game UI as input allowing UIs built with only humans in mind to be read by AI. Although further work and testing are needed for stronger conclusions, initial results seem promising as the algorithm successfully captured the game state from screenshots under ideal conditions.
Software Architecture Diagram
GitHub
The files for this project can be found on the git hub page here: https://code.cs.earlham.edu/tadean21/Capstone.git
Demonstration Video
Poster
