This website was my very first personal project. As you will see when navigating the site, the end result was to have a user-friendly “database” for all video games made for the Nintendo Switch console released in America.
This idea came from my inspiration when trying to use my console. For those who are unfamiliar with the Switch, there are two controller “halves” called Joy-Cons that can combine together to form one single controller or be used as two separate controllers for multiplayer games. While the concept is fantastic and I do thoroughly enjoy playing with these controllers, the issue arose from Nintendo’s lack of explanation on when each controller could be utilized. Various games support the single Joy-con mode, while other lack this function. This also goes for Nintendo’s specialized “Pro Controller” in which not all games are compatible.
Seeing as I wanted anyone in the world to be able to utilize my creation, I knew that I would need to make it a website with mobile functionality. Before I began constructing the website, I knew that I did not want to manually sift through all of the games and update each one. This lead me to the wonders of web scraping and the technologies that exist in this specialized realm of computer science.
I immediately found Python to be the simplest in terms of connecting to the website, parsing the links from the HTML data, and spitting out the results that I wanted. This was accomplished using BeautifulSoup and it honestly could not have been any easier. However, I quickly realized that I was going to need additional resources in order to efficiently navigate the hundreds of different links as well as create click events and fill out forms to make the process completely automated. I soon discovered the ability to use a WebDriver and had Firefox bending to my will with simple Python command in just another couple of hours. Once I saw that the program completed, I quickly glanced at the data that I had collected and realized that it still needed to be more efficient if I were to ever keep up with things.
I began work on the front end at this point and found DataTables to be the best option for the functionality that I wanted to implement. I was able to import my data using AJAX and create a really quick layout for the information provided. By this point I was able to see how much data was missing from my table and employed the help of the Reddit community to help fill in my gaps. I created a simple form at the bottom of the page that allowed users to inform me of what games were missing, fill in missing control sets, etc. By this point, the site was available to the world and was pretty usable. I had added in customer sorting plugins, the ability to move the columns around, display correctly on mobile and more.
As the project winded down and I had a steady flow of users, I made sure to initiate some additional fail-safes such as email alerts for the scraping, back-ups of all of the data, and a front-end admin page for me to manipulate data without a text editor. All in all, I am proud of my website and hope to continue making stylistic changes and code optimization as time progresses.