How I Hacked My University’s Registration System with Python and Twilio

University on a beautiful day.

University students know the pain of trying to register for a class only to realize it’s full. At my university we didn’t even have a waitlist system for most classes. We had to resort to logging in and checking the site multiple times a day. This seemed like something a computer could do, so I set out to automate it with a bit of Python and the Twilio API.

Getting Started

Because the university’s course registration system is behind a password login, we’re going to use a simplified site I set up. For the purposes of this demo, it alternates each minute between having no open seats in CS 101 and having one open seat.

We’re going to use a few libraries to help us with this project. Assuming you already have pip installed, go ahead and install them by running the following pip command:

We’ll dive into using each one of these libraries as we get further along.

Scraping the Registration System

We need to write a program that can determine whether there are seats available in a given course. To do this, we’ll use a technique called web scraping, in which we download a page from the internet and find the important bits. Two popular libraries that make this easy are Requests and BeautifulSoup. Requests makes it easy to get a web page, and BeautifulSoup can help us find the parts of that page that are important to us.

The meat here is in the get_open_seats function. In this function, we use requests.get to download a page’s HTML source, then we parse it with BeautifulSoup. We use find_all('tr') to get all the rows in the table, updating the courses dictionary to indicate the number of seats available in a given course. find_all can be used more powerfully than this simple example, so check out the documentation if you’re interested in learning more. Finally, we return the courses dictionary so that our program can look up how many seats are in a given course (i.e. courses['CS 101'] is the number of seats available in CS 101).

Hooray, now we can determine whether a course has open seats. A great way to test this function out is in the Python interpreter. Save this code into a file called scraper.py, then run the script and drop into interactive mode to see what this function does:

While this is great, we’re not quite to a solution; we still need some way to notify users when a seat opens up. Twilio SMS to the rescue!

Getting Updates via SMS

When building a user interface, we want simple things to be simple. In this case, users want to get notified when seats in a course open up. The simplest way for them to communicate that intent to us is sharing the course number. Let’s implement a subscription functionality by setting up and handling a webhook. I’m choosing to use Redis (a tool which provides data structures that can be accessed from multiple processes) to store subscriptions.

Here we use a web framework for Python called Flask to create a little service that handles SMS messages. After some initial setup, we indicate that requests to the /sms endpoint should be handled by the handle_sms function. In this function, we grab the user’s phone number and the course they were looking for, and store them in a set named after the course.

This is great as far as capturing the subscriptions, but it is a frustrating user interface because it doesn’t provide any feedback to users. We want to get back to users and tell them whether we’re able to service their request as soon as possible. To do that, we’ll provide a TwiML response. The additional lines needed for that are highlighted below.

We’ve made two major changes in the code above. First, we validate that the user is asking for a valid course. Second, we respond to users when they ask for updates. In the respond function, we construct a TwiML response to a given number with a given message.

Make sure to install Redis and start it up with the redis-server command. Save the above code into a file called sms_handler.py and then run python sms_handler.py.

Admittedly, the response messages here are a bit silly, but I was surprised to see how much users enjoyed them. In some contexts a personal touch can make for a better user experience.

Let’s extend our earlier scraping script to actually notify those people now that we know who wants to be notified of a course opening up.

We can run this scraper on a one-off basis to test it by running python scraper.py.

Keeping Tabs on Courses with a Cron Job

While simplifying the process of checking the course registration site into a single script is nice, we want the script to automatically run every few minutes. This problem is easily solved by using Cron. We can add a task to run every three minutes by running crontab -e and adding the following line:

With that code in place, the Cron daemon will run our scraper every three minutes. We can see the scheduled tasks by running crontab -l. And that’s it! We can subscribe to updates for a class and get back to the important things in life. As a fun side benefit, your friends will be very appreciative when you get them into that packed “Rest and Relaxation” course. While getting into the classes I wanted was plenty of reward for the work, it also ended up helping around a dozen people get their ideal schedules.

Using the techniques from this post, you can set up notifications for a wide variety of things. For instance, someone used Ruby and Twilio to track craft beer availability. To get all the code from this post, check out this gist. You can also contact me via:

Disclaimer: Make sure to check that setting up notifications will not violate your university’s student system terms of service. When in doubt, ask someone who knows!

  • this is not “hacking”.

    • Mitch

      It sure is.

      This is using the definition of the term ‘hack’ as a quick fix.

      • “Figured out a hack” is different from “I hacked it/hacking”.

        • Chivster

          I hacked it/hacking is the act of figuring out/implementing a hack. Security vulnerabilities are a subset of hacking, not the definition.

        • Mr.Sabbath

          Whats really interesting is your a mechanical engineer, which must have been a lot of schooling, but can’t read.
          “Because the university’s course registration system is behind a password login, we’re going to use a simplified site I set up.”
          He doesn’t need CS101, the code on this site is “forked” from his actual code to work with a site you can actually access.

          • Imo, You don’t have much of a sense of humor or you’re hell bent of getting every one to agree with you. The second part of my comment hints more at the education system vs skills that kids already have.

    • Hey Sandeep, you are absolutely correct that this application doesn’t take advantage of a security vulnerability. The word hack refers a bit more to the original definition of a hack as a creative shortcut: http://www.catb.org/jargon/html/meaning-of-hack.html

    • Kanna-chan!

      Two kinds of people.

  • Kyle Wagner

    My Python’s a little rusty–and was never great to be honest. Should the “soup” variable in the get_open_seats function be defined globally (or returned)? The way get_open_seats is currently written, it is a fruitless function that creates a variable used outside of the function’s scope (global scope, I believe). I know Python handles scope a little differently than some other languages (e.g. Javascript). Thoughts?

    • Kyle, you may feel a bit rusty with Python but you were spot on- there was an issue with the spacing on that function. I’ve updated the post so it matches Samuel’s correct code.

      Thanks for the assist on this one. Mind sending me a quick email makai@twilio.com so we can ship you a Twilio t-shirt?

  • Dev.

    It’d be more useful if it signed up for you.

    • And take the class for you too!

      My guess is that if the pages for the course registration were so hard to see that Samuel needed a script to notify him that they were open, then they probably weren’t available to spend the time to build a script to fill them in. At least this method did get around a dozen people onto the courses they wanted, that’s a great result!

      • Dev.

        It’s useful, but not fully automated which should be the goal with technology imo

        • It’s a hack and it served its purpose. I guess if you have a similar issue then you are welcome to take this code on and fill it in to complete the sign up for you too. I’m impressed with Samuel’s work and generosity to share this with his fellow students. 🙂

  • TwoCents

    Nice idea ,but really, in 2017 people still using sms? I know it’s twilio blog but it looks to me this would work way better as a messenger chat bot.

    • Mark

      Jajaja do you even understand the difference between cell network and the internet. Take a few more years in school buddy

      • Ben

        Acutally, Mr Mark, from a US consumer standpoint the answer is very little. So I’d cool your heels on the patronizing.

    • In 2017 people can certainly still receive SMS messages and it is a viable notification platform. I also think Facebook actually restricts notification boths quite a bit and SMS is a great option in this case. For a start, as this post shows it is easy to send SMS messages using Twilio and they will definitely be able to alert a user who has signed up to SMS alerts.

      On the other hand, building a Messenger bot would mean a bunch more hoops to jump through. You have to set up and verify your bot’s webhook
      (https://developers.facebook.com/docs/messenger-platform/guides/quick-start). You have to make sure you sign up your bot as a subscription service so that it has permission to send unprompted messages (https://developers.facebook.com/docs/messenger-platform/policy-overview#subscription_messaging).

      And I think most importantly, your bot would need to go through the bot approval process (https://developers.facebook.com/docs/messenger-platform/app-review) before any of your friends could use it.

      For a short term hack, I definitely think sending SMS with Twilio would be more straightforward and have the same result than setting up a Messenger bot. 🙂

      • Marlysson Silva

        Why you don’t try Telegram not?
        And you can to respond it..

  • cooganb

    This project has been really tough to replicate. There are some missing pieces, such as how to expose the redis server to the Twilio API, and even straight-up cloning the git project is still giving me errors.

  • Larry Meech
  • roy todd

    OKAY. let me drop this here i had troubles hiring a programmer to get into my school website and register some courses i wasn’t able to register but it was very difficult to find myself one, some friends spoke to me about a coder who could help me,he did so pretty well and was amazed by how excellent his results was..
    yall can contact him on this email I am gon drop below and tell him activation code 2323 referred you to him.

    email: deeppandahack AT gmail DOT com