Tune in to watch the keynote — October 17 & 18 at 10am
Build the future of communications.
Start building for free

Extending Python and Flask Web Applications with APIs

-AIwKqzeW5Wm1Fy07nYVq2Wr0qxddiMwZq65vqL1QO10efIaF4kWJOTAIZ1YVFM6wxujUu54aywZUxhDXtUH3QLrAt0YyYN5X4vJjaOaKHeXS8u7AqUpDha8Vo9mAQqZPotKy0ae

Application Programming Interfaces (APIs) can be a great way to programmatically gather or distribute data. API creation really distills down to creating a set of routes that receive requests and return useful data. For example, visiting this page may be easy for a person to read, but it’s harder for a computer program to extract the information it may need. An API can return just the information a program needs.

In this tutorial, we’ll make an API of I Love Dogs, a basic Flask site I created to show cool articles and photos about different types of dogs.

The site is organized by sections the user can visit to learn about different topics. For this API, we’ll create an endpoint so that when a user pings it with a topic they want to get content resources for, it returns a list of resources.

Before we start coding, let’s see what we need to have installed.

Development Environment Setup

To make this program, you’ll need:

  • Python 3.6 (you can download this here).
  • Pip (you can download this here).
  • Optional: Python Virtual Environment (read more on this here)
  • Flask (version 0.12.2 or newer)

Once you have downloaded the above tools, you’ll need to install the following libraries:

pip3 install Flask==0.12.2
pip3 install PyYAML==3.12

In this tutorial, our RESTful API will return data in the following format:

{
  "response": [CODE],
  "results": [DATA]
}

When the API sends a response, it should also return a response code. This allows the developer to understand the result of their query before trying to use the resulting data. In this tutorial, we’ll focus on response code 200 (the query was successful and the API returned data) and 404 (the query could not find what was requested).

For those who want to skip ahead, the finished code can be found on GitHub.

To begin, fork the template repository (this includes the basic website but no API -- yet!), clone your fork locally to your computer, and then open app.py in an IDE/text editor of your choice.

git clone git@github.com:wessilfie/twilio-dog-site-template.git
cd twilio-dog-site-template/
open app.py

Topics API Endpoint

The first endpoint we will make is a topics endpoint. When queried, this endpoint will return a list of the topics available on the site. In the web app, all of the topics are saved in a dictionary in the definitions.py file which means if we return all the keys in that dictionary, we have the results we need for the API.

Add the following method in your app.py file:

@app.route("/api/topics", methods=['GET'])
def get_topics():
    return jsonify({'response': 200, 'results': list(definitions.keys())})

Now that we have our first endpoint created, let’s test it:

Start the application from your terminal:

python app.py

We’ll use the command line tool curl for testing. In a second Terminal window, query the API:

curl -i http://127.0.0.1:5000/api/topics

You should then get a response that looks like this:

HTTP/1.0 200 OK
Content-Type: application/json
Content-Length: 94
Server: Werkzeug/0.14.1 Python/3.6.4
Date: Fri, 24 Aug 2018 23:08:05 GMT

{
  "response": 200, 
  "results": [
    "corgis", 
    "samoyed", 
    "mixed_puppies"
  ]
}

Adding a Search Endpoint

Now that we have a topics API endpoint, we will make a dynamic endpoint /api/search/<topic>. By placing a topic between these <> symbols in Flask, “topic” becomes a placeholder so that regardless of whether the user queried the endpoint for “corgis” or “samoyed,” the endpoint can handle it.  

With this dynamic endpoint, we need to be prepared to handle the different ways in which the user might query for a topic versus how the web app names the topics. In this specific app, all topics are formatted in lowercase and by replacing all spaces with underscores (“_”). To deal with converting the user’s submission with this formatting, we’ll write a small method that automatically converts what the user submitted to the above rule:

#modify topic to match format in definitions.py file
def modify_topic(topic):
    #replace all spaces with underscores and lowercase result
    return topic.replace(" ", "_").lower()

We need to be able to read in the data, which are stored in .yml files. Thankfully, PyYaml offers an easy way to read in .yml data and convert it into a Python dictionary that we can easily work with. Reading in a data file and converting it into a dictionary looks like this:

@app.route("/api/search/<topic>", methods=['GET'])
def get_data(topic):
    topic_modified = modify_topic(topic)
    file_path = "data/{0}.yml".format(topic_modified)
    
    # read in file 
    with open(file_path, 'r') as yml_data:
        #convert file to dictionary
        data = yaml.safe_load(yml_data)
        return jsonify({'response': 200, 'results': data['data'][topic_modified]})

In the above code, the modify_topic function converts the user-submitted string to the app’s string format, then read in the relevant YAML file from the data/folder where they are stored, and save the data as a dictionary. The results are then returned to the user in JSON format using Jsonify, part of the Flask library.

Test out your search with the following query:

curl -i http://127.0.0.1:5000/api/search/corgis

You should see some information about corgis returned!

HTTP/1.0 200 OK
Content-Type: application/json
Content-Length: 343
Server: Werkzeug/0.14.1 Python/3.6.4
Date: Fri, 24 Aug 2018 23:13:17 GMT

{
  "response": 200, 
  "results": [
    {
      "content_link": "https://twitter.com/dog_rates/status/1004029683225292800", 
      "description": "This dog is adorable and we will always love the pupper.", 
      "social_media": "https://twitter.com/dog_rates", 
      "submitted_by": "We Rate Dogs", 
      "title": "Corgi Love"
    }
  ]
}

Handling Malicious Users and Invalid Topic Types

While the above endpoint works, it does not handle a user querying for a topic that does not exist or a data file not being found. To deal with this, we’ll add a method that will send a 404 response when the user’s request cannot be found.

def api_return_404():
    return make_response(jsonify({'response': 404, 'results': 'No data returned for this topic.'}), 404)    

We’ll next add a few modifications to the code to check if the topic searched for is available on the site and to handle the data file not being found when being read in. The updated method looks like this:

@app.route("/api/search/<topic>", methods=['GET'])
def get_data(topic):
    if len(topic) == 0 or topic is None:
        return api_return_404()
    topic_modified = modify_topic(topic)
    file_path = "data/{0}.yml".format(topic_modified)
    
    # if the topic the user queried for does not exist, return a 404 error
    if topic_modified not in definitions:
        return api_return_404()

    try: 
        # read in file 
        with open(file_path, 'r') as yml_data:
            try:
                #convert file to dictionary
                data = yaml.safe_load(yml_data)
                return jsonify({'response': 200, 'results': data['data'][topic_modified]})
            except:
                return api_return_404()
    
    #return 404 if reading file not found 
    except:
        return api_return_404() 

We can set a requirement for what data types the API accepts to handle a user trying to query the API with an invalid topic type. In this case, we want to reject any query that is not a string.

The main change is updating the endpoint formatting:

@app.route("/api/search/<string:topic>", methods=['GET'])

By adding this, we have specified that the endpoint will only accept strings as topics. More on the possibilities of this feature can be found in the Flask documentation.

Let’s test out the updated method to confirm it works for invalid topics. Again, we’ll use curl to test it:

curl -i <a href="http://127.0.0.1:5000/api/search/invalid_topic">http://127.0.0.1:5000/api/search/invalid_topic</a>
HTTP/1.0 200 OK
Content-Type: application/json
Content-Length: 343
Server: Werkzeug/0.14.1 Python/3.6.4
Date: Fri, 24 Aug 2018 23:15:17 GMT

{
  "response": 404, 
  "results": "No data returned for this topic."
}

Final Endpoints and Testing

With all of these changes, we have our final API endpoints.

@app.route("/api/search/<string:topic>", methods=['GET'])
def get_data(topic):
    if len(topic) == 0 or topic is None:
        return api_return_404()
    topic_modified = modify_topic(topic)
    file_path = "data/{0}.yml".format(topic_modified)
    
    # if the topic the user queried for does not exist, return a 404 error
    if topic_modified not in definitions:
        return api_return_404()

    try: 
        # read in file 
        with open(file_path, 'r') as yml_data:
            try:
                #convert file to dictionary
                data = yaml.safe_load(yml_data)
                return jsonify({'response': 200, 'results': data['data'][topic_modified]})
            except:
                return api_return_404()
    
    #return 404 if reading file not found 
    except:
        return api_return_404() 

@app.route("/api/topics", methods=['GET'])
def get_topics():
    return jsonify({'response': 200, 'results': list(definitions.keys())})

In Review

In this post, we took a Flask web application and extended it to have a public API. Now it will be easier for developers to use the site’s data in their own applications. While this post focuses only on offering a way for users to get data from the API, if you wanted to build on this, you could add an endpoint that allows users to submit data to be added to the site.

If you enjoyed this post, you can follow me on GitHub @wessilfie or Twitter @WillEssilfie.

Authors
Sign up and start building
Not ready yet? Talk to an expert.