Build Your Own API Analytics in Vue.js with PHP, Laravel, and Twilio Sync

Twilio Sync is a service provided by Twilio that helps you keep your application state in sync across different services. Specifically, it provides a “stack-agnostic” API that helps you add real-time capabilities to your application.

In this article, we will explore how we can build a custom real-time API analytics tool powered by Sync Lists, a Sync primitive type that helps us synchronize individual JSON objects.

NOTE: You can learn more about Sync Lists and other primitives from the Twilio Sync Object Overview.

Prerequisites

To complete this tutorial you will need the following:

Laravel CLI, Composer, and npm installed on your computer
Basic knowledge of Laravel
Basic knowledge of Vue.js
A Twilio Account
cURL or Postman installed to test our API endpoints

Creating the Sample Application

We will generate a fresh application with the Laravel installer and enter into the project directory with:

$ laravel new api-analytics && cd api-analytics

Next, install the Twilio SDK for PHP with the command below:

$ composer require twilio/sdk

Now, create a new Sync service in your Twilio console and populate your .env file with the generated service SID, together with your Twilio API credentials as follows:

TWILIO_API_KEY="TWILIO_API_KEY"
TWILIO_API_SECRET="TWILIO_API_SECRET"
TWILIO_ACCOUNT_SID="TWILIO_ACCOUNT_SID"
TWILIO_SYNC_SERVICE_SID="SYNC_SERVICE_SID"
TWILIO_AUTH_TOKEN="TWILIO_AUTH_TOKEN"

Also, remember to update your database credentials in the same file to match your local setup.

Models and Migrations

Our application will contain two models; a Recipe and Hit. They will both be used in setting the conditions and recording the API hits of our users, respectively. Create the Recipe model and its corresponding migration file with:

$ php artisan make:model Recipe -m

Now, open the generated recipes migration file and make the following changes:

<?php

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

class CreateRecipesTable extends Migration
{
        public function up()
        {
            Schema::create('recipes', function (Blueprint $table) {
                $table->id();
                $table->string('title');
                $table->text('body');
                $table->timestamps();
            });
        }

        public function down()
        {
            Schema::dropIfExists('recipes');
        }
}

Our recipes need to contain formatted texts such as lists and headers. We achieve this using Mutators in the getBodyAttribute method below i.e, the body string is stored in markdown and transformed into HTML using the CommonMark parser which comes bundled with Laravel. Replace the content of the Recipe model class, located at app/Recipe.php with the following:

<?php
namespace App;

use Illuminate\Database\Eloquent\Model;
use League\CommonMark\CommonMarkConverter;

class Recipe extends Model
{
        protected $fillable = [
            'title', 'body'
        ];

        public static function getBodyAttribute($value) {
            $converter = new CommonMarkConverter([
                'html_input' => 'strip',
                'allow_unsafe_links' => false
            ]);
            return $converter->convertToHtml($value);
        }
}

Similarly, generate the Hit model and it’s migration file with: php artisan make:model Hit -m and paste in the following in the migration file that Laravel generates for us:

<?php

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

class CreateHitsTable extends Migration
{
        public function up()
        {
            Schema::create('hits', function (Blueprint $table) {
                $table->id();
                $table->string('path');
                $table->string('method');
                $table->string('query_params')->default(null);
                $table->string('request_ip');
                $table->string('response_code');
                $table->timestamps();
            });
        }
        public function down()
        {
            Schema::dropIfExists('hits');
        }
}

Now update the $fillable attribute of the generated Hit model to the code below:

protected $fillable = [
    'path', 'method', 'query_params', 'request_ip', 'response_code'
];

Apply the migrations we have created with the artisan command:

$ php artisan migrate

Seeding our Database

To try things out, we need some dummy data in our recipe table. Create the seeder file by running php artisan make:seed RecipesTableSeeder from your terminal. Next, replace the content of the RecipesTableSeeder file at database/seeds/RecipesTableSeeder.php with the following:

<?php
use Illuminate\Database\Seeder;
Use app\Recipe;

class RecipeTableSeeder extends Seeder
{
        public function run()
        {
            Recipe::insert([
                [
                    'title' => "Nigerian Jollof Rice with Chicken",
                    'body' => "Test Recipe. Chicken is first sauteed on the stove top to produce a wonderful aromatic base for the rice.",
                    'created_at' => \Carbon\Carbon::now(),
                    'updated_at' => \Carbon\Carbon::now()
                ]
            ]);
        }
}

Next, we need to make Laravel aware of our seeder file by opening the DatabaseSeeder class file (database/seeds/DatabaseSeeder.php) and updating the run method to the following:


public function run()
        {
            $this->call(RecipeTableSeeder::class);
}

Populate the database with the dummy data by running:

$ php artisan db:seed

Building the API Controller

With our models in place, let’s create the corresponding RecipesController class. This controller class will handle all recipe-related operations in our API. Open up your terminal and run the command below in your application directory:

$ php artisan make:controller Api/RecipesController

This will create the Api folder in your Controllers directory (if it doesn’t exist) and generate the RecipesController.php there. Open the created file and add the following to it:

<?php
namespace App\Http\Controllers\Api;

use App\Http\Controllers\Controller;
use App\Recipe;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\Validator;

class RecipesController extends Controller
{
        public function index() {
            $recipes = Recipe::orderBy('created_at', 'desc')->get();
            return response()->json($recipes);
        }

        public function store(Request $request) {
            $validator = Validator::make($request->all(), [
                'name' => 'string|required',
                'body' => 'required'
            ]);
            if ($validator->fails()) {
                return response()->json($validator->errors(), 400);
            }
            $recipe = new Recipe([
                'name' => $request->name,
                'body' => $request->body
            ]);
            $recipe->save();
            return response()->json($recipe, 201);
        }

        public function show($id) {
            $recipe = Recipe::findOrFail($id);
            return response()->json($recipe);
        }

        public function update(Request $request, $id) {
            $recipe = Recipe::findOrFail($id);
            $recipe->update(['name' => $request->name, 'body' => $request->body]);

            return response()->json($recipe);
        }

        public function destroy($id) {
            Recipe::findOrFail($id)->delete();
            return response()->json();
        }
}

Using Laravel’s After Middlewares

Our core application API is ready now, but we still need to monitor all requests and responses our API is processing. We will achieve this with Laravel after middlewares. After middlewares are regular Laravel middlewares, except they perform their tasks after the request has been handled by the application. Let’s create a SaveEndpointHit middleware for this case with:

$ php artisan make:middleware SaveEndpointHit

This will generate a new file at app/Http/Middleware/SaveEndpointHit.php. Fill it with the code below:

<?php
namespace App\Http\Middleware;

use App\Events\ApiHit;
use App\Hit;
use Closure;
use Illuminate\Support\Facades\Log;
use Twilio\Rest\Client;

class SaveEndpointHit
{
        /**
         * Handle an incoming request.
         *
         * @param  \Illuminate\Http\Request  $request
         * @param  \Closure  $next
         * @return mixed
         */
        public function handle($request, Closure $next)
        {
            $response = $next($request);
            $path = $request->path();
            $method = $request->method();
            $ip = $request->ip();
            $params = json_encode($request->query());
            $code = $response->status();

            $hit = new Hit([
                'path' => $path,
                'method' => $method,
                'query_params' => $params,
                'request_ip' => $ip,
                'response_code' => $code
            ]);
            $hit->save();
            $this->updateSyncList($hit);

            return $response;
        }

        private function updateSyncList(Hit $hit) {
            $client = new Client(env('TWILIO_ACCOUNT_SID'), env('TWILIO_AUTH_TOKEN'));
            $serviceSID = env('TWILIO_SYNC_SERVICE_SID');
            $syncList = $client->sync->v1->services($serviceSID)->syncLists("api_calls");
            $sid = $syncList->syncListItems->create($hit->toArray());
            Log::debug($sid);
        }
}

Next, register the SaveEndpointHit class as a route middleware by appending it to the $routeMiddleware array in app/Http/Kernel.php.

protected $routeMiddleware = [
            ... 
            'hit.save' => \App\Http\Middleware\SaveEndpointHit::class,
        ];

At this point, our controller and middlewares are ready. Let’s proceed to create our application routes. We will be putting our API related routes in routes/api.php. That way, they’ll become accessible at APP_URL/api instead of the regular APP_URL. Open the API routes file (routes/api.php) and replace its content with the code below:

<?php
use Illuminate\Support\Facades\Route;

Route::resource('recipes', 'Api\RecipesController')
        ->only(['index', 'store', 'show', 'update', 'destroy'])->middleware('hit.save');

In the code block above, we have attached the hit.save middleware (which is an alias to the SaveEndpointHit class) to the recipe route resource. That way, every call to any of the endpoints triggers the middleware..

Setting up our Dashboard with Vue

We will now hook into our API from a Vue-powered dashboard, responsible for displaying the API calls. We achieve this by using the Laravel UI package which makes it easy to work with frontend libraries and frameworks like React and Vue.js. Install and configure the package by running the commands below in your terminal:

$ composer require laravel/ui --dev
$ php artisan ui vue
$ npm install && npm run dev

For our dashboard, we will create a new Dashboard component for viewing our API hits.

$ touch resources/js/components/DashboardComponent.vue

Next, replace the default example component with the dashboard component by replacing the content of the resources/js/app.js file with the following:

require('./bootstrap');
window.Vue = require('vue');

Vue.component('dashboard-component', require('./components/DashboardComponent.vue').default);

const app = new Vue({
        el: '#app',
});

Also, get rid of everything inside the resources/views/welcome.blade.php template file and replace its content with the following:

<!DOCTYPE html>
<html>
<head>
        <meta charset="utf-8">
        <meta name="viewport" content="width=device-width, initial-scale=1">

        <title>Overseer!</title>
        <link href="https://fonts.googleapis.com/css?family=Nunito:200,600" rel="stylesheet">
        <link href="{{mix('css/app.css')}}" rel="stylesheet" />
</head>
<body>
<div id="app">
        <dashboard-component :token="'{{ $token }}'"></dashboard-component>
</div>
<script src="{{mix('js/app.js')}}"></script>
</body>
</html>

Notice that we are passing a token prop to our dashboard component. This token represents our Twilio Sync token which is passed down from our controller (which we will create next).

Create the controller responsible for rendering the dashboard with:

$ php artisan make:controller DashboardController

The command above creates a DashboardController.php file in app/Http/Controllers. Open the file and paste in the following code:

<?php
namespace App\Http\Controllers;

use App\Hit;
use Twilio\Jwt\AccessToken;
use Twilio\Jwt\Grants\SyncGrant;
use Twilio\Rest\Client;

class DashboardController extends Controller
{
        public function showDashboard() {
            $data = [
                'token' => $this->getToken()
            ];
            return view('welcome', $data);
        }

        private function getToken() {
            $identity = "Overseer";
            // Create access token, which we will serialize and send to the client
            $token = new AccessToken(
                env('TWILIO_ACCOUNT_SID'),
                env('TWILIO_API_KEY'),
                env('TWILIO_API_SECRET'),
                3600,
                $identity
            );
            // grant access to Sync
            $syncGrant = new SyncGrant();
            $syncGrant->setServiceSid(env('TWILIO_SYNC_SERVICE_SID'));
            $token->addGrant($syncGrant);
            return $token->toJWT();
        }
}

Next, register the controller in our web routes by replacing the content of routes/web.php with the following:

<?php

use Illuminate\Support\Facades\Route;

Route::get('/admin/{any}', 'DashboardController@showDashboard');

Viewing API Calls in Real-time

Our controllers and Vue components are ready, but when you visit the dashboard URL at http://localhost:8000/admin/dashboard, it renders a blank page. This is because we have no way of populating the API calls for now. We will use the Twilio Sync SDK to fetch the hits from our Sync service and prepare them for rendering when our dashboard component is created. Install the twilio-sync package with:

$ npm i --save twilio-sync

Next, replace the content of the dashboard component file (resources/js/components/DashboardComponent.vue) with the following:

<template>
        <div class="container">
            <nav class="navbar navbar-expand-lg">
                <a class="navbar-brand font-weight-bold" href="#">Overseer!</a>
                <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarNav" aria-controls="navbarNav" aria-expanded="false" aria-label="Toggle navigation">
                    <span class="navbar-toggler-icon"></span>
                </button>
                <div class="collapse navbar-collapse" id="navbarNav">
                </div>
            </nav>

            <div class="row">
                <div class="col-12 hits-container mt-5">
                    <table class="table table-hover">
                        <thead>
                        <tr>
                            <th scope="col">#</th>
                            <th scope="col">Timestamp</th>
                            <th scope="col">Path</th>
                            <th scope="col">Method</th>
                            <th scope="col">Request IP</th>
                            <th scope="col">Status</th>
                        </tr>
                        </thead>
                        <tbody>

                        <tr v-for="hit in endpointHits" :key="hit.id">
                            <th scope="row">{{hit.id}}</th>
                            <td class="text-muted">{{ hit.created_at }}</td>
                            <td class="font-weight-bold">{{ hit.path }}</td>
                            <td>{{ hit.method }}</td>
                            <td>{{ hit.request_ip }}</td>
                            <td v-if="Number(hit.response_code) <= 202" class="text-success font-weight-bold">
                                {{ hit.response_code }}
                            </td>
                            <td v-else-if="Number(hit.response_code) <= 308" class="text-info font-weight-bold">
                                {{ hit.response_code }}
                            </td>
                            <td v-else-if="Number(hit.response_code) <= 418" class="text-orange font-weight-bold">
                                {{hit.response_code}}
                            </td>
                            <td v-else class="text-danger font-weight-bold">
                                {{hit.response_code}}
                            </td>
                        </tr>
                        </tbody>
                    </table>
                </div>
            </div>
        </div>
</template>

<style>
        body, .navbar {
            background-color: #e3e3ec !important;
        }
        thead th {
            border-top: none;
        }
        .hits-container {
            background-color: #fff;
            border-radius: 3px;
        }
        .hits-container td {
            padding-top: 0.90rem;
            border-top: 1px solid #f1f1f1;
        }
        .text-orange {
            color: #f6993f;
        }
</style>

<script>
        import SyncClient from 'twilio-sync';
        export default
        {
            props: ['token'],

            data() {
                return {
                    endpointHits: [],
                }
            },

            methods: {
                updateHitsList(items) {
                    items.map((item) => {
                        this.endpointHits.push(item.data.value);
                    })
                }
            },

            created() {
                let token = this.token;
                let syncClient = new SyncClient(token, {logLevel: 'info'});

                syncClient.on('connectionStateChanged', (state) => {
                    if (state !== 'connected') {
                        console.log("Sync not connected");
                    }
                });

                syncClient.list("api_calls").then((syncList) => {
                    syncList.getItems().then(endpointHits => {
                        console.log(endpointHits);
                        this.updateHitsList(endpointHits.items);
                    });

// this is called when new item is added to our list (whether remotely or locally)
                    syncList.on("itemAdded", (args) => {
                        this.endpointHits.unshift(args.item.data.value);
                    });
                });
            },
        }
</script>

Testing

We’re now ready to test the application. Your files will need to be compiled by running the following command:

$ npm run dev

The server will also need to be started in a new terminal:

$ php artisan serve

Now visit (or refresh) the dashboard URL at http://localhost:8000/admin/dashboard in your browser and make some API calls with curl like so in a separate terminal:

$ curl -v -i GET http://localhost:8000/api/recipes

You should see subsequent requests being rendered in real time.

Analytics dashboard

Monitoring Application Crashes

Because our middleware depends on successful responses, it fails in cases where exceptions are thrown before Laravel can finish processing the request. To handle such cases, open the exceptions handler at (app/Exceptions/Handler.php) and create a new private method responsible for logging the errors as in the code block below:

private function logError($request, $response) {
            $hit = new Hit([
                'path' => $request->path(),
                'method' => $request->method(),
                'query_params' => json_encode($request->query()),
                'request_ip' => $request->ip(),
                'response_code' => $response->status()
            ]);
            $hit->save();
            $client = new Client(env('TWILIO_ACCOUNT_SID'), env('TWILIO_AUTH_TOKEN'));
            $serviceSID = env('TWILIO_SYNC_SERVICE_SID');
            try {
                $syncList = $client->sync->v1->services($serviceSID)->syncLists->create([
                    "uniqueName" => "api_calls"
                ]);
            } catch (RestException $e) {
                $syncList = $client->sync->v1->services($serviceSID)->syncLists("api_calls");
            }

            $syncList->syncListItems->create($hit->toArray());
        }

Next, call the method above in the render method of the Handler class as shown:

public function render($request, Throwable $exception)
        {
            $response = parent::render($request, $exception);
            $this->logError($request, $response);
            return $response;
        }

Build your files again with npm run dev and start your Laravel server if it isn't running already and visit http://localhost:8000/admin/dashboard. You should see all API requests being displayed on your dashboard - whether they were completed or not.

Conclusion

Twilio Sync provides a powerful and flexible API that helps us add real-time capabilities to our applications, and in this article, we have seen how it can be used to power a simple API analytics tool. You can find the complete source code on Github, and if you have an issue or a question, feel free to create a new issue in the repository.

Michael Okoko is a student currently pursuing a degree in Computer Science and Mathematics at Obafemi Awolowo University, Nigeria. He loves open source and is mostly interested in Linux, Golang, PHP, and fantasy novels! You can reach him via:

Rate this post

1 2 3 4 5

Authors

Michael Okoko