Mutation and Conflict Resolution

Introduction

Sync supports the standard HTTP If-Match and ETag headers for the purpose of conditional mutation. Every response containing a Sync object will return an ETag value, which currently correspond 1:1 with revision in the object body. If you specify this value in some subsequent updates operation, that operation will only succeed if there has been no interleaving update.

In this guide we'll show you how to leverage ETags in REST API requests and also how these are exposed in our JavaScript SDK

REST API conflict walkthrough

Let's walk through an example of updating a document through the REST API to understand how optimistic concurrency works.

curl -X POST https://sync.twilio.com/v1/Sync/Services/ISxx/Documents \
 -d 'UniqueName=MyFirstDocument' \
 -d 'Data={firstName:Alice,lastName:Xavier}'  \
 -u 'ACXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX:your_auth_token'

This will create a document with the following structure:

{
  "account_sid": "ACxx",
  "service_sid": "ISxx",
  "sid": "ETxx",
  "unique_name": "MyFirstDocument",
  "revision": "0", 
  "date_created": "2015-11-24T22:18:57Z",
  "date_updated": "2015-11-24T22:18:57Z",
  "created_by": "system",
  "url": "https://sync.twilio.com/v1/Services/ISxx/Documents/ETxx",
  "data": {
      "firstName":"Alice",
      "lastName":"Xavier"
  }
}

Now let's update that object through the REST API.

curl -X POST https://sync.twilio.com/v1/Sync/Services/ISxx/Documents/MyFirstDocument \
 -d 'Data={firstName:Bob,lastName:Xavier}'  \
 -u 'ACXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX:your_auth_token'

We'll get a response back, and now the revision will be 1.

Similar to our counter example above, let's say there is an argument over whether the first name is Alice, Bob or someone else. Now we have two POST requests hitting Sync in quick succession:

POST A

curl -X POST https://sync.twilio.com/v1/Sync/Services/ISxx/Documents/MyFirstDocument \
 -d 'Data={firstName:Charlie,lastName:Xavier}'  \
 -u 'ACXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX:your_auth_token'

POST B

curl -X POST https://sync.twilio.com/v1/Sync/Services/ISxx/Documents/MyFirstDocument \
 -d 'Data={firstName:Dave,lastName:Xavier}'  \
 -u 'ACXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX:your_auth_token'

In the above case, it will be the last write wins. Post B will simply overwrite post A. Just like in the mutate example above, what if we want to make sure we are up to date before we make any changes?

The answer is to use the If-Match header mentioned above. First, we make sure we GET the document revision. Then we pass this revision as a value in the If-Match header when we post our update. This ensures that if we are not the most up to date revision, our update will be rejected.

POST B

curl -X POST https://sync.twilio.com/v1/Sync/Services/ISxx/Documents/MyFirstDocument \
 --header 'If-Match':'1a' \
 -d 'Data={firstName:Dave,lastName:Xavier}'  \
 -u 'ACXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX:your_auth_token'

Note the addition of the header If-Match. Now, this operation will fail unless the revision of our document is 1a. The error generated will be:

The revision of the Document does not match the expected revision

JavaScript SDK conflict walkthrough

Let's take the example of a counter, and we want each separate browser tab to increment the value. The JSON representation is:

{ count:0 }

We will store this in a document called "counter":

syncClient.document("counter").then(function(doc) {
    doc.set({count:1});
});

The above works fine for the first count in the first browser tab, but all that will do is repeatedly set the count to 1 when executed. So let's extend the functionality:

syncClient.document("counter").then(function(doc) {
  if(!doc.value.count) {
    doc.set({count:1});
  }
  else {
    var newCount = doc.value.count + 1;
    doc.set({count:newCount});
  }
});

Now, each time the browser is loaded, the document is opened and the count is increased by 1. If we add a subscription to the document, then browsers which are already open will receive an update to the count themselves:

syncClient.document("counter").then(function(doc) {
  if(!doc.value.count) {
    doc.set({count:1});
  }
  else {
    var newCount = doc.value.count + 1;
    doc.set({count:newCount});
  }

  doc.on("updated", function(item) {
    console.log("count", item.count);
  });    
});

So now we have a system where we're counting each browser tab being opened. But now it gets interesting. What if two browser tabs open at the same time, so both of them increment the count at the same time. So lets say there's already 10 open.

Browser tab A opens at the same time as B. So this happens:

A: 10 + 1 = 11 //I'm the 11th Tab!
B: 10 + 1 = 11 //No... I'm the 11th Tab!

Using doc.set will mean that both browser tabs will attempt to write 11, and both will succeed. So now our count is wrong. So how do we fix this? Let's mutate, people.

syncClient.document("counter").then(function (doc) {
  doc.mutate(function (remoteValue) {
    console.log("mutate", remoteValue.count);
    if (!remoteValue.count) {
      remoteValue.count = 1;
    } else {
      remoteValue.count += 1;
    }
    return remoteValue;
  }).then(function () {
    console.log("mutate_done", doc.value.count);
  });

  doc.on("updated", function (item) {
    console.log("count", item.count);
  });

});

What's occurring? Using mutate allows us to specify that we want to modify the value only when the local copy of the value is synchronised. So if we use mutate, let's look at our problem again:

A: 10 + 1 = 11 //I'm the 11th Tab!
B: 10 + 1 = 11 //No... I'm the 11th Tab!

With mutate, B will try to execute, but the server will actually return an error, putting B into conflict. By using the mutate function instead of set, we instruct our client to keep trying the function passed to mutate until it can operate on an in-sync value. This means that even if browser tab C came on the picture at the same time, all three tabs would eventually increment the value correctly.

Important to remember

• Revisions are strings, not integers. They are not designed to be programmatically comparable as numbers.
• When posting updates via the REST API, a concurrency control check only occurs if the If-Match header is present. Posts without an If-Match header will overwrite any existing data.

In the client SDKs (JavaScript, iOS and Android) the above revision management is handled for you and managed through the SDK methods. In the REST API, using If-Match headers enables you to ensure you are operating on the most up to date versions of your data.

Racing Guarded & Unguarded Requests

Unconditional updates (no If-Match header) will sometimes succeed even when a conditional update (with If-Match) ultimately squashes it. If you need to detect the winner of such a race, you will need to examine the final data. We recommend not racing conditional updates with unconditional ones.