How to implement the AWS recommended exponential backoff and jitter in JavaScript (and Observable)

Also known as Solving “ThrottlingException: Rate exceeded” in Javascript with Exponential backoff + Jitter

The ThrottlingException: Rate exceeded error is a frustrating one, mainly because it has a series of bad habits: it happens mostly in production, randomly (or so it seems), and often in the middle of a critical loop in the code. The error is an AWS protection mechanism to avoid being flooded with requests from buggy code, like an infinite loop, yet the responsibility is left to us, developers in the coding trenches, to get around the error.

I was one of the persons frustrated by this exception. Ok, I kind of searched for it: I was using a new language (Javascript) on a new platform (Observable) to connect the AWS pricing API (mature, so complicated).

I learned Javascript while coding on Observable. I had to learn what Promises are, and yes, I still add await on most of my calls; I forget async before functions too often, and I still need some time to read code like () => {}.

The story

I love data, I love graphs showing insights from data, and I love finance and the cloud. My goal is to create beautiful and insightful graphs from the AWS pricing data, with text to give it context, all of it without having to choose colours (whatever I take, I take wrong). For that, I found Observable, a platform that mixes JavaScript in a python-like notebook with beautiful graphics capabilities. The creators of Observable are the people behind the beautiful D3js(https://d3js.org/) library.

So with my beginner knowledge of Javascript, I went on to read some documentation (little) and deep dive into the doing and making all the beginner’s mistakes. One of the first things to do was to learn how to connect to AWS from Observable (which will be a topic for a future article) and then ask for data, then use that data to ask for more data… and so on. All was working great until my nested for loops generated too many calls, and the ThrottlingException: Rate exceeded happened. BOOM!

Going through the AWS documentation helped and guided me to exponential backoff, and then this excellent article (https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/) that goes deep into the algorithm and why it works. Unfortunately, it is all theory. Searching online for ’exponential backoff Javascript’ provided some guidance, but no ready-made solution seemed to work, and most did not include the jitter aspect of the AWS recommended implementation. Most code available online works for simple use cases but breaks when used as part of a class making recursive calls, using functions with parameters, or using _this. Obviously, I need all of that.

So after many copy/paste, swear words, and long evenings looking at the screen, I did manage to get something that is ugly, but WORKS!

Three solutions

What does any good IT expert do when facing a problem? Look at a solution that can be copy-pasted on a search engine; I use Duckduckgo.

Option #1 - for functions without arguments

There are a number of very well written (better than this one at least) articles online, for example, https://advancedweb.hu/how-to-implement-an-exponential-backoff-retry-strategy-in-javascript/ by Tamás Sallai, who proposes:

const callWithRetry = async (fn, depth = 0) => {
    try {
        return await fn();
    }catch(e) {
        if (depth > 7) {
            throw e;
        }
        await wait(2 ** depth * 10);
    
        return callWithRetry(fn, depth + 1);
    }
}

And the function can be called as follow:

const result = await callWithRetry(maybeFailingOperation);

The drawback is that this function retries for any error and assumes the function does not need any argument. The post continues with another method that is interesting yet does not work for my use case.

Option #2 - Using the above but adding the parameters

That turned out to be easy and simply required adding a parameter to the callWithRetry function. I used an Object called params. That object has one field per parameter required by the function to run. Inside the function, I extract the required parameter from the param object.

const callWithRetry = async (fn, params, depth = 0) => {
    try {
        // if needed extract
        return await fn(params);
    }catch(e) {
        if (depth > 7) {
            throw e;
        }
        await wait(2 ** depth * 10);
    
        return callWithRetry(fn, depth + 1);
    }
}

And the function can be called as follow:

const result = await callWithRetry(maybeFailingOperation, params);

This function worked ok in tests but did not in an object. In my code, a recursive call using this made it fail. It seems that this is lost somewhere in the ether. (I am sure that if I knew more about JS, I would be able to address the issue, but for now…I cannot).

Option #3 - Back to be KISS (keep it simple, stupid)

So I went back to what I knew (not much) and used simple JS constructs. Nothing like ()=>… just simple for loops. This is a snippet of the code used in my publicly visible AWS library for Observable.

async describeServices(params){
      let response
      let data
      for (let attempt=0; attempt<5; attempt++){
        console.debug("[getAttributeValues].attempt: "+attempt)
        if(attempt>=5){ throw "Too many attempts to connect to AWS API" }
        try{
          response = await self._client.describeServices(params);
          data = response.Services;
          break
        } catch (error) {
          const toWait = randomBetween(0, Math.min(cap, 100 *  Math.pow(2, attempt)));
              console.log("To wait: "+toWait)
          await sleep(toWait)
          continue  
        }
      }

it uses two support functions: one to generate a random number between two values, named, without much immagination, randomBetween
function randomBetween(min, max) {
  return Math.floor(Math.random() * (max - min + 1) ) + min;
}

And one to wait for a certain amount of ms.

sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms))

Ugly, but it works, and once implemented, I can simply forget about how ugly it is. :)

What next

I never want to stop learning, so please share your findings with me: leave a comment, send me an email, or contact me on LinkedIn (I am not on Facebook).

I learn in public; I share as fast and as much as possible on https://zt.frankcontrepois.com. It can be messy, but it is the best way I found to remember all the connections between topics that my brain built (because often, after making such connections, my brain forgets!).

have a great day Frank Contrepois