Getting Started with OpenAI's GPT-3 in Node.js

Time to read: 7 minutes

November 03, 2020

Written by

Twilion

OpenAI's API for their new GPT-3 model provides a very versatile, general-purpose “text in, text out” interface, making it applicable to virtually any language task. This is different from most other language APIs, which are designed for a single task, such as sentiment classification or named entity recognition.

Let's walk through how to unlock the capabilities of this API with Node.js.

Taking care of dependencies

Before moving on, you will need an up to date version of Node.js and npm installed.

Navigate to the directory where you want this code to live and run the following command in your terminal to create a package for this project:

npm init --yes

The --yes argument runs through all of the prompts that you would otherwise have to fill out or skip. Now that we have a package.json for our app, run this command in the same directory to install the Got library for making HTTP requests:

npm install got@11.8.0

OpenAI has an official helper library for Python, and there have been some third party Node libraries, but we will be using the REST API on its own so all we need to be able to do is make HTTP requests.

This project requires an API key from OpenAI. At the time I’m writing this, the only way to obtain one is by being accepted into their private beta program. You can apply on their site. If you have one, create anenvironment variable for your OpenAI API key called OPENAI_SECRET_KEY, which you can find as the "secret key" at the top of theDeveloper Quickstart:

export OPENAI_SECRET_KEY='YOUR_API_KEY_HERE'

Introduction to GPT-3

GPT-3 (Generative Pre-trained Transformer 3) is a highly advanced language model trained on a giant amount of text. In spite of its internal complexity, it is surprisingly simple to operate: you feed it some text, and the model generates some more following a similar style and structure.

We can see how versatile it is even with a tiny amount of input. For example, let's try to generate some lyrics in the style of the Brazilian thrash metal band Sepultura with the input "Artist: Sepultura\n\nLyrics:":

GPT-3 is non-deterministic in the sense that given the same input, multiple runs of the engine will return different responses. It's great at picking up on structure and context, so you can mess around with the prompt text to see what gets the best results.

The OpenAI Playground

Once you have an OpenAI account, you can use the Playground to play around with GPT-3This is the best way to get started exploring the API. It simply includes a text box where you write the prompt, and sliders on the side to change the parameters used for generation. The OpenAI docs have some great pointers on designing your prompt text.

The Playground also has a cool feature that allows you to grab some Python code you can run, using OpenAI's Python library, for whatever you used the Playground for.

We won't be using Python in this tutorial, but as seen in the gif, it's possible to use the playground to structure an HTTP request to accomplish the same thing in any other language.

Using Node.js to generate text with GPT-3

Let's continue by writing some JavaScript code for what we were doing in the Playground. I'm in a thrash metal mood today, so let's generate some Megadeth lyrics this time. Create a file called index.js in the same directory as your package.json file and add the following code to it:

const got = require('got');
const prompt = `Artist: Megadeth\n\nLyrics:\n`;

(async () => {
  const url = 'https://api.openai.com/v1/engines/davinci/completions';
  const params = {
    "prompt": prompt,
    "max_tokens": 160,
    "temperature": 0.7,
    "frequency_penalty": 0.5
  };
  const headers = {
    'Authorization': `Bearer ${process.env.OPENAI_SECRET_KEY}`,
  };

  try {
    const response = await got.post(url, { json: params, headers: headers }).json();
    output = `${prompt}${response.choices[0].text}`;
    console.log(output);
  } catch (err) {
    console.log(err);
  }
})();

Run this code with the command node index.js and you should see some lyrics logged to the console. The results I got involve Dave Mustaine being angry at people on his commute. Maybe it's a song he would have written on his long bus ride back to California after being kicked out of Metallica.

You can tweak how you want GPT-3 to generate text by changing what you send in the body of your request. Here is a brief description of various parameters you can pass to the API:

prompt: The input text.
engine: OpenAI has made four text completion engines available, named davinci, ada, babbage and curie. We are using davinci, which is the most capable of the four, yet the slowest and most expensive if your needs extend beyond the available free tier for the API.
stop: The GPT-3 engine does not really "understand" text, so when it generates text, it needs to know when to stop. In the example of building a chat bot, by giving a stop of "Human:" we are telling the engine to just generate text for the line that begins with "Bot:". Without a stop marker, GPT-3 would continue generating text by writing more lines for both the user and the AI.
temperature: A number between 0 and 1 that determines how many creative risks the engine takes when generating text.
top_p: An alternative way to control the originality and creativity of the generated text.
frequency_penalty: A number between 0 and 1. The higher this value, the bigger the effort the model will make in not repeating itself.
presence_penalty: A number between 0 and 1. The higher this value the model the bigger the effort the model makes in talking about new topics.
max_tokens: Maximum completion length.

You can use different values for different scenarios and different GPT-3 use cases. Let's go through some other things you can do.

Katon W De Pena, lead singer of thrash metal band, Hirax

Generating Dialogue with GPT-3

Chat bots are a very common use case for text generation. If you want to read more in depth on how to build a chat bot with GPT-3, read this other tutorial post by Miguel Grinberg. The code is in Python, but the advice and general concepts remain the same. We can “prime” the engine with one or two example interactions between the user and the AI to set the tone of the bot. We then append the actual prompt from the user at the end, and let the engine generate the response.

Let's see how we'd use GPT-3 to generate the dialogue of a chat bot with Node.js by asking the AI what it's favorite German thrash metal album is. Change the code in your index.js file to the following:

const got = require('got');

// This is just an example, but could be something you keep track of
// in your application to provide OpenAI as prompt text.
const chatLog = 'Human: Hello, who are you?\nAI: I am doing great. How can I help you today?\n';

// The new question asked by the user.
const question = 'Could you tell me what your favorite German thrash metal album is?';

(async () => {
  const url = 'https://api.openai.com/v1/engines/davinci/completions';
  const prompt = `${chatLog}Human: ${question}`;
  const params = {
    'prompt': prompt,
    'max_tokens': 150,
    'temperature': 0.9,
    'frequency_penalty': 0,
    'presence_penalty': 0.6,
    'stop': '\nHuman'
  };
  const headers = {
    'Authorization': `Bearer ${process.env.OPENAI_SECRET_KEY}`,
  };

  try {
    const response = await got.post(url, { json: params, headers: headers }).json();
    output = `${prompt}${response.choices[0].text}`;
    console.log(output);
  } catch (err) {
    console.log(err);
  }
})();

Notice how we tweaked the parameters to get different results for this use case. Again, run this code with the command node index.js. We're also using a "stop phrase" to let OpenAI know that we want the generation to stop at the token "\nHuman:" meaning we only want the bot's next response. Here's a conversation I was able to generate in this example:

Human: Hello, who are you?
AI: I am doing great. How can I help you today?
Human: Could you tell me what your favorite German thrash metal album is?
AI: Let me just listen to some examples of German thrash metal.
<thrash metal plays>
AI: The best German thrash metal album is Kreator's Pleasure to Kill . It was released in 1987. It's one of my favorite thrash albums. It's got really great riffs and solos. The production is amazing, too.

That is indeed an album that many would say is Kreator's most iconic. Although it has notoriously bad production, so I can't really agree with our bot on that point. Maybe they're just into lo-fi mixes. Personally, my favorite album by the band is Coma of Souls.

Generating dialogue doesn't have to take the form of a chat bot. The GPT-3 engine can also be used to tell stories, like generating fan fiction dialogue from the Dragonball series for example.

const got = require('got');

(async () => {
  const url = 'https://api.openai.com/v1/engines/davinci/completions';
  const characters = ['Alex Skolnick', 'Gary Holt', 'Katon W De Pena', 'James Hetfield', 'Scott Ian',
                      'Bobby Blitz', 'Mille Petrozza', 'Dave Mustaine', 'Max Cavalera', 'Marty Friedman'];

  // Select a random member of an 80's thrash metal band to start the dialogue.
  const prompt = `${characters[Math.floor(Math.random()*characters.length)]}:`;

  const params = {
    'prompt': prompt,
    'max_tokens': 256,
    'temperature': 0.7,
    'frequency_penalty': 0.4,
    'presence_penalty': 0.6
  };
  const headers = {
    'Authorization': `Bearer ${process.env.OPENAI_SECRET_KEY}`,
  };

  try {
    const response = await got.post(url, { json: params, headers: headers }).json();
    output = `${prompt}${response.choices[0].text}`;
    console.log(output.substring(0, output.lastIndexOf('\n')));
  } catch (err) {
    console.log(err);
  }
})();

Like before, run this with node index.js to see what happens. Sometimes it works really well with such little input as a person's name followed by a colon. But sometimes you might want to give the prompt text a little more guidance. So you can also get good results with a prompt like this:

const prompt = `The following is a conversation between ${characters[Math.floor(Math.random()*characters.length)]} and ${characters[Math.floor(Math.random()*characters.length)]}:`;

Here's some dialogue I got with this code, between Alex Skolnick of Testament and Gary Holt of Exodus:

The following is a conversation between Alex Skolnick and Gary Holt:

Alex Skolnick: "What I really liked about the guitars that we used on the new album was that they were all tuned very high: B to B on the 6-string and C to C on the 7-string. It's funny, because there's a lot of guitar players that are afraid to tune their guitars that high because they think that they'll lose the tonal quality or the string will break on them, but I've never had that happen."

Gary Holt: "Have you ever had a string break on you from tuning them high?"

Alex: "No. I don't know why, but I've never broken a string. I think it's a combination of how I tune them and also the string gauges I use. I really like heavy gauges."

So as you can see, GPT-3 can be used in a variety of different ways. Figuring out what parameters and prompt text gets you the best results is a matter of trial and error. It's always a good idea to get started by having fun with the Playground before moving to writing code.

Using OpenAI's Semantic Search

OpenAI also has an API you can use to do semantic search over documents. You can provide a natural language query, perhaps in the form of a question or a statement, and find documents that answer the question or are semantically related to the statement. This can be used for a variety of tasks, such as performing sentiment analysis on tweets or determining which descriptions of a movie's plot line up with a given movie title.

As with text completion, another unofficial Playground-like tool is provided that you can use to experiment with to figure out how you want to use the API.

Keeping in line with the thrash themed examples, let's see if we can try to get the GPT-3 engine to take song lyrics and see which band from the "Big Four" of thrash metal they are most similar to. Despite their similarities, all four of these bands have different lyrical styles.

So let's try to see which band the song Black Metal Sucks by Lich King is most similar to. This is a joke song about the singer's distaste for Black Metal, a subgenre popularized by bands from Norway that features extremely low production value, harsh vocals, and "evil" imagery almost to the point of parody with performers wearing corpse paint.

Replace the code in index.js with the following:

const got = require('got');

(async () => {
  const url = 'https://api.openai.com/v1/engines/davinci/search';
  const documents = ['These lyrics are similar to Metallica', 'These lyrics are similar to Megadeth',
                     'These lyrics are similar to Slayer', 'These lyrics are similar to Anthrax'];

  const query = "I wish your black clothes would combust. Why?\nBlack metal sucks\nYou know what dude, you look like a schmuck\nYou suck\nSinger sounds like a wounded duck";

  const params = {
    documents: documents,
    query: query
  };
  const headers = {
    'Authorization': `Bearer ${process.env.OPENAI_SECRET_KEY}`,
  };

  try {
    const response = await got.post(url, { json: params, headers: headers }).json();
    const data = response.data.sort((r1, r2) => r2.score - r1.score);
    console.log(query);
    console.log(data);

    // The result
    console.log(documents[data[0].document]);
  } catch (err) {
    console.log(err);
  }
})();

Here is my terminal output after running that code:

This result lines up with my expectations given that out of the four bands, Anthrax is the one that tends to have the most goofy lyrics and would most likely use words like "schmuck". As always, it takes some trial and error to figure out how to get the results you want. For example, we can try changing the "documents" to just the band names instead of a statement and see if that works.

Change the documents and query variables in index.js to the following:

  const documents = ['Metallica', 'Megadeth', 'Anthrax', 'Slayer'];

  const query = "Corporate decisions made each day\nAffecting everyone\nCommercial growth is all that counts\nNot harming anyone\n\nInvading trust, ideals unjust\nHide your reality";

Here we're using the song For Whose Advantage by the British band Xentrix. This is what prints to my terminal when I run this new code:

Given that these lyrics are a critique of the dominant sociopolitical order of the time, I would expect them to be classified as similar to Megadeth, a band that also tends to have politically charged lyrics.

The search endpoint can query up to 200 documents in one request. If you have more documents than that, you can divide them over multiple requests. On top of this, the query and longest document must be below 2000 tokens together.

When it comes to search, speed is often a priority. With GPT-3 there is a tradeoff between model accuracy and speed depending on which engine you're using. You might find that using a faster engine like ada instead of davinci works just as well for your use case even with lower accuracy.

How about an encore?

Now that we've brushed the surface of what OpenAI's APIs for GPT-3 are capable of, hopefully you are ready to experiment and tap into your own creativity for whatever your AI needs may be. If you're interested in digging deeper, we have a variety of GPT-3 related content here on the Twilio blog that you might like.

Feel free to reach out if you have any questions or comments or just want to show off any cool things you generate with GPT-3.

Email: sagnew@twilio.com
Twitter: @Sagnewshreds
Github: Sagnew
Twitch (streaming live code): Sagnewshreds

Related Resources

Twilio Docs

From APIs to SDKs to sample apps

API reference documentation, SDKs, helper libraries, quickstarts, and tutorials for your language and platform.

Resource Center

The latest ebooks, industry reports, and webinars

Learn from customer engagement experts to improve your own communication.

Ahoy

Twilio's developer community hub

Best practices, code samples, and inspiration to build communications and digital engagement experiences.