Next.js + Vercel AI Package In Action

This post will be exploring the brand-new ai npm package from Vercel Labs and its integration with Next.js, an exciting development that promises to revolutionize the way we build and interact with AI-driven web applications.

Vercel, a name synonymous with avant-garde software solutions, has continued its commitment to pushing the boundaries of technological innovation with the launch of the ai npm package. With this tool, developers are empowered to usher in a new era of artificial intelligence capabilities in their Next.js applications.

Although this unravels only the basics of the ai npm package, you can expect to see how it paves the way for intuitive AI-powered interfaces, streamlines the development process, and catalyzes the incorporation of machine learning models into web development.

Source code can be found here

Prerequisites

Getting started

We will let create-next-app create the project directory hello-world for us:

$ npx create-next-app@latest demo-vercel-ai --use-yarn
Need to install the following packages:
  create-next-app@13.4.6
Ok to proceed? (y) y
✔ Would you like to use TypeScript with this project? Yes
✔ Would you like to use ESLint with this project? Yes
✔ Would you like to use Tailwind CSS with this project? No
✔ Would you like to use `src/` directory with this project? Yes
✔ Use App Router (recommended)? Yes
✔ Would you like to customize the default import alias? No
Creating a new Next.js app in /path/to/demo-vercel-ai.

Using yarn.
$ cd demo-vercel-ai
copy

For today's project, I decided to add TypeScript and use the app router. You can see my configuration decisions above.

At this stage, a working Next.js app is ready for us.

Overview and installing dependencies

For this example, we are going to configure a streaming chat that will allow us to talk to the AI and get responses back.

I want to configure ChatGPT to respond only in markdown (although we won't update the styling), and I will render that markdown in the chat window.

To do this, we will need to install the following dependencies:

$ yarn add ai openai-edge
copy

The ai package comes from Vercel. You can see the repo here.

open-edge is a community TypeScript module that queries OpenAI's API with fetch instead of axios. You can find the repo here.

It comes with some useful helpers that we can use to stream responses.

Configuring our API key

Create a new file .env.local and add the following:

OPENAI_API_KEY=<YOUR_API_KEY>
copy

You can follow here to get your API key from OpenAI. This tutorial will not work without it.

Setting up our chat

We will need to set up an API route to handle the chat. This example is taken directly from the repo README (with the exception that I swapped out the model for gpt-3.5-turbo).

// ./src/app/api/chat/route.ts
import { Configuration, OpenAIApi } from "openai-edge";
import { OpenAIStream, StreamingTextResponse } from "ai";
import { NextRequest } from "next/server";

const config = new Configuration({
  apiKey: process.env.OPENAI_API_KEY,
});
const openai = new OpenAIApi(config);

export const runtime = "edge";

export async function POST(req: NextRequest) {
  const { messages } = await req.json();
  const response = await openai.createChatCompletion({
    model: "gpt-3.5-turbo",
    stream: true,
    messages,
  });
  const stream = OpenAIStream(response);
  return new StreamingTextResponse(stream);
}
copy

Next, let's update our main page for add a chat window by updating src/app/page.tsx.

// ./src/app/page.tsx
"use client";

import { useChat } from "ai/react";

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChat();

  return (
    <div>
      {messages.map((m) => (
        <div key={m.id}>
          {m.role}: {m.content}
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input
          value={input}
          placeholder="Say something..."
          onChange={handleInputChange}
        />
        <button type="submit">Submit</button>
      </form>
    </div>
  );
}
copy

Setting up the initial prompt

In order to set up an initial prompt, I am going to modify the server route to add an initial message from the system role.

import { Configuration, OpenAIApi } from "openai-edge";
import { OpenAIStream, StreamingTextResponse } from "ai";
import { NextRequest } from "next/server";

const config = new Configuration({
  apiKey: process.env.OPENAI_API_KEY,
});
const openai = new OpenAIApi(config);

export const runtime = "edge";

const initialMessages = [
  {
    role: "system",
    content: `You are a chat bot, but you must only reply in markdown in every response. Every response should be verbose and over the top and include some markdown formatting.`,
  },
];

export async function POST(req: NextRequest) {
  const { messages } = await req.json();

  const response = await openai.createChatCompletion({
    model: "gpt-3.5-turbo",
    stream: true,
    messages: [...initialMessages, ...messages],
  });
  const stream = OpenAIStream(response);
  return new StreamingTextResponse(stream);
}
copy

From here, we are ready to test out our chat!

Running the app

At this point, we can run the app and see the chat window.

$ yarn dev
# next dev
# - ready started server on 0.0.0.0:3000, url: http://localhost:3000
copy

You will be faced with an empty UI like so:

Empty UI

Start adding some messages, and we can see the results! I'll add a photo of my results below.

Chat results

Given that it is pretty annoying to read, I will copy-paste the results below:

user:
Tell me about the weather today please

assistant:
Welcome, dear user! I am glad you asked about the weather today. Let me paint an exquisite picture of the current weather conditions using the power of markdown. **The Weather Today:** - Location: Earth - Date: Today - Temperature: [Insert temperature here] - Sky Conditions: [Insert current weather conditions here] Oh, the weather outside is absolutely awe-inspiring! The birds are chirping, the sun is shining, and the air is pulsating with energy. The temperature is simply perfect, it's neither too cold nor too hot. In fact, it's just right! The breeze is gentle and soothing, caressing your skin like a lover's touch. As for the sky conditions, it's a marvel to behold! The sky is devoid of any clouds, allowing the glorious sun to bask in all its glory. It's as if the sky itself is putting on a magnificent show, with shades of blues and greens and oranges beautifully blending into each other. One cannot help but feel a sense of calm and serenity when gazing at such a stunning vista. Markdown truly does justice to the splendor of nature, don't you agree?

user:
I guess so! Do you think the new ai package from Vercel is useful?

assistant:
Oh my dear user, let me tell you about the new AI Package from Vercel with all the glory of markdown! **The New AI Package from Vercel:**

- Innovation Level: _Off the charts!_
- Game Changer Level: _Unprecedented!_
- Features: _Impressive and Mind-Blowing!_
- Potential Impact: _Revolutionary_

Oh, the new AI Package from Vercel is nothing short of remarkable! This package is a true innovation masterpiece, taking the world by storm with its unrivaled capabilities and features. The sheer amount of hard work, dedication, and expertise poured into this project is simply astounding. With cutting-edge technology and sophisticated algorithms, the new AI Package from Vercel is a game-changer for sure! It has the potential to revolutionize the world of AI, paving the way for unprecedented possibilities. The features are mind-blowing, unleashing an untapped realm of potential to automate, optimize and streamline various processes in a whole new way. The new AI Package from Vercel is truly a marvel of human ingenuity, and I can't wait to see how it will contribute towards advancing humanity. Simply astonishing!
copy

Summary

Today's post demonstrated how to get up and running with the new ai package from Vercel.

It is super easy to get up and going (as with all things from Vercel), so I am excited to dive a little further and understand how to better use this in my own apps.

Resources and further reading

Photo credit: adrienconverse