ChatGPT Clone Tutorial

In this tutorial, you'll build a custom ChatGPT using Next.js and streaming using Humanloop TypeScript SDK.

At the end of this tutorial, you’ll have built a simple ChatGPT-style interface using Humanloop as the backend to manage interactions with your model provider, track user engagement and experiment with model configuration.

If you just want to leap in, the complete repo for this project is available on GitHub here.

A simple ChatGPT-style interface using the Humanloop SDK to manage interaction with your model provider.

Step 1: Create a new Prompt in Humanloop

First, create a Prompt with the name customer-support-agent. Go to the Editor tab on the left. Here, we can play with parameters and prompt templates to create a model which will be accessible via the Humanloop SDK.

Model Provider API keys

If this is your first time using the Prompt Editor, you’ll be prompted to enter an OpenAI API key. You can create one by going here.

The Prompt Editor is an interactive environment where you can experiment with prompt templates to create a model which will be accessible via the Humanloop SDK.

Let’s try to create a customer support agent. Paste the following system message into the Chat template box on the left-hand side.

You are a customer support agent for {{ companyName }}
Your goal is to assist customers with their inquiries, resolve their issues, and provide helpful information in a polite and professional manner.
Use the details provided by the customer to understand and address their needs. If necessary, ask clarifying questions to gather more information.
Always maintain a friendly and empathetic tone.

In the Parameters section above, select gpt-4 as the model. Click Commit and enter a commit message such as “Adding customer support template”.

Navigate back to the Dashboard tab in the sidebar. Your new Prompt Version is visible in the table at the bottom of the Prompt dashboard.

Notice your new Prompt Version not deployed into your Production environment. To deploy it, expand the menu of the Version in the table, and select Deploy. Check the Production box in the dialog that opens, and press Deploy. By deploying the Prompt Version it will be used in your prompt.call commands, as the version deployed in the Production environment is used by default.

Step 2: Set up a Next.js application

Now, let’s turn to building out a simple Next.js application. We’ll use the Humanloop TypeScript SDK to provide programmatic access to the model we just created.

Run npx create-next-app@latest to create a fresh Next.js project. Accept all the default config options in the setup wizard (which includes using TypeScript, Tailwind, and the Next.js app router). Now npm run dev to fire up the development server.

Next npm i humanloop to install the Humanloop SDK in your project.

Edit app/page.tsx to the following. This code stubs out the basic React components and state management we need for a chat interface.

page.tsx
1"use client";
2
3import { ChatMessage } from "humanloop/api";
4import * as React from "react";
5
6const { useState } = React;
7
8export default function Home() {
9 const [messages, setMessages] = useState<ChatMessage[]>([]);
10 const [userInputValue, setUserInputValue] = useState("");
11
12 const onSend = async () => {
13 const userMessage: ChatMessage = {
14 role: "user",
15 content: userInputValue,
16 };
17
18 setUserInputValue("");
19
20 const newMessages = [...messages, userMessage];
21
22 setMessages(newMessages);
23
24 // REPLACE ME LATER
25 const res = "I'm not a language model. I'm just a string. 😞";
26 // END REPLACE ME
27
28 const assistantMessage: ChatMessage = {
29 role: "assistant",
30 content: res,
31 };
32
33 setMessages([...newMessages, assistantMessage]);
34 };
35
36 const handleKeyDown = (e: React.KeyboardEvent<HTMLInputElement>) => {
37 if (e.key === "Enter") {
38 onSend();
39 }
40 };
41
42 return (
43 <main className="flex flex-col items-center min-h-screen p-8 md:p-24">
44 <h1 className="text-2xl font-bold leading-7 text-gray-900 dark:text-gray-200 sm:truncate sm:text-3xl sm:tracking-tight">
45 Customer Support Chat
46 </h1>
47
48 <div className="flex-col w-full mt-8">
49 {messages.map((msg, idx) => (
50 <MessageRow key={idx} msg={msg}></MessageRow>
51 ))}
52
53 <div className="flex w-full">
54 <div className="min-w-[70px] uppercase text-xs text-gray-500 dark:text-gray-300 pt-2">
55 User
56 </div>
57 <input
58 className="w-full px-4 py-1 mr-3 leading-tight text-gray-700 break-words bg-transparent border-none appearance-none dark:text-gray-200 flex-grow-1 focus:outline-none"
59 type="text"
60 placeholder="Type your message here..."
61 aria-label="Prompt"
62 value={userInputValue}
63 onChange={(e) => setUserInputValue(e.target.value)}
64 onKeyDown={(e) => handleKeyDown(e)}
65 ></input>
66 <button
67 className="px-3 font-medium text-gray-500 uppercase border border-gray-300 rounded dark:border-gray-100 dark:text-gray-200 hover:border-blue-500 hover:text-blue-500"
68 onClick={() => onSend()}
69 >
70 Send
71 </button>
72 </div>
73 </div>
74 </main>
75 );
76}
77
78interface MessageRowProps {
79 msg: ChatMessage;
80}
81
82const MessageRow: React.FC<MessageRowProps> = ({ msg }) => {
83 return (
84 <div className="flex pb-4 mb-4 border-b border-gray-300">
85 <div className="min-w-[80px] uppercase text-xs text-gray-500 leading-tight pt-1">
86 {msg.role}
87 </div>
88 <div className="pl-4 whitespace-pre-line">{msg.content as string}</div>
89 </div>
90 );
91};

We shouldn’t call the Humanloop SDK from the client’s browser as this would require giving out the Humanloop API key, which you should not do! Instead, we’ll create a simple backend API route in Next.js which can perform the Humanloop requests on the Node server and proxy these back to the client.

Create a file containing the code below at app/api/chat/route.ts. This will automatically create an API route at /api/chat. In the call to the Humanloop SDK, you’ll need to pass the project name you created in step 1.

app/api/chat/route.ts
1import { HumanloopClient } from "humanloop";
2
3if (!process.env.HUMANLOOP_API_KEY) {
4 throw Error(
5 "no Humanloop API key provided; add one to your .env.local file with: `HUMANLOOP_API_KEY=..."
6 );
7}
8
9const humanloop = new HumanloopClient({
10 environment: "https://api.humanloop.com/v5",
11 apiKey: process.env.HUMANLOOP_API_KEY,
12});
13
14export async function POST(req: Request): Promise<Response> {
15 const messages = await req.json();
16
17 const response = await humanloop.prompts.call({
18 path: "customer-support-agent",
19 messages,
20 // This is the name of your company. You can change it to any string you like.
21 // It matches the companyName input defined in the Prompt Version template.
22 inputs: {
23 companyName: "Acme Co.",
24 },
25 providerApiKeys: {
26 openai: process.env.OPENAI_API_KEY,
27 },
28 });
29
30 return new Response(JSON.stringify(response.logs[0].output));
31}

In this code, we’re calling humanloop.prompts.call. This function is used to target the model which is actively deployed on your file - in this case it should be the model we set up in step 1. To see other related Prompt functions, use the the SDK reference.

When we receive a response from Humanloop, we strip out just the text of the chat response and send this back to the client via a Response object (see Next.js - Route Handler docs). The Humanloop SDK response contains much more data besides the raw text, which you can inspect by logging to the console.

For the above to work, you’ll need to ensure that you have a .env.local file at the root of your project directory with your Humanloop API key. You can generate a Humanloop API key by clicking your name in the bottom left and selecting API keys. This environment variable will only be available on the Next.js server, not on the client (see Next.js - Environment Variables).

.env.local
HUMANLOOP_API_KEY=...

Now, modify page.tsx to use a fetch request against the new API route.

page.tsx
1// REPLACE ME LATER <-- Remove these now
2const response = await fetch("/api/chat", {
3 method: "POST",
4 headers: {
5 "Content-Type": "application/json",
6 },
7 body: JSON.stringify(newMessages),
8});
9
10const res = await response.json();
11// END REPLACE ME <-- Remove these now

You should now find that your application works as expected. When we send messages from the client, a GPT response appears beneath (after a delay).

Back in your Humanloop Prompt dashboard you should see Logs being recorded as clients interact with your model.

Step 3: Streaming tokens

(Note: requires Node version 18+).

You may notice that model responses can take a while to appear on screen. Currently, our Next.js API route blocks while the entire response is generated, before finally sending the whole thing back to the client browser in one go. For longer generations, this can take some time, particularly with larger models like GPT-4. Other model config settings can impact this too.

To provide a better user experience, we can deal with this latency by streaming tokens back to the client as they are generated and have them display eagerly on the page. The Humanloop SDK wraps the model providers’ streaming functionality so that we can achieve this. Let’s incorporate streaming tokens into our app next.

Edit the API route at to look like the following. Notice that we have switched to using the humanloop.prompts.callStream function, which offers Server Sent Event streaming as new tokens arrive from the model provider.

app/api/chat/route.ts
1import { HumanloopClient } from "humanloop";
2import { readableStreamAsyncIterable } from "humanloop/core/streaming-fetcher/Stream";
3
4if (!process.env.HUMANLOOP_API_KEY) {
5 throw Error(
6 "no Humanloop API key provided; add one to your .env.local file with: `HUMANLOOP_API_KEY=..."
7 );
8}
9
10const humanloop = new HumanloopClient({
11 environment: "https://api.humanloop.com/v5",
12 apiKey: process.env.HUMANLOOP_API_KEY,
13});
14
15export async function POST(req: Request): Promise<Response> {
16 const messages = await req.json();
17
18 const response = await humanloop.prompts.callStream({
19 path: "customer-support-agent",
20 messages,
21 // This is the name of your company. You can change it to any string you like.
22 // It matches the companyName input defined in the Prompt Version template.
23 inputs: {
24 companyName: "Acme Co.",
25 },
26 providerApiKeys: {
27 openai: process.env.OPENAI_API_KEY,
28 },
29 });
30
31 const stream = readableStreamAsyncIterable(response);
32
33 // Create a ReadableStream from the async iterable
34 const readableStream = new ReadableStream({
35 async start(controller) {
36 try {
37 for await (const chunk of stream) {
38 // Serialize the chunk to a string
39 const serializedChunk = JSON.stringify(chunk);
40 // Enqueue the serialized chunk as a Uint8Array
41 controller.enqueue(new TextEncoder().encode(serializedChunk));
42 }
43 controller.close();
44 } catch (error) {
45 controller.error(error);
46 }
47 },
48 });
49
50 return new Response(readableStream, {
51 headers: { "Content-Type": "application/json" },
52 });
53}

Now, modify the onSend function in page.tsx to the following. This streams the response body in chunks, updating the UI each time a new chunk arrives.

app/page.tsx
1const onSend = async () => {
2 const userMessage: ChatMessage = {
3 role: "user",
4 content: userInputValue,
5 };
6
7 setUserInputValue("");
8
9 const newMessages = [
10 ...messages,
11 userMessage,
12 { role: ChatRole.Assistant, content: "" },
13 ];
14
15 setMessages(newMessages);
16
17 const response = await fetch("/api/chat", {
18 method: "POST",
19 headers: {
20 "Content-Type": "application/json",
21 },
22 body: JSON.stringify(newMessages),
23 });
24
25 const decoder = new TextDecoder();
26 const reader = response?.body?.getReader();
27 let done = false;
28 while (!done && reader) {
29 const chunk = await reader.read();
30 const value = chunk.value;
31 done = chunk.done;
32 const val = decoder.decode(value);
33 const jsonChunks = val
34 .split("}{")
35 .map(
36 (s) => (s.startsWith("{") ? "" : "{") + s + (s.endsWith("}") ? "" : "}")
37 );
38 const tokens = jsonChunks.map((s) => JSON.parse(s).output).join("");
39 setMessages((messages) => {
40 const updatedLastMessage = messages.slice(-1)[0];
41 return [
42 ...messages.slice(0, -1),
43 {
44 ...updatedLastMessage,
45 content: (updatedLastMessage.content as string) + tokens,
46 },
47 ];
48 });
49 }
50};

You should now find that tokens stream onto the screen as soon as they are available.

Conclusion

Congratulations! You’ve now built a working chat interface and used Humanloop to handle interaction with the model provider and log chats. You used a system message (which is invisible to your end user) to make GPT-4 behave like a customer support agent.

Now that you’ve seen how to create a simple Humanloop project and build a chat interface on top of it, try visiting the Humanloop project dashboard to view the logs and iterate on your Prompts.

All the code for this project is available on Github.