Streaming Responses

Stream updates from Restate Services to the UI via Server-Sent Events.

This guide shows how to stream real-time updates from Restate services to a chat UI.

Via Server-Sent Events (SSE)

Restate has a TypeScript library to implement streaming updates from Restate services to frontends via SSE. You can use this, for example, to stream agent updates to your chat UI.

restatedev/pubsub

Explore the library.

To use the library, have a look at the example and the steps below.

nextjs-example-app

NextJS Chat app backed by an AI SDK Agent that streams updates using SSE.

1. Install dependencies

   npm install @restatedev/pubsub @restatedev/pubsub-client
   
2. Register the pubsub object

   Add the pubsub Virtual Object to your Restate endpoint:

   restate/endpoint.ts

   import * as restate from "@restatedev/restate-sdk/fetch";
   import { createPubsubObject } from "@restatedev/pubsub";
   import { agent } from "./services/agent";
   
   const pubsub = createPubsubObject("pubsub", {});
   
   export const endpoint = restate.createEndpointHandler({
       services: [agent, pubsub],
   });
   
3. Publish messages from your service

   Use the pubsub client to publish messages:

   restate/services/agent.ts

   import * as restate from "@restatedev/restate-sdk";
   import { createPubsubClient } from "@restatedev/pubsub-client";
   
   const pubsub = createPubsubClient({
       url: "http://localhost:8080",
       name: "pubsub",
   });
   
   // In your handler:
   export const agent = restate.service({
       name: "MyAgent",
       handlers: {
           run: async (ctx: restate.Context, prompt: string) => {
               await pubsub.publish("session-123", { role: "user", content: prompt }, ctx.rand.uuidv4());
           },
       },
   });
   

   Note that the UUID, in the publish step, is an idempotency key that Restate will use to deduplicate events. This is necessary since this publish step goes via the Restate ingress. Without this idempotency key, each replay/retry leads to duplicate events. Alternatively, you can use service-to-service calls to publish events.

4. Create an SSE endpoint

   For example for NextJS, create an API route that streams pubsub messages:

   app/pubsub/[topic]/route.ts

   import { createPubsubClient } from "@restatedev/pubsub-client";
   import { NextRequest } from "next/server";
   
   const pubsub = createPubsubClient({
       url: process.env.INGRESS_URL || "http://localhost:8080",
       name: "pubsub",
   });
   
   export async function GET(request: NextRequest, { params }: any) {
       const topic = (await params).topic;
       const offset = Number(request.nextUrl.searchParams.get("offset") || 0);
   
       const stream = pubsub.sse({
           topic,
           offset,
           signal: request.signal,
       });
   
       return new Response(stream, {
           headers: { "Content-Type": "text/event-stream" },
       });
   }
   
5. Subscribe from the frontend

   Use the browser’s EventSource API to receive updates:

   app/agent/[topic]/page.tsx

   let offset = 0;
   const evtSource = new EventSource(`/pubsub/${topic}?offset=${offset}`);
   
   evtSource.onmessage = (event) => {
       const data = JSON.parse(event.data);
       // show the new messages in the UI...
   };
   

We are planning to add support for SSE with the Python SDK. Contact us on Discord or Slack for more info.

Streaming token-by-token

Token-by-token streaming is not yet supported but is on the roadmap. Contact us on Discord or Slack for more info.