面向 AI 助手:本页的 Markdown 版本位于 https://ark.volcengine.com/region:cn-beijing/docs/streaming-output.md,全部文档索引见 https://ark.volcengine.com/llms.txt。

流式输出

在实时对话、长内容生成或思考过程展示中,等待完整响应后再显示结果会让用户长时间看不到进展。流式输出让客户端边接收边展示生成内容;如果应用需要更快反馈、增量处理或保留并标记不完整的中断前结果,本文将帮助你在现有模型调用 API 中启用流式输出、正确判断终态并管理连接。

快速开始#

效果示意 优势说明
非流式输出等待完整响应后一次返回,流式输出持续显示已生成内容并在终态完成的效果对比 更早展示内容:收到首个片段后即可开始渲染,不必等待完整响应。
提供实时反馈:对话界面、长内容生成和思考过程可以持续更新。
支持增量处理:客户端可以分别聚合正文、思考内容和工具参数。
保留部分结果:终态前断连时可保留已接收内容,但必须标记为不完整。

动画中的时间仅用于展示返回方式,不代表实际首段时延或总生成时间。

按现有 API 接入#

启用流式输出不需要迁移 API。根据应用当前使用的 API,增加流式请求字段,并适配对应的响应事件:

当前 API 请求改动 响应处理
Chat API 在请求体中设置 stream=true。 按 choices[].index 聚合 delta,并根据 finish_reason 判断结果。实现方法参见 处理流式响应。
Responses API 在请求体中设置 stream=true;Go SDK 使用 CreateResponsesStream。 按事件 type 分派并使用复合索引聚合。实现方法参见 处理流式响应。
Messages API 在请求体中设置 stream=true。 按消息与内容块生命周期聚合,并结合 stop_reason 与 message_stop 判断结果。实现方法参见 处理流式响应。

选择与现有应用相同的 API 示例。三类 API 的事件结构不同,不得共用事件解析分支。首次体验可直接运行 Chat API 的 cURL 或 Python 示例。

运行 Python 方舟 SDK 示例前,执行 python -m pip install --upgrade arkruntime;运行 Go 方舟 SDK 示例前,执行 go get github.com/volcengine/ark-runtime-go@latest。Messages Python REST 示例需使用 Python 3.9 或更高版本并安装 requests。

Chat API#

Bash
curl https://ark.cn-beijing.volces.com/api/v3/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $ARK_API_KEY" \
  -d '{
    "model": "doubao-seed-2-1-pro-260915",
    "messages": [
        {
            "role": "user",
            "content": "常见的十字花科植物有哪些?"
        }
    ],
    "stream": true
  }'
Python
import os
# Install SDK: python -m pip install --upgrade arkruntime
from arkruntime import Ark

client = Ark(
    # The base URL for model invocation
    base_url="https://ark.cn-beijing.volces.com/api/v3",
    # Get API Key: https://ark.volcengine.com/region:cn-beijing/apikey
    api_key=os.getenv('ARK_API_KEY'),
)

completion = client.chat.completions.create(
    # Replace with Model ID
    model="doubao-seed-2-1-pro-260915",
    messages=[
        {"role": "user", "content": "常见的十字花科植物有哪些?"},
    ],
    stream=True,
)

# Ensure the connection is closed automatically to prevent connection leaks.
with completion:
    for chunk in completion:
        if chunk.choices[0].delta.content is not None:
            print(chunk.choices[0].delta.content, end="")
Go
package main

import (
	"github.com/volcengine/ark-runtime-go/arkruntime"
	"github.com/volcengine/ark-runtime-go/arkruntime/model/chat"
	"context"
	"fmt"
	"io"
	"os"
)

func main() {
	client := arkruntime.NewClientWithApiKey(
		os.Getenv("ARK_API_KEY"),
		// The base URL for model invocation
		arkruntime.WithBaseUrl("https://ark.cn-beijing.volces.com/api/v3"),
	)

	ctx := context.Background()

	fmt.Println("----- standard request -----")
	req := &chat.ChatCompletionRequest{
		// Replace with Model ID
		Model: "doubao-seed-2-1-pro-260915",
		Messages: []chat.ChatCompletionRequestMessage{
			userMessage("常见的十字花科植物有哪些?"),
		},
		Stream: chat.NewOptBool(true),
	}

	// Use CreateChatCompletionStream; the non-streaming method cannot yield chunks.
	resp, err := client.CreateChatCompletionStream(ctx, req)
	if err != nil {
		fmt.Printf("standard chat error: %v", err)
		return
	}

	defer resp.Close()
	for {
		chunk, err := resp.Recv()
		if err == io.EOF {
			break
		}
		if err != nil {
			fmt.Printf("stream error: %v", err)
			break
		}
		fmt.Print(chunk.Choices[0].Delta.Content.Or(""))
	}
	fmt.Println()
}

func userMessage(content string) chat.ChatCompletionRequestMessage {
	return chat.ChatCompletionRequestMessage{
		OneOf: chat.NewChatCompletionRequestUserMessageChatCompletionRequestMessageSum(
			chat.ChatCompletionRequestUserMessage{
				Role:    chat.ChatCompletionRequestUserMessageRoleUser,
				Content: chat.NewOptNilChatCompletionMessageContent(chat.NewStringChatCompletionMessageContent(content)),
			},
		),
	}
}
Java
package com.volcengine.ark.runtime;

import com.volcengine.ark.runtime.models.chat.*;
import com.volcengine.ark.runtime.service.ArkService;
import java.util.ArrayList;
import java.util.List;

public class ChatCompletionsExample {
    public static void main(String[] args) {
        String apiKey = System.getenv("ARK_API_KEY");
        // The base URL for model invocation
        ArkService service = ArkService.builder().apiKey(apiKey)
                .baseUrl("https://ark.cn-beijing.volces.com/api/v3")
                .build();
        final List<ChatCompletionRequestMessage> messages = new ArrayList<>();
        final ChatCompletionRequestMessage userMessage = ChatCompletionRequestUserMessage.builder().role(ChatCompletionRequestMessageType.USER).content(ChatCompletionMessageContent.ofString("常见的十字花科植物有哪些?")).build();
        messages.add(userMessage);

        ChatCompletionRequest chatCompletionRequest = ChatCompletionRequest.builder()
               .model("doubao-seed-2-1-pro-260915") // Replace with Model ID
               .messages(messages)
               .stream(true)
               .thinking(Thinking.builder().type(ThinkingMode.DISABLED).build())
               .build();
        try {
            service.streamChatCompletion(chatCompletionRequest)
                    .doOnError(Throwable::printStackTrace) // Print stream errors.
                    .blockingForEach(response -> {
                        if (response.getChoices() != null && !response.getChoices().isEmpty()) {
                            Object content = response.getChoices().get(0).getDelta().getContent();
                            if (content != null) {
                                System.out.print(content); // Keep chunks contiguous.
                            }
                        }
                    });
        } finally {
            service.shutdownExecutor();
        }
    }
}
Python
import os
from openai import OpenAI

client = OpenAI(
    # Get API Key: https://ark.volcengine.com/region:cn-beijing/apikey
    api_key=os.environ.get("ARK_API_KEY"),
    base_url="https://ark.cn-beijing.volces.com/api/v3",
    )

completion = client.chat.completions.create(
    # Replace with Model ID
    model = "doubao-seed-2-1-pro-260915",
    messages=[
        {"role": "user", "content": "常见的十字花科植物有哪些?"},
    ],
    stream=True,
)

# Ensure the connection is closed automatically to prevent connection leaks.
with completion:
    for chunk in completion:
        if chunk.choices[0].delta.content is not None:
            print(chunk.choices[0].delta.content, end="")

Responses API#

以下 Go 示例调用专用流式方法 CreateResponsesStream,该方法会按流式模式发送请求,因此请求对象中无需再次设置 stream=true。

Bash
curl https://ark.cn-beijing.volces.com/api/v3/responses \
  -H "Authorization: Bearer $ARK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
      "model": "doubao-seed-2-1-pro-260915",
      "input": "常见的十字花科植物有哪些?",
      "stream": true
  }'
Python
import os
from arkruntime import Ark
from arkruntime.types.responses.response_completed_event import ResponseCompletedEvent
from arkruntime.types.responses.response_reasoning_summary_text_delta_event import ResponseReasoningSummaryTextDeltaEvent
from arkruntime.types.responses.response_output_item_added_event import ResponseOutputItemAddedEvent
from arkruntime.types.responses.response_text_delta_event import ResponseTextDeltaEvent
from arkruntime.types.responses.response_text_done_event import ResponseTextDoneEvent

# Get API Key: https://ark.volcengine.com/region:cn-beijing/apikey
api_key = os.getenv('ARK_API_KEY')

client = Ark(
    base_url='https://ark.cn-beijing.volces.com/api/v3',
    api_key=api_key,
)

# The context manager closes the stream on normal completion and exceptions.
with client.responses.create(
    model="doubao-seed-2-1-pro-260915",
    input="常见的十字花科植物有哪些?",
    stream=True,
) as response:
    for event in response:
        if isinstance(event, ResponseReasoningSummaryTextDeltaEvent):
            print(event.delta, end="")
        if isinstance(event, ResponseOutputItemAddedEvent):
            print("\nOutput item " + event.type + " start:")
        if isinstance(event, ResponseTextDeltaEvent):
            print(event.delta, end="")
        if isinstance(event, ResponseTextDoneEvent):
            print("\nOutput text done.")
        if isinstance(event, ResponseCompletedEvent):
            print("Response Completed. Usage = " + event.response.usage.model_dump_json())
Go
package main

import (
    "context"
    "fmt"
    "io"
    "os"

    "github.com/volcengine/ark-runtime-go/arkruntime"
    "github.com/volcengine/ark-runtime-go/arkruntime/model/responses"
)

func main() {
    client := arkruntime.NewClientWithApiKey(
        // Get API Key: https://ark.volcengine.com/region:cn-beijing/apikey
        os.Getenv("ARK_API_KEY"),
        arkruntime.WithBaseUrl("https://ark.cn-beijing.volces.com/api/v3"),
    )
    ctx := context.Background()

    resp, err := client.CreateResponsesStream(ctx, &responses.ResponsesRequest{
        Model:    "doubao-seed-2-1-pro-260915",
        Input:    responses.NewStringResponsesInput("常见的十字花科植物有哪些?"),

    })
    if err != nil {
        fmt.Printf("stream error: %v", err)
        return
    }
    defer resp.Close()
    for {
        event, err := resp.Recv()
        if err == io.EOF {
            break
        }
        if err != nil {
            fmt.Printf("stream error: %v", err)
            return
        }
        handleEvent(event)
    }
}
func handleEvent(event *responses.ResponseStreamEvent) {
    switch event.OneOf.Type {
    case responses.ResponseReasoningSummaryTextDeltaEventResponseStreamEventSum:
        print(event.OneOf.ResponseReasoningSummaryTextDeltaEvent.Delta.Or(""))
    case responses.ResponseReasoningSummaryTextDoneEventResponseStreamEventSum:
        fmt.Printf("\nAggregated reasoning text: %s\n", event.OneOf.ResponseReasoningSummaryTextDoneEvent.Text.Or(""))
    case responses.ResponseTextDeltaEventResponseStreamEventSum:
        print(event.OneOf.ResponseTextDeltaEvent.Delta.Or(""))
    case responses.ResponseTextDoneEventResponseStreamEventSum:
        fmt.Printf("\nAggregated output text: %s\n", event.OneOf.ResponseTextDoneEvent.Text.Or(""))
    case responses.ResponseCompletedEventResponseStreamEventSum:
        fmt.Println("\nResponse completed.")
    case responses.ResponseIncompleteEventResponseStreamEventSum:
        fmt.Println("\nResponse incomplete; inspect incomplete details before retrying.")
    case responses.ResponseFailedEventResponseStreamEventSum:
        fmt.Println("\nResponse failed; inspect the response error before retrying.")
    case responses.ResponseErrorEventResponseStreamEventSum:
        fmt.Println("\nStream error event received.")
    }
}
Java
package com.ark.example;
import com.volcengine.ark.runtime.service.ArkService;
import com.volcengine.ark.runtime.models.responses.*;
public class demo {
    public static void main(String[] args) {
        String apiKey = System.getenv("ARK_API_KEY");
        // The base URL for model invocation
        ArkService arkService = ArkService.builder().apiKey(apiKey)
                .baseUrl("https://ark.cn-beijing.volces.com/api/v3")
                .build();

        ResponsesRequest request = ResponsesRequest.builder()
                .model("doubao-seed-2-1-pro-260915")
                .stream(true)
                .input(ResponsesInput.ofString("常见的十字花科植物有哪些?"))

                .build();
        try {
            arkService.streamResponse(request)
                    .doOnError(Throwable::printStackTrace)
                    .blockingForEach(event -> {
                        if (event instanceof ResponseReasoningSummaryTextDeltaEvent) {
                            System.out.print(((ResponseReasoningSummaryTextDeltaEvent) event).getDelta());
                        }
                        if (event instanceof ResponseOutputItemAddedEvent) {
                            System.out.println("OutputItem " + (((ResponseOutputItemAddedEvent) event).getItem().getType()) + " Start: ");
                        }
                        if (event instanceof ResponseTextDeltaEvent) {
                            System.out.print(((ResponseTextDeltaEvent) event).getDelta());
                        }
                        if (event instanceof ResponseTextDoneEvent) {
                            System.out.println("OutputText End.");
                        }
                        if (event instanceof ResponseOutputItemDoneEvent) {
                            System.out.println("OutputItem " + ((ResponseOutputItemDoneEvent) event).getItem().getType() + " End.");
                        }
                        if (event instanceof ResponseFunctionCallArgumentsDoneEvent) {
                            System.out.println("FunctionCall Arguments: " + ((ResponseFunctionCallArgumentsDoneEvent) event).getArguments());
                        }
                        if (event instanceof ResponseCompletedEvent) {
                            System.out.println("Response Completed. Usage = " + ((ResponseCompletedEvent) event).getResponse().getUsage());
                        }
                    });
        } finally {
            arkService.shutdownExecutor();
        }
    }
}
Python
import os
from openai import OpenAI

# Get API Key: https://ark.volcengine.com/region:cn-beijing/apikey
api_key = os.getenv('ARK_API_KEY')

client = OpenAI(
    base_url='https://ark.cn-beijing.volces.com/api/v3',
    api_key=api_key,
)

# The context manager closes the stream on normal completion and exceptions.
with client.responses.create(
    model="doubao-seed-2-1-pro-260915",
    input="常见的十字花科植物有哪些?",
    stream=True
) as response:
    for event in response:
        if event.type == "response.reasoning_summary_text.delta":
            print(event.delta, end="")
        if event.type == "response.output_item.added":
            print("\nOutput item " + event.type + " start:")
        if event.type == "response.output_text.delta":
            print(event.delta, end="")
        if event.type == "response.output_item.done":
            print("\nOutput text done.")
        if event.type == "response.completed":
            print("\nResponse Completed. Usage = " + event.response.usage.model_dump_json())

Messages API#

Bash
curl -N https://ark.cn-beijing.volces.com/api/compatible/v1/messages \
  -H "Content-Type: application/json" \
  -H "x-api-key: $ARK_API_KEY" \
  -d '{
    "model": "doubao-seed-2-1-pro-260915",
    "max_tokens": 256,
    "stream": true,
    "messages": [
      {
        "role": "user",
        "content": "请用一句中文介绍什么是向量数据库。"
      }
    ]
  }'
Python
import json
import os

import requests

buffers = {}
block_types = {}

with requests.post(
    "https://ark.cn-beijing.volces.com/api/compatible/v1/messages",
    headers={
        "Content-Type": "application/json",
        "x-api-key": os.environ["ARK_API_KEY"],
    },
    json={
        "model": "doubao-seed-2-1-pro-260915",
        "max_tokens": 256,
        "stream": True,
        "messages": [
            {"role": "user", "content": "请用一句中文介绍什么是向量数据库。"}
        ],
    },
    stream=True,
    timeout=1800,
) as response:
    response.raise_for_status()

    for line in response.iter_lines(decode_unicode=True):
        if not line or not line.startswith("data: "):
            continue
        event = json.loads(line.removeprefix("data: "))
        event_type = event.get("type")

        if event_type == "content_block_start":
            index = event["index"]
            block_types[index] = event.get("content_block", {}).get("type", "")
            buffers[index] = {
                "text": [],
                "thinking": [],
                "signature": [],
                "partial_json": [],
            }
        elif event_type == "content_block_delta":
            index = event["index"]
            delta = event.get("delta", {})
            delta_type = delta.get("type")
            field = {
                "text_delta": "text",
                "thinking_delta": "thinking",
                "signature_delta": "signature",
                "input_json_delta": "partial_json",
            }.get(delta_type)
            if field:
                fragment = delta.get(field, "")
                state = buffers.setdefault(
                    index,
                    {"text": [], "thinking": [], "signature": [], "partial_json": []},
                )
                state[field].append(fragment)
                if delta_type == "text_delta":
                    print(fragment, end="", flush=True)
        elif event_type == "content_block_stop":
            index = event["index"]
            state = buffers.get(index, {})
            partial_json = "".join(state.get("partial_json", []))
            if block_types.get(index) == "tool_use" and partial_json:
                print(f"\nTool arguments: {json.loads(partial_json)}")
        elif event_type == "message_delta":
            stop_reason = event.get("delta", {}).get("stop_reason")
            if stop_reason:
                print(f"\nStop reason: {stop_reason}")
        elif event_type == "error":
            raise RuntimeError(json.dumps(event.get("error", {}), ensure_ascii=False))
        elif event_type == "message_stop":
            break
Go
package main

import (
	"bufio"
	"encoding/json"
	"fmt"
	"log"
	"net/http"
	"os"
	"strings"
)

type streamEvent struct {
	Type         string       `json:"type"`
	Index        int          `json:"index"`
	ContentBlock contentBlock `json:"content_block"`
	Delta        delta        `json:"delta"`
	Error        any          `json:"error"`
}

type contentBlock struct {
	Type string `json:"type"`
}

type delta struct {
	Type        string `json:"type"`
	Text        string `json:"text"`
	Thinking    string `json:"thinking"`
	Signature   string `json:"signature"`
	PartialJSON string `json:"partial_json"`
	StopReason  string `json:"stop_reason"`
}

type blockState struct {
	BlockType   string
	Text        strings.Builder
	Thinking    strings.Builder
	Signature   strings.Builder
	PartialJSON strings.Builder
}

func main() {
	body := `{"model":"doubao-seed-2-1-pro-260915","max_tokens":256,"stream":true,"messages":[{"role":"user","content":"请用一句中文介绍什么是向量数据库。"}]}`
	req, err := http.NewRequest("POST", "https://ark.cn-beijing.volces.com/api/compatible/v1/messages", strings.NewReader(body))
	if err != nil {
		log.Fatal(err)
	}
	req.Header.Set("Content-Type", "application/json")
	req.Header.Set("x-api-key", os.Getenv("ARK_API_KEY"))

	resp, err := http.DefaultClient.Do(req)
	if err != nil {
		log.Fatal(err)
	}
	defer resp.Body.Close()
	if resp.StatusCode < 200 || resp.StatusCode >= 300 {
		log.Fatalf("request failed: %s", resp.Status)
	}

	scanner := bufio.NewScanner(resp.Body)
	blocks := make(map[int]*blockState)
	for scanner.Scan() {
		line := scanner.Text()
		if !strings.HasPrefix(line, "data: ") {
			continue
		}

		var event streamEvent
		if err := json.Unmarshal([]byte(strings.TrimPrefix(line, "data: ")), &event); err != nil {
			log.Fatal(err)
		}
		switch event.Type {
		case "content_block_start":
			blocks[event.Index] = &blockState{BlockType: event.ContentBlock.Type}
		case "content_block_delta":
			state := blocks[event.Index]
			if state == nil {
				state = &blockState{}
				blocks[event.Index] = state
			}
			switch event.Delta.Type {
			case "text_delta":
				state.Text.WriteString(event.Delta.Text)
				fmt.Print(event.Delta.Text)
			case "thinking_delta":
				state.Thinking.WriteString(event.Delta.Thinking)
			case "signature_delta":
				state.Signature.WriteString(event.Delta.Signature)
			case "input_json_delta":
				state.PartialJSON.WriteString(event.Delta.PartialJSON)
			}
		case "content_block_stop":
			state := blocks[event.Index]
			if state != nil && state.BlockType == "tool_use" && state.PartialJSON.Len() > 0 {
				var arguments any
				if err := json.Unmarshal([]byte(state.PartialJSON.String()), &arguments); err != nil {
					log.Fatal(err)
				}
				fmt.Printf("\nTool arguments: %v\n", arguments)
			}
		case "message_delta":
			if event.Delta.StopReason != "" {
				fmt.Printf("\nStop reason: %s\n", event.Delta.StopReason)
			}
		case "error":
			log.Fatalf("stream error: %v", event.Error)
		case "message_stop":
			return
		}
	}
	if err := scanner.Err(); err != nil {
		log.Fatal(err)
	}
}
Java
import com.volcengine.ark.runtime.service.ArkService;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import okhttp3.MediaType;
import okhttp3.OkHttpClient;
import okhttp3.Request;
import okhttp3.RequestBody;
import okhttp3.Response;

public class MessagesStreamingExample {
    private static class BlockState {
        private String blockType = "";
        private final StringBuilder text = new StringBuilder();
        private final StringBuilder thinking = new StringBuilder();
        private final StringBuilder signature = new StringBuilder();
        private final StringBuilder partialJson = new StringBuilder();
    }

    public static void main(String[] args) throws IOException {
        OkHttpClient client = ArkService.defaultApiKeyClient(
                System.getenv("ARK_API_KEY"), ArkService.DEFAULT_TIMEOUT);
        ObjectMapper mapper = ArkService.defaultObjectMapper();
        Map<Integer, BlockState> blocks = new HashMap<>();
        String json = "{\"model\":\"doubao-seed-2-1-pro-260915\",\"max_tokens\":256,"
                + "\"stream\":true,\"messages\":[{\"role\":\"user\","
                + "\"content\":\"请用一句中文介绍什么是向量数据库。\"}]}";
        Request request = new Request.Builder()
                .url("https://ark.cn-beijing.volces.com/api/compatible/v1/messages")
                .header("Content-Type", "application/json")
                .header("x-api-key", System.getenv("ARK_API_KEY"))
                .post(RequestBody.create(MediaType.parse("application/json"), json))
                .build();
        try (Response response = client.newCall(request).execute()) {
            if (!response.isSuccessful() || response.body() == null) {
                throw new IOException("Request failed: " + response.code());
            }
            String line;
            while ((line = response.body().source().readUtf8Line()) != null) {
                if (!line.startsWith("data: ")) {
                    continue;
                }
                JsonNode event = mapper.readTree(line.substring("data: ".length()));
                String eventType = event.path("type").asText();
                int index = event.path("index").asInt(-1);

                if ("content_block_start".equals(eventType)) {
                    BlockState state = new BlockState();
                    state.blockType = event.path("content_block").path("type").asText();
                    blocks.put(index, state);
                } else if ("content_block_delta".equals(eventType)) {
                    JsonNode delta = event.path("delta");
                    String deltaType = delta.path("type").asText();
                    BlockState state = blocks.computeIfAbsent(index, ignored -> new BlockState());
                    if ("text_delta".equals(deltaType)) {
                        String fragment = delta.path("text").asText();
                        state.text.append(fragment);
                        System.out.print(fragment);
                    } else if ("thinking_delta".equals(deltaType)) {
                        state.thinking.append(delta.path("thinking").asText());
                    } else if ("signature_delta".equals(deltaType)) {
                        state.signature.append(delta.path("signature").asText());
                    } else if ("input_json_delta".equals(deltaType)) {
                        state.partialJson.append(delta.path("partial_json").asText());
                    }
                } else if ("content_block_stop".equals(eventType)) {
                    BlockState state = blocks.get(index);
                    if (state != null && "tool_use".equals(state.blockType)
                            && state.partialJson.length() > 0) {
                        JsonNode arguments = mapper.readTree(state.partialJson.toString());
                        System.out.println("\nTool arguments: " + arguments);
                    }
                } else if ("message_delta".equals(eventType)) {
                    String stopReason = event.path("delta").path("stop_reason").asText();
                    if (!stopReason.isEmpty()) {
                        System.out.println("\nStop reason: " + stopReason);
                    }
                } else if ("error".equals(eventType)) {
                    throw new IOException("Stream error: " + event.path("error"));
                } else if ("message_stop".equals(eventType)) {
                    break;
                }
            }
        } finally {
            client.dispatcher().executorService().shutdown();
            client.connectionPool().evictAll();
        }
    }
}

Chat 和 Responses 示例优先复用方舟 SDK 或兼容 SDK。当前方舟 SDK 没有统一的 Messages 原生资源方法,因此 Messages 示例使用 REST;Java 示例复用方舟 SDK 的 API Key HTTP 客户端。三类 API 使用不同的事件结构,不得共用事件解析分支。

支持模型#

流式输出适用于能够通过 Chat API、Responses API 或 Messages API 调用的 LLM 模型。代码中的 Model ID 只是示例值,不构成流式输出白名单。模型存在 API 或流式专项限制时,以 模型列表 和对应 API 参考为准。

配置流式请求#

快速开始中的示例已经包含启用流式输出所需的 stream=true。当流式输出与深度思考或工具调用组合时,还需在同一请求体中配置对应能力字段。

与深度思考组合#

json
{
  "model": "doubao-seed-2-1-pro-260915",
  "messages": [{"role": "user", "content": "证明两个奇数之和是偶数。"}],
  "thinking": {"type": "enabled"},
  "stream": true
}
json
{
  "model": "doubao-seed-2-1-pro-260915",
  "input": "证明两个奇数之和是偶数。",
  "thinking": {"type": "enabled"},
  "stream": true
}
json
{
  "model": "doubao-seed-2-1-pro-260915",
  "max_tokens": 1024,
  "thinking": {"type": "enabled"},
  "messages": [{"role": "user", "content": "证明两个奇数之和是偶数。"}],
  "stream": true
}

三类 API 的请求结构不能混用。思考内容与正文的分流方式参见 响应示例与解析。

与工具调用组合#

json
{
  "model": "doubao-seed-2-1-pro-260915",
  "messages": [{"role": "user", "content": "查询北京天气。"}],
  "tools": [{
    "type": "function",
    "function": {
      "name": "get_weather",
      "description": "Get weather for a city.",
      "parameters": {
        "type": "object",
        "properties": {"city": {"type": "string"}},
        "required": ["city"]
      }
    }
  }],
  "stream": true
}
json
{
  "model": "doubao-seed-2-1-pro-260915",
  "input": "查询北京天气。",
  "tools": [{
    "type": "function",
    "name": "get_weather",
    "description": "Get weather for a city.",
    "parameters": {
      "type": "object",
      "properties": {"city": {"type": "string"}},
      "required": ["city"]
    }
  }],
  "stream": true
}
json
{
  "model": "doubao-seed-2-1-pro-260915",
  "max_tokens": 1024,
  "messages": [{"role": "user", "content": "查询北京天气。"}],
  "tools": [{
    "name": "get_weather",
    "description": "Get weather for a city.",
    "input_schema": {
      "type": "object",
      "properties": {"city": {"type": "string"}},
      "required": ["city"]
    }
  }],
  "stream": true
}

工具参数可能跨多个事件返回。完整参数到达前不得解析 JSON 或执行工具;具体聚合键和完成边界参见 响应示例与解析 和 判断响应结果。

​​

处理流式响应#

响应示例与解析#

以下示例仅保留参与解析的字段,省略请求 ID、时间戳等通用元数据。事件中的文本和 token 数为结构示例,不代表固定输出。

data: {"choices":[{"index":0,"delta":{"reasoning_content":"正在整理常见植物。"}}],"usage":null}

data: {"choices":[{"index":0,"delta":{"content":"常见的十字花科植物包括白菜、甘蓝和油菜。"}}],"usage":null}

data: {"choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":null}

data: [DONE]

解析说明:

  • 先按 choices[].index 为每个候选结果建立独立状态。默认只请求一个候选时,该索引通常为 0,但客户端不应将所有数据块固定写入同一缓冲区。
  • 将同一候选结果的 delta.content 和 delta.reasoning_content 分别追加到正文与思考缓冲区;reasoning_content 的具体语义取决于模型。
  • delta.encrypted_content 返回完整值时,应整体保存或覆盖,不按字符串增量拼接。
  • 工具调用时,在候选结果下继续按 delta.tool_calls[].index 建立独立参数缓冲区。finish_reason 和 [DONE] 的结果分类参见 判断响应结果。
event: response.reasoning_summary_text.delta
data: {"type":"response.reasoning_summary_text.delta","item_id":"rs_...","output_index":0,"summary_index":0,"delta":"正在整理常见植物。","sequence_number":4}

event: response.output_text.delta
data: {"type":"response.output_text.delta","item_id":"msg_...","output_index":1,"content_index":0,"delta":"常见的十字花科植物包括白菜、甘蓝和油菜。","sequence_number":5}

event: response.completed
data: {"type":"response.completed","response":{"id":"resp_...","status":"completed","usage":{"input_tokens":18,"output_tokens":24,"total_tokens":42}},"sequence_number":6}

data: [DONE]

解析说明:

  • 按 item_id、output_index 和 content_index 聚合 response.output_text.delta,按 item_id、output_index 和 summary_index 聚合 response.reasoning_summary_text.delta。
  • 函数参数按 item_id 和 output_index 聚合 response.function_call_arguments.delta;对应的 response.function_call_arguments.done 只表示该函数参数已完整。
  • response.completed 中的 response.status 决定响应是否完整,response.usage 提供本次请求的用量;[DONE] 只表示传输结束。
event: message_start
data: {"type":"message_start","message":{"id":"msg_...","type":"message","role":"assistant","content":[],"stop_reason":null,"usage":{"input_tokens":18,"output_tokens":0}}}

event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"常见的十字花科植物包括白菜、甘蓝和油菜。"}}

event: content_block_stop
data: {"type":"content_block_stop","index":0}

event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"end_turn","stop_sequence":null},"usage":{"input_tokens":18,"output_tokens":24}}

event: message_stop
data: {"type":"message_stop"}

解析说明:

  • 按内容块 index 建立缓冲区,再根据 delta.type 将 text、thinking、signature 或 partial_json 追加到对应字段。
  • content_block_stop 只完成匹配 index 的内容块。工具内容块应在此时解析已聚合的 partial_json,但整条消息仍未完成。
  • 从 message_delta.delta.stop_reason 读取结果原因,并在 message_stop 到达后结束消费。usage 快照的覆盖规则参见 记录用量。

增量字段非空只表示当前事件带来新片段,不代表响应完整。usage 缺失或为 null 时,将统计标记为未采集,不记为 0。

判断响应结果#

语义终态是 API 对本次生成结果状态的明确结论,用于判断结果是完整、部分、等待工具还是失败。[DONE]、EOF 或连接关闭只是传输结束,不能单独证明生成成功。

API 语义信号 结果分类 客户端动作
Chat finish_reason=stop 完整 提交该 choice.index 的正文结果。
Chat finish_reason=length 或 content_filter 部分 保留已接收内容并标记不完整;前者可在确认业务需要后调整输出或输入长度,后者不得通过自动重试规避拦截。
Chat finish_reason=tool_calls 工具中间态 将该候选结果下各工具参数缓冲区标记为完整,解析参数并进入工具调用流程;不得当作最终文本回答。
Chat data: [DONE] 仅传输结束 以此前各候选结果的 finish_reason 判定状态;缺少语义终态时按提前中断处理。
Responses response.completed 且 response.status=completed 完整 提交已聚合的输出项。
Responses response.incomplete 部分 读取 response.incomplete_details,保留部分结果,再决定是否续写或调整限制。
Responses response.failed 或 error 失败 读取 response.error 或错误事件中的 code、message;仅在原因可重试且业务允许重复生成时重新请求。
Responses data: [DONE] 仅传输结束 以此前的 response.completed、response.incomplete、response.failed 或 error 判定状态。
Messages stop_reason=end_turn 或 stop_reason=stop_sequence,随后收到 message_stop 完整 提交已完成的内容块;命中停止序列时同时记录 stop_sequence。
Messages stop_reason=max_tokens,随后收到 message_stop 部分 保留已接收内容并标记不完整;确认业务需要后调整 max_tokens 或输入长度。
Messages stop_reason=tool_use,随后收到 message_stop 工具中间态 执行已完成的工具调用,并按 Messages 工具调用流程续接;不得当作最终文本回答。
Messages error 事件或 HTTP 请求失败 失败 保留错误信息并关闭连接;仅在原因可重试且业务允许重复生成时重新请求。
Messages 仅收到 message_stop,没有可识别的 stop_reason 未确认 不将结果标记为完整,记录原始事件并按异常结果处理。

记录用量#

  • 完整用量:将请求体字段 stream_options.include_usage 设为 true。服务端通常在 data: [DONE] 前增加一个 choices=[] 的数据块,其 usage 表示整次请求的 token 用量。
  • 逐块累计用量:将 stream_options.chunk_include_usage 设为 true。每个数据块的 usage 表示截至当前时刻的累计值;客户端只保留最新值,不得逐块相加。
  • 模型例外:模型 deepseek-v4-1-flash-economy-260910 不受 stream_options.include_usage 控制。该模型默认在带 finish_reason 的数据块中返回 usage,不额外返回独立的用量数据块。

在 response.completed、response.incomplete 或 response.failed 终态事件的 response.usage 中读取当前请求的用量。终态前断连且没有返回该字段时,将统计标记为未采集,不记为 0。

Messages 的 usage 是累计快照,不是需要求和的增量:

  1. 收到 message_start 时,用 message_start.message.usage 初始化当前快照。该快照包含输入和缓存相关用量;生成刚开始时,output_tokens 通常为 0。
  2. 收到带 usage 的 message_delta 时,用事件中的同名字段覆盖当前快照。不得将 input_tokens、output_tokens 或缓存 token 与上一快照相加。
  3. 收到 message_stop 后,将最新快照标记为本次请求的最终统计。
  4. 如果在 message_stop 前断连,只能保留最后一次快照并标记为“未最终确认”;没有收到 usage 时标记为“未采集”,不得记为 0。

处理连接中断#

在正常终态前出现 EOF、超时或网络断开时,已接收内容只是部分结果。客户端应标记部分结果并释放连接;仅在具备幂等策略且业务允许重复生成时重新请求。本文不定义断点续传。

Messages 以 message_stop 收尾,不使用 [DONE] 代替消息终态;结果分类仍由此前 message_delta.delta.stop_reason 决定。

工作原理#

客户端、方舟 API 和模型之间的流式请求、增量推送与终态收尾时序
  • 建立连接:客户端发送包含 stream=true 的请求,并为每个并发请求维护独立的 SSE 连接。
  • 聚合增量:客户端按所选 API 的事件类型分派数据,再使用该事件声明的索引将片段追加到对应输出项。
  • 判断完成:客户端使用所选 API 的语义终态提交结果;若在终态前断连,则将已接收内容标记为部分结果并释放连接。

使用说明与限制#

运行观测#

观测维度 预期影响 指标或信号 阈值或单位
首段内容等待时间 客户端可在完整响应结束前开始处理内容;这不代表总生成时间缩短。 从发出请求到收到首个正文或思考 delta 的时间,并以毫秒记录。 毫秒;本文不提供固定阈值。
token 用量 三类 API 的统计位置不同,中断时统计可能缺失。 Chat 的有效 usage 数据块;Responses 终态事件的 response.usage;Messages 的 message_start.message.usage 或 message_delta.usage。 token;缺失不记为零,累计快照不重复相加。
连接资源 流对象未关闭会持续占用客户端连接资源。 活跃连接数、客户端超时、消费循环是否退出、流对象是否关闭。 连接数、秒;按业务容量设置告警。

限制与不支持项#

维度 具体值 后果 处理方式
参数类型 stream 不是 boolean 请求参数类型不符合 Chat API、Responses API 或 Messages API 定义。 仅传 true 或 false。
能力组合 Responses API 创建前缀缓存时 stream=true 无法在创建前缀缓存的同一次请求中使用流式输出。 先以非流式请求创建前缀缓存,再通过 previous_response_id 续接后续请求。
连接释放 客户端停止消费但未关闭流对象 连接和套接字资源可能继续占用。 正常和异常路径都关闭流对象;设置有界的连接与读取超时。

相关文档#

  • Chat API:核对数据块结构、delta、finish_reason 和 stream_options。
  • Responses API:核对类型化事件、响应状态和 response.usage。
  • Messages API:核对消息与内容块事件、stop_reason 和公开用量字段。
  • Function Calling:了解工具定义、参数回传和多轮工具执行流程。
  • 上下文缓存:了解流式输出与隐式缓存、显式缓存及前缀缓存的组合限制。