在实时对话、长内容生成或思考过程展示中,等待完整响应后再显示结果会让用户长时间看不到进展。流式输出让客户端边接收边展示生成内容;如果应用需要更快反馈、增量处理或保留并标记不完整的中断前结果,本文将帮助你在现有模型调用 API 中启用流式输出、正确判断终态并管理连接。
快速开始#
| 效果示意 | 优势说明 |
|---|---|
![]() |
更早展示内容:收到首个片段后即可开始渲染,不必等待完整响应。 |
| 提供实时反馈:对话界面、长内容生成和思考过程可以持续更新。 | |
| 支持增量处理:客户端可以分别聚合正文、思考内容和工具参数。 | |
| 保留部分结果:终态前断连时可保留已接收内容,但必须标记为不完整。 |
动画中的时间仅用于展示返回方式,不代表实际首段时延或总生成时间。
按现有 API 接入#
启用流式输出不需要迁移 API。根据应用当前使用的 API,增加流式请求字段,并适配对应的响应事件:
| 当前 API | 请求改动 | 响应处理 |
|---|---|---|
| Chat API | 在请求体中设置 stream=true。 |
按 choices[].index 聚合 delta,并根据 finish_reason 判断结果。实现方法参见 处理流式响应。 |
| Responses API | 在请求体中设置 stream=true;Go SDK 使用 CreateResponsesStream。 |
按事件 type 分派并使用复合索引聚合。实现方法参见 处理流式响应。 |
| Messages API | 在请求体中设置 stream=true。 |
按消息与内容块生命周期聚合,并结合 stop_reason 与 message_stop 判断结果。实现方法参见 处理流式响应。 |
选择与现有应用相同的 API 示例。三类 API 的事件结构不同,不得共用事件解析分支。首次体验可直接运行 Chat API 的 cURL 或 Python 示例。
运行 Python 方舟 SDK 示例前,执行 python -m pip install --upgrade arkruntime;运行 Go 方舟 SDK 示例前,执行 go get github.com/volcengine/ark-runtime-go@latest。Messages Python REST 示例需使用 Python 3.9 或更高版本并安装 requests。
Chat API#
curl https://ark.cn-beijing.volces.com/api/v3/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $ARK_API_KEY" \
-d '{
"model": "doubao-seed-2-1-pro-260915",
"messages": [
{
"role": "user",
"content": "常见的十字花科植物有哪些?"
}
],
"stream": true
}'
import os
# Install SDK: python -m pip install --upgrade arkruntime
from arkruntime import Ark
client = Ark(
# The base URL for model invocation
base_url="https://ark.cn-beijing.volces.com/api/v3",
# Get API Key: https://ark.volcengine.com/region:cn-beijing/apikey
api_key=os.getenv('ARK_API_KEY'),
)
completion = client.chat.completions.create(
# Replace with Model ID
model="doubao-seed-2-1-pro-260915",
messages=[
{"role": "user", "content": "常见的十字花科植物有哪些?"},
],
stream=True,
)
# Ensure the connection is closed automatically to prevent connection leaks.
with completion:
for chunk in completion:
if chunk.choices[0].delta.content is not None:
print(chunk.choices[0].delta.content, end="")
package main
import (
"github.com/volcengine/ark-runtime-go/arkruntime"
"github.com/volcengine/ark-runtime-go/arkruntime/model/chat"
"context"
"fmt"
"io"
"os"
)
func main() {
client := arkruntime.NewClientWithApiKey(
os.Getenv("ARK_API_KEY"),
// The base URL for model invocation
arkruntime.WithBaseUrl("https://ark.cn-beijing.volces.com/api/v3"),
)
ctx := context.Background()
fmt.Println("----- standard request -----")
req := &chat.ChatCompletionRequest{
// Replace with Model ID
Model: "doubao-seed-2-1-pro-260915",
Messages: []chat.ChatCompletionRequestMessage{
userMessage("常见的十字花科植物有哪些?"),
},
Stream: chat.NewOptBool(true),
}
// Use CreateChatCompletionStream; the non-streaming method cannot yield chunks.
resp, err := client.CreateChatCompletionStream(ctx, req)
if err != nil {
fmt.Printf("standard chat error: %v", err)
return
}
defer resp.Close()
for {
chunk, err := resp.Recv()
if err == io.EOF {
break
}
if err != nil {
fmt.Printf("stream error: %v", err)
break
}
fmt.Print(chunk.Choices[0].Delta.Content.Or(""))
}
fmt.Println()
}
func userMessage(content string) chat.ChatCompletionRequestMessage {
return chat.ChatCompletionRequestMessage{
OneOf: chat.NewChatCompletionRequestUserMessageChatCompletionRequestMessageSum(
chat.ChatCompletionRequestUserMessage{
Role: chat.ChatCompletionRequestUserMessageRoleUser,
Content: chat.NewOptNilChatCompletionMessageContent(chat.NewStringChatCompletionMessageContent(content)),
},
),
}
}
package com.volcengine.ark.runtime;
import com.volcengine.ark.runtime.models.chat.*;
import com.volcengine.ark.runtime.service.ArkService;
import java.util.ArrayList;
import java.util.List;
public class ChatCompletionsExample {
public static void main(String[] args) {
String apiKey = System.getenv("ARK_API_KEY");
// The base URL for model invocation
ArkService service = ArkService.builder().apiKey(apiKey)
.baseUrl("https://ark.cn-beijing.volces.com/api/v3")
.build();
final List<ChatCompletionRequestMessage> messages = new ArrayList<>();
final ChatCompletionRequestMessage userMessage = ChatCompletionRequestUserMessage.builder().role(ChatCompletionRequestMessageType.USER).content(ChatCompletionMessageContent.ofString("常见的十字花科植物有哪些?")).build();
messages.add(userMessage);
ChatCompletionRequest chatCompletionRequest = ChatCompletionRequest.builder()
.model("doubao-seed-2-1-pro-260915") // Replace with Model ID
.messages(messages)
.stream(true)
.thinking(Thinking.builder().type(ThinkingMode.DISABLED).build())
.build();
try {
service.streamChatCompletion(chatCompletionRequest)
.doOnError(Throwable::printStackTrace) // Print stream errors.
.blockingForEach(response -> {
if (response.getChoices() != null && !response.getChoices().isEmpty()) {
Object content = response.getChoices().get(0).getDelta().getContent();
if (content != null) {
System.out.print(content); // Keep chunks contiguous.
}
}
});
} finally {
service.shutdownExecutor();
}
}
}
import os
from openai import OpenAI
client = OpenAI(
# Get API Key: https://ark.volcengine.com/region:cn-beijing/apikey
api_key=os.environ.get("ARK_API_KEY"),
base_url="https://ark.cn-beijing.volces.com/api/v3",
)
completion = client.chat.completions.create(
# Replace with Model ID
model = "doubao-seed-2-1-pro-260915",
messages=[
{"role": "user", "content": "常见的十字花科植物有哪些?"},
],
stream=True,
)
# Ensure the connection is closed automatically to prevent connection leaks.
with completion:
for chunk in completion:
if chunk.choices[0].delta.content is not None:
print(chunk.choices[0].delta.content, end="")
Responses API#
以下 Go 示例调用专用流式方法 CreateResponsesStream,该方法会按流式模式发送请求,因此请求对象中无需再次设置 stream=true。
curl https://ark.cn-beijing.volces.com/api/v3/responses \
-H "Authorization: Bearer $ARK_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "doubao-seed-2-1-pro-260915",
"input": "常见的十字花科植物有哪些?",
"stream": true
}'
import os
from arkruntime import Ark
from arkruntime.types.responses.response_completed_event import ResponseCompletedEvent
from arkruntime.types.responses.response_reasoning_summary_text_delta_event import ResponseReasoningSummaryTextDeltaEvent
from arkruntime.types.responses.response_output_item_added_event import ResponseOutputItemAddedEvent
from arkruntime.types.responses.response_text_delta_event import ResponseTextDeltaEvent
from arkruntime.types.responses.response_text_done_event import ResponseTextDoneEvent
# Get API Key: https://ark.volcengine.com/region:cn-beijing/apikey
api_key = os.getenv('ARK_API_KEY')
client = Ark(
base_url='https://ark.cn-beijing.volces.com/api/v3',
api_key=api_key,
)
# The context manager closes the stream on normal completion and exceptions.
with client.responses.create(
model="doubao-seed-2-1-pro-260915",
input="常见的十字花科植物有哪些?",
stream=True,
) as response:
for event in response:
if isinstance(event, ResponseReasoningSummaryTextDeltaEvent):
print(event.delta, end="")
if isinstance(event, ResponseOutputItemAddedEvent):
print("\nOutput item " + event.type + " start:")
if isinstance(event, ResponseTextDeltaEvent):
print(event.delta, end="")
if isinstance(event, ResponseTextDoneEvent):
print("\nOutput text done.")
if isinstance(event, ResponseCompletedEvent):
print("Response Completed. Usage = " + event.response.usage.model_dump_json())
package main
import (
"context"
"fmt"
"io"
"os"
"github.com/volcengine/ark-runtime-go/arkruntime"
"github.com/volcengine/ark-runtime-go/arkruntime/model/responses"
)
func main() {
client := arkruntime.NewClientWithApiKey(
// Get API Key: https://ark.volcengine.com/region:cn-beijing/apikey
os.Getenv("ARK_API_KEY"),
arkruntime.WithBaseUrl("https://ark.cn-beijing.volces.com/api/v3"),
)
ctx := context.Background()
resp, err := client.CreateResponsesStream(ctx, &responses.ResponsesRequest{
Model: "doubao-seed-2-1-pro-260915",
Input: responses.NewStringResponsesInput("常见的十字花科植物有哪些?"),
})
if err != nil {
fmt.Printf("stream error: %v", err)
return
}
defer resp.Close()
for {
event, err := resp.Recv()
if err == io.EOF {
break
}
if err != nil {
fmt.Printf("stream error: %v", err)
return
}
handleEvent(event)
}
}
func handleEvent(event *responses.ResponseStreamEvent) {
switch event.OneOf.Type {
case responses.ResponseReasoningSummaryTextDeltaEventResponseStreamEventSum:
print(event.OneOf.ResponseReasoningSummaryTextDeltaEvent.Delta.Or(""))
case responses.ResponseReasoningSummaryTextDoneEventResponseStreamEventSum:
fmt.Printf("\nAggregated reasoning text: %s\n", event.OneOf.ResponseReasoningSummaryTextDoneEvent.Text.Or(""))
case responses.ResponseTextDeltaEventResponseStreamEventSum:
print(event.OneOf.ResponseTextDeltaEvent.Delta.Or(""))
case responses.ResponseTextDoneEventResponseStreamEventSum:
fmt.Printf("\nAggregated output text: %s\n", event.OneOf.ResponseTextDoneEvent.Text.Or(""))
case responses.ResponseCompletedEventResponseStreamEventSum:
fmt.Println("\nResponse completed.")
case responses.ResponseIncompleteEventResponseStreamEventSum:
fmt.Println("\nResponse incomplete; inspect incomplete details before retrying.")
case responses.ResponseFailedEventResponseStreamEventSum:
fmt.Println("\nResponse failed; inspect the response error before retrying.")
case responses.ResponseErrorEventResponseStreamEventSum:
fmt.Println("\nStream error event received.")
}
}
package com.ark.example;
import com.volcengine.ark.runtime.service.ArkService;
import com.volcengine.ark.runtime.models.responses.*;
public class demo {
public static void main(String[] args) {
String apiKey = System.getenv("ARK_API_KEY");
// The base URL for model invocation
ArkService arkService = ArkService.builder().apiKey(apiKey)
.baseUrl("https://ark.cn-beijing.volces.com/api/v3")
.build();
ResponsesRequest request = ResponsesRequest.builder()
.model("doubao-seed-2-1-pro-260915")
.stream(true)
.input(ResponsesInput.ofString("常见的十字花科植物有哪些?"))
.build();
try {
arkService.streamResponse(request)
.doOnError(Throwable::printStackTrace)
.blockingForEach(event -> {
if (event instanceof ResponseReasoningSummaryTextDeltaEvent) {
System.out.print(((ResponseReasoningSummaryTextDeltaEvent) event).getDelta());
}
if (event instanceof ResponseOutputItemAddedEvent) {
System.out.println("OutputItem " + (((ResponseOutputItemAddedEvent) event).getItem().getType()) + " Start: ");
}
if (event instanceof ResponseTextDeltaEvent) {
System.out.print(((ResponseTextDeltaEvent) event).getDelta());
}
if (event instanceof ResponseTextDoneEvent) {
System.out.println("OutputText End.");
}
if (event instanceof ResponseOutputItemDoneEvent) {
System.out.println("OutputItem " + ((ResponseOutputItemDoneEvent) event).getItem().getType() + " End.");
}
if (event instanceof ResponseFunctionCallArgumentsDoneEvent) {
System.out.println("FunctionCall Arguments: " + ((ResponseFunctionCallArgumentsDoneEvent) event).getArguments());
}
if (event instanceof ResponseCompletedEvent) {
System.out.println("Response Completed. Usage = " + ((ResponseCompletedEvent) event).getResponse().getUsage());
}
});
} finally {
arkService.shutdownExecutor();
}
}
}
import os
from openai import OpenAI
# Get API Key: https://ark.volcengine.com/region:cn-beijing/apikey
api_key = os.getenv('ARK_API_KEY')
client = OpenAI(
base_url='https://ark.cn-beijing.volces.com/api/v3',
api_key=api_key,
)
# The context manager closes the stream on normal completion and exceptions.
with client.responses.create(
model="doubao-seed-2-1-pro-260915",
input="常见的十字花科植物有哪些?",
stream=True
) as response:
for event in response:
if event.type == "response.reasoning_summary_text.delta":
print(event.delta, end="")
if event.type == "response.output_item.added":
print("\nOutput item " + event.type + " start:")
if event.type == "response.output_text.delta":
print(event.delta, end="")
if event.type == "response.output_item.done":
print("\nOutput text done.")
if event.type == "response.completed":
print("\nResponse Completed. Usage = " + event.response.usage.model_dump_json())
Messages API#
curl -N https://ark.cn-beijing.volces.com/api/compatible/v1/messages \
-H "Content-Type: application/json" \
-H "x-api-key: $ARK_API_KEY" \
-d '{
"model": "doubao-seed-2-1-pro-260915",
"max_tokens": 256,
"stream": true,
"messages": [
{
"role": "user",
"content": "请用一句中文介绍什么是向量数据库。"
}
]
}'
import json
import os
import requests
buffers = {}
block_types = {}
with requests.post(
"https://ark.cn-beijing.volces.com/api/compatible/v1/messages",
headers={
"Content-Type": "application/json",
"x-api-key": os.environ["ARK_API_KEY"],
},
json={
"model": "doubao-seed-2-1-pro-260915",
"max_tokens": 256,
"stream": True,
"messages": [
{"role": "user", "content": "请用一句中文介绍什么是向量数据库。"}
],
},
stream=True,
timeout=1800,
) as response:
response.raise_for_status()
for line in response.iter_lines(decode_unicode=True):
if not line or not line.startswith("data: "):
continue
event = json.loads(line.removeprefix("data: "))
event_type = event.get("type")
if event_type == "content_block_start":
index = event["index"]
block_types[index] = event.get("content_block", {}).get("type", "")
buffers[index] = {
"text": [],
"thinking": [],
"signature": [],
"partial_json": [],
}
elif event_type == "content_block_delta":
index = event["index"]
delta = event.get("delta", {})
delta_type = delta.get("type")
field = {
"text_delta": "text",
"thinking_delta": "thinking",
"signature_delta": "signature",
"input_json_delta": "partial_json",
}.get(delta_type)
if field:
fragment = delta.get(field, "")
state = buffers.setdefault(
index,
{"text": [], "thinking": [], "signature": [], "partial_json": []},
)
state[field].append(fragment)
if delta_type == "text_delta":
print(fragment, end="", flush=True)
elif event_type == "content_block_stop":
index = event["index"]
state = buffers.get(index, {})
partial_json = "".join(state.get("partial_json", []))
if block_types.get(index) == "tool_use" and partial_json:
print(f"\nTool arguments: {json.loads(partial_json)}")
elif event_type == "message_delta":
stop_reason = event.get("delta", {}).get("stop_reason")
if stop_reason:
print(f"\nStop reason: {stop_reason}")
elif event_type == "error":
raise RuntimeError(json.dumps(event.get("error", {}), ensure_ascii=False))
elif event_type == "message_stop":
break
package main
import (
"bufio"
"encoding/json"
"fmt"
"log"
"net/http"
"os"
"strings"
)
type streamEvent struct {
Type string `json:"type"`
Index int `json:"index"`
ContentBlock contentBlock `json:"content_block"`
Delta delta `json:"delta"`
Error any `json:"error"`
}
type contentBlock struct {
Type string `json:"type"`
}
type delta struct {
Type string `json:"type"`
Text string `json:"text"`
Thinking string `json:"thinking"`
Signature string `json:"signature"`
PartialJSON string `json:"partial_json"`
StopReason string `json:"stop_reason"`
}
type blockState struct {
BlockType string
Text strings.Builder
Thinking strings.Builder
Signature strings.Builder
PartialJSON strings.Builder
}
func main() {
body := `{"model":"doubao-seed-2-1-pro-260915","max_tokens":256,"stream":true,"messages":[{"role":"user","content":"请用一句中文介绍什么是向量数据库。"}]}`
req, err := http.NewRequest("POST", "https://ark.cn-beijing.volces.com/api/compatible/v1/messages", strings.NewReader(body))
if err != nil {
log.Fatal(err)
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("x-api-key", os.Getenv("ARK_API_KEY"))
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatal(err)
}
defer resp.Body.Close()
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
log.Fatalf("request failed: %s", resp.Status)
}
scanner := bufio.NewScanner(resp.Body)
blocks := make(map[int]*blockState)
for scanner.Scan() {
line := scanner.Text()
if !strings.HasPrefix(line, "data: ") {
continue
}
var event streamEvent
if err := json.Unmarshal([]byte(strings.TrimPrefix(line, "data: ")), &event); err != nil {
log.Fatal(err)
}
switch event.Type {
case "content_block_start":
blocks[event.Index] = &blockState{BlockType: event.ContentBlock.Type}
case "content_block_delta":
state := blocks[event.Index]
if state == nil {
state = &blockState{}
blocks[event.Index] = state
}
switch event.Delta.Type {
case "text_delta":
state.Text.WriteString(event.Delta.Text)
fmt.Print(event.Delta.Text)
case "thinking_delta":
state.Thinking.WriteString(event.Delta.Thinking)
case "signature_delta":
state.Signature.WriteString(event.Delta.Signature)
case "input_json_delta":
state.PartialJSON.WriteString(event.Delta.PartialJSON)
}
case "content_block_stop":
state := blocks[event.Index]
if state != nil && state.BlockType == "tool_use" && state.PartialJSON.Len() > 0 {
var arguments any
if err := json.Unmarshal([]byte(state.PartialJSON.String()), &arguments); err != nil {
log.Fatal(err)
}
fmt.Printf("\nTool arguments: %v\n", arguments)
}
case "message_delta":
if event.Delta.StopReason != "" {
fmt.Printf("\nStop reason: %s\n", event.Delta.StopReason)
}
case "error":
log.Fatalf("stream error: %v", event.Error)
case "message_stop":
return
}
}
if err := scanner.Err(); err != nil {
log.Fatal(err)
}
}
import com.volcengine.ark.runtime.service.ArkService;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import okhttp3.MediaType;
import okhttp3.OkHttpClient;
import okhttp3.Request;
import okhttp3.RequestBody;
import okhttp3.Response;
public class MessagesStreamingExample {
private static class BlockState {
private String blockType = "";
private final StringBuilder text = new StringBuilder();
private final StringBuilder thinking = new StringBuilder();
private final StringBuilder signature = new StringBuilder();
private final StringBuilder partialJson = new StringBuilder();
}
public static void main(String[] args) throws IOException {
OkHttpClient client = ArkService.defaultApiKeyClient(
System.getenv("ARK_API_KEY"), ArkService.DEFAULT_TIMEOUT);
ObjectMapper mapper = ArkService.defaultObjectMapper();
Map<Integer, BlockState> blocks = new HashMap<>();
String json = "{\"model\":\"doubao-seed-2-1-pro-260915\",\"max_tokens\":256,"
+ "\"stream\":true,\"messages\":[{\"role\":\"user\","
+ "\"content\":\"请用一句中文介绍什么是向量数据库。\"}]}";
Request request = new Request.Builder()
.url("https://ark.cn-beijing.volces.com/api/compatible/v1/messages")
.header("Content-Type", "application/json")
.header("x-api-key", System.getenv("ARK_API_KEY"))
.post(RequestBody.create(MediaType.parse("application/json"), json))
.build();
try (Response response = client.newCall(request).execute()) {
if (!response.isSuccessful() || response.body() == null) {
throw new IOException("Request failed: " + response.code());
}
String line;
while ((line = response.body().source().readUtf8Line()) != null) {
if (!line.startsWith("data: ")) {
continue;
}
JsonNode event = mapper.readTree(line.substring("data: ".length()));
String eventType = event.path("type").asText();
int index = event.path("index").asInt(-1);
if ("content_block_start".equals(eventType)) {
BlockState state = new BlockState();
state.blockType = event.path("content_block").path("type").asText();
blocks.put(index, state);
} else if ("content_block_delta".equals(eventType)) {
JsonNode delta = event.path("delta");
String deltaType = delta.path("type").asText();
BlockState state = blocks.computeIfAbsent(index, ignored -> new BlockState());
if ("text_delta".equals(deltaType)) {
String fragment = delta.path("text").asText();
state.text.append(fragment);
System.out.print(fragment);
} else if ("thinking_delta".equals(deltaType)) {
state.thinking.append(delta.path("thinking").asText());
} else if ("signature_delta".equals(deltaType)) {
state.signature.append(delta.path("signature").asText());
} else if ("input_json_delta".equals(deltaType)) {
state.partialJson.append(delta.path("partial_json").asText());
}
} else if ("content_block_stop".equals(eventType)) {
BlockState state = blocks.get(index);
if (state != null && "tool_use".equals(state.blockType)
&& state.partialJson.length() > 0) {
JsonNode arguments = mapper.readTree(state.partialJson.toString());
System.out.println("\nTool arguments: " + arguments);
}
} else if ("message_delta".equals(eventType)) {
String stopReason = event.path("delta").path("stop_reason").asText();
if (!stopReason.isEmpty()) {
System.out.println("\nStop reason: " + stopReason);
}
} else if ("error".equals(eventType)) {
throw new IOException("Stream error: " + event.path("error"));
} else if ("message_stop".equals(eventType)) {
break;
}
}
} finally {
client.dispatcher().executorService().shutdown();
client.connectionPool().evictAll();
}
}
}
Chat 和 Responses 示例优先复用方舟 SDK 或兼容 SDK。当前方舟 SDK 没有统一的 Messages 原生资源方法,因此 Messages 示例使用 REST;Java 示例复用方舟 SDK 的 API Key HTTP 客户端。三类 API 使用不同的事件结构,不得共用事件解析分支。
支持模型#
流式输出适用于能够通过 Chat API、Responses API 或 Messages API 调用的 LLM 模型。代码中的 Model ID 只是示例值,不构成流式输出白名单。模型存在 API 或流式专项限制时,以 模型列表 和对应 API 参考为准。
配置流式请求#
快速开始中的示例已经包含启用流式输出所需的 stream=true。当流式输出与深度思考或工具调用组合时,还需在同一请求体中配置对应能力字段。
与深度思考组合#
{
"model": "doubao-seed-2-1-pro-260915",
"messages": [{"role": "user", "content": "证明两个奇数之和是偶数。"}],
"thinking": {"type": "enabled"},
"stream": true
}
{
"model": "doubao-seed-2-1-pro-260915",
"input": "证明两个奇数之和是偶数。",
"thinking": {"type": "enabled"},
"stream": true
}
{
"model": "doubao-seed-2-1-pro-260915",
"max_tokens": 1024,
"thinking": {"type": "enabled"},
"messages": [{"role": "user", "content": "证明两个奇数之和是偶数。"}],
"stream": true
}
三类 API 的请求结构不能混用。思考内容与正文的分流方式参见 响应示例与解析。
与工具调用组合#
{
"model": "doubao-seed-2-1-pro-260915",
"messages": [{"role": "user", "content": "查询北京天气。"}],
"tools": [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get weather for a city.",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"]
}
}
}],
"stream": true
}
{
"model": "doubao-seed-2-1-pro-260915",
"input": "查询北京天气。",
"tools": [{
"type": "function",
"name": "get_weather",
"description": "Get weather for a city.",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"]
}
}],
"stream": true
}
{
"model": "doubao-seed-2-1-pro-260915",
"max_tokens": 1024,
"messages": [{"role": "user", "content": "查询北京天气。"}],
"tools": [{
"name": "get_weather",
"description": "Get weather for a city.",
"input_schema": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"]
}
}],
"stream": true
}
工具参数可能跨多个事件返回。完整参数到达前不得解析 JSON 或执行工具;具体聚合键和完成边界参见 响应示例与解析 和 判断响应结果。
处理流式响应#
响应示例与解析#
以下示例仅保留参与解析的字段,省略请求 ID、时间戳等通用元数据。事件中的文本和 token 数为结构示例,不代表固定输出。
data: {"choices":[{"index":0,"delta":{"reasoning_content":"正在整理常见植物。"}}],"usage":null}
data: {"choices":[{"index":0,"delta":{"content":"常见的十字花科植物包括白菜、甘蓝和油菜。"}}],"usage":null}
data: {"choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":null}
data: [DONE]
解析说明:
- 先按
choices[].index为每个候选结果建立独立状态。默认只请求一个候选时,该索引通常为0,但客户端不应将所有数据块固定写入同一缓冲区。 - 将同一候选结果的
delta.content和delta.reasoning_content分别追加到正文与思考缓冲区;reasoning_content的具体语义取决于模型。 delta.encrypted_content返回完整值时,应整体保存或覆盖,不按字符串增量拼接。- 工具调用时,在候选结果下继续按
delta.tool_calls[].index建立独立参数缓冲区。finish_reason和[DONE]的结果分类参见 判断响应结果。
event: response.reasoning_summary_text.delta
data: {"type":"response.reasoning_summary_text.delta","item_id":"rs_...","output_index":0,"summary_index":0,"delta":"正在整理常见植物。","sequence_number":4}
event: response.output_text.delta
data: {"type":"response.output_text.delta","item_id":"msg_...","output_index":1,"content_index":0,"delta":"常见的十字花科植物包括白菜、甘蓝和油菜。","sequence_number":5}
event: response.completed
data: {"type":"response.completed","response":{"id":"resp_...","status":"completed","usage":{"input_tokens":18,"output_tokens":24,"total_tokens":42}},"sequence_number":6}
data: [DONE]
解析说明:
- 按
item_id、output_index和content_index聚合response.output_text.delta,按item_id、output_index和summary_index聚合response.reasoning_summary_text.delta。 - 函数参数按
item_id和output_index聚合response.function_call_arguments.delta;对应的response.function_call_arguments.done只表示该函数参数已完整。 response.completed中的response.status决定响应是否完整,response.usage提供本次请求的用量;[DONE]只表示传输结束。
event: message_start
data: {"type":"message_start","message":{"id":"msg_...","type":"message","role":"assistant","content":[],"stop_reason":null,"usage":{"input_tokens":18,"output_tokens":0}}}
event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"常见的十字花科植物包括白菜、甘蓝和油菜。"}}
event: content_block_stop
data: {"type":"content_block_stop","index":0}
event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"end_turn","stop_sequence":null},"usage":{"input_tokens":18,"output_tokens":24}}
event: message_stop
data: {"type":"message_stop"}
解析说明:
- 按内容块
index建立缓冲区,再根据delta.type将text、thinking、signature或partial_json追加到对应字段。 content_block_stop只完成匹配index的内容块。工具内容块应在此时解析已聚合的partial_json,但整条消息仍未完成。- 从
message_delta.delta.stop_reason读取结果原因,并在message_stop到达后结束消费。usage 快照的覆盖规则参见 记录用量。
增量字段非空只表示当前事件带来新片段,不代表响应完整。usage 缺失或为 null 时,将统计标记为未采集,不记为 0。
判断响应结果#
语义终态是 API 对本次生成结果状态的明确结论,用于判断结果是完整、部分、等待工具还是失败。[DONE]、EOF 或连接关闭只是传输结束,不能单独证明生成成功。
| API | 语义信号 | 结果分类 | 客户端动作 |
|---|---|---|---|
| Chat | finish_reason=stop |
完整 | 提交该 choice.index 的正文结果。 |
| Chat | finish_reason=length 或 content_filter |
部分 | 保留已接收内容并标记不完整;前者可在确认业务需要后调整输出或输入长度,后者不得通过自动重试规避拦截。 |
| Chat | finish_reason=tool_calls |
工具中间态 | 将该候选结果下各工具参数缓冲区标记为完整,解析参数并进入工具调用流程;不得当作最终文本回答。 |
| Chat | data: [DONE] |
仅传输结束 | 以此前各候选结果的 finish_reason 判定状态;缺少语义终态时按提前中断处理。 |
| Responses | response.completed 且 response.status=completed |
完整 | 提交已聚合的输出项。 |
| Responses | response.incomplete |
部分 | 读取 response.incomplete_details,保留部分结果,再决定是否续写或调整限制。 |
| Responses | response.failed 或 error |
失败 | 读取 response.error 或错误事件中的 code、message;仅在原因可重试且业务允许重复生成时重新请求。 |
| Responses | data: [DONE] |
仅传输结束 | 以此前的 response.completed、response.incomplete、response.failed 或 error 判定状态。 |
| Messages | stop_reason=end_turn 或 stop_reason=stop_sequence,随后收到 message_stop |
完整 | 提交已完成的内容块;命中停止序列时同时记录 stop_sequence。 |
| Messages | stop_reason=max_tokens,随后收到 message_stop |
部分 | 保留已接收内容并标记不完整;确认业务需要后调整 max_tokens 或输入长度。 |
| Messages | stop_reason=tool_use,随后收到 message_stop |
工具中间态 | 执行已完成的工具调用,并按 Messages 工具调用流程续接;不得当作最终文本回答。 |
| Messages | error 事件或 HTTP 请求失败 |
失败 | 保留错误信息并关闭连接;仅在原因可重试且业务允许重复生成时重新请求。 |
| Messages | 仅收到 message_stop,没有可识别的 stop_reason |
未确认 | 不将结果标记为完整,记录原始事件并按异常结果处理。 |
记录用量#
- 完整用量:将请求体字段
stream_options.include_usage设为true。服务端通常在data: [DONE]前增加一个choices=[]的数据块,其usage表示整次请求的 token 用量。 - 逐块累计用量:将
stream_options.chunk_include_usage设为true。每个数据块的usage表示截至当前时刻的累计值;客户端只保留最新值,不得逐块相加。 - 模型例外:模型
deepseek-v4-1-flash-economy-260910不受stream_options.include_usage控制。该模型默认在带finish_reason的数据块中返回usage,不额外返回独立的用量数据块。
在 response.completed、response.incomplete 或 response.failed 终态事件的 response.usage 中读取当前请求的用量。终态前断连且没有返回该字段时,将统计标记为未采集,不记为 0。
Messages 的 usage 是累计快照,不是需要求和的增量:
- 收到
message_start时,用message_start.message.usage初始化当前快照。该快照包含输入和缓存相关用量;生成刚开始时,output_tokens通常为0。 - 收到带
usage的message_delta时,用事件中的同名字段覆盖当前快照。不得将input_tokens、output_tokens或缓存 token 与上一快照相加。 - 收到
message_stop后,将最新快照标记为本次请求的最终统计。 - 如果在
message_stop前断连,只能保留最后一次快照并标记为“未最终确认”;没有收到 usage 时标记为“未采集”,不得记为0。
处理连接中断#
在正常终态前出现 EOF、超时或网络断开时,已接收内容只是部分结果。客户端应标记部分结果并释放连接;仅在具备幂等策略且业务允许重复生成时重新请求。本文不定义断点续传。
Messages 以 message_stop 收尾,不使用 [DONE] 代替消息终态;结果分类仍由此前 message_delta.delta.stop_reason 决定。
工作原理#
- 建立连接:客户端发送包含
stream=true的请求,并为每个并发请求维护独立的 SSE 连接。 - 聚合增量:客户端按所选 API 的事件类型分派数据,再使用该事件声明的索引将片段追加到对应输出项。
- 判断完成:客户端使用所选 API 的语义终态提交结果;若在终态前断连,则将已接收内容标记为部分结果并释放连接。
使用说明与限制#
运行观测#
| 观测维度 | 预期影响 | 指标或信号 | 阈值或单位 |
|---|---|---|---|
| 首段内容等待时间 | 客户端可在完整响应结束前开始处理内容;这不代表总生成时间缩短。 | 从发出请求到收到首个正文或思考 delta 的时间,并以毫秒记录。 | 毫秒;本文不提供固定阈值。 |
| token 用量 | 三类 API 的统计位置不同,中断时统计可能缺失。 | Chat 的有效 usage 数据块;Responses 终态事件的 response.usage;Messages 的 message_start.message.usage 或 message_delta.usage。 |
token;缺失不记为零,累计快照不重复相加。 |
| 连接资源 | 流对象未关闭会持续占用客户端连接资源。 | 活跃连接数、客户端超时、消费循环是否退出、流对象是否关闭。 | 连接数、秒;按业务容量设置告警。 |
限制与不支持项#
| 维度 | 具体值 | 后果 | 处理方式 |
|---|---|---|---|
| 参数类型 | stream 不是 boolean |
请求参数类型不符合 Chat API、Responses API 或 Messages API 定义。 | 仅传 true 或 false。 |
| 能力组合 | Responses API 创建前缀缓存时 stream=true |
无法在创建前缀缓存的同一次请求中使用流式输出。 | 先以非流式请求创建前缀缓存,再通过 previous_response_id 续接后续请求。 |
| 连接释放 | 客户端停止消费但未关闭流对象 | 连接和套接字资源可能继续占用。 | 正常和异常路径都关闭流对象;设置有界的连接与读取超时。 |
相关文档#
- Chat API:核对数据块结构、
delta、finish_reason和stream_options。 - Responses API:核对类型化事件、响应状态和
response.usage。 - Messages API:核对消息与内容块事件、
stop_reason和公开用量字段。 - Function Calling:了解工具定义、参数回传和多轮工具执行流程。
- 上下文缓存:了解流式输出与隐式缓存、显式缓存及前缀缓存的组合限制。
