Image Process（图像处理工具）

Image Process（图像处理）工具支持通过 Responses API 调用对输入图片执行画点、画线、旋转、缩放、框选或裁剪关键区域等基础操作，适用于需要模型通过视觉处理提升图片理解的场景（例如图文内容分析、物体定位标注、多轮视觉推理等）。工具通过模型自动判断图像处理逻辑，支持与自定义函数混合使用，且可处理多轮视觉输入（即上一轮输出的图片作为下一轮的输入）。

核心功能#

丰富的图像处理工具：支持启用或禁用画点（Point）、框选（Grounding）、缩放（Zoom）、旋转（Rotate）等子功能，满足不同视觉处理需求。
支持多轮图像处理：复杂视觉任务（如多步缩放+旋转）支持多轮工具调用，上一轮输出图片自动作为下一轮输入（例：image0→image1→image2）。
支持混合调用：可与用户自定义函数混合使用。（暂不支持与 Web Search 联网内容插件混合调用）
兼容多种图片格式：支持 Base64 编码的 .gif、.jpg、.jpeg 等主流图片格式，但对图片规格有明确限制，详见注意事项

功能演示#

以下示例代码演示了图像处理工具的各项功能。实际调用时，请将 <ARK_API_KEY> 替换为您的实际 API Key。

示例一：缩放（Zoom）工具#

以下代码展示了在视觉问答场景下，如何启用缩放（zoom）工具，让模型在放大图片后读取前方路牌文字，并以流式方式实时返回答案。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41curl --location 'https://ark.cn-beijing.volces.com/api/v3/responses' \
--header 'Authorization: Bearer <ARK_API_KEY>' \
--header 'Content-Type: application/json' \
--header 'ark-beta-image-process: true' \
--data '{
    "model": "doubao-seed-2-0-lite-260215",
    "stream": true,
    "tools": [
        {
            "type": "image_process",
            "point": {
                "type": "disabled"
            },
            "grounding": {
                "type": "disabled"
            },
            "zoom": {
                "type": "enabled"
            },
            "rotate": {
                "type": "disabled"
            }
        }
    ],
    "input": [
        {
            "type": "message",
            "role": "user",
            "content": [
                {
                    "type": "input_image",
                    "image_url": "https://arkdocs.tos-cn-beijing.volces.com/images/advanced-usage/image_process_1.jpg"
                },
                {
                    "type": "input_text",
                    "text": "前方路牌写了什么？"
                }
            ]
        }
    ]
}'

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60from arkruntime import Ark
import os

# 从环境变量获取 API 密钥，配置方法见：https://ark.volcengine.com/region:cn-beijing/docs/quick-start

api_key = os.getenv('ARK_API_KEY')

# 初始化客户端，配置 API 地址与工具启用头

client = Ark(
    base_url='https://ark.cn-beijing.volces.com/api/v3',
    api_key=api_key,
)

# 发起图像处理请求

response = client.responses.create(
    model="doubao-seed-2-0-lite-260215",
    tools=[
        {
            "type": "image_process",
            "point": {
                "type": "disabled"
            },
            "grounding": {
                "type": "disabled"
            },
            "zoom": {
                "type": "enabled"
            },
            "rotate": {
                "type": "disabled"
            }
        }
    ],
    input=[
        {
            "type": "message",
            "role": "user",
            "content": [
                {
                    "type": "input_image",
                    "image_url": "https://arkdocs.tos-cn-beijing.volces.com/images/advanced-usage/image_process_1.jpg"  # 输入图片 URL
                },
                {
                    "type": "input_text",
                    "text": "前方路牌写了什么？"  # 系统提示文本
                }
            ]
        }
    ],
    extra_headers={"ark-beta-image-process": "true"},
    stream=True  # 启用流式响应，实时获取处理结果
)

# 打印流式响应结果

for chunk in response:
    if hasattr(chunk, 'delta'):
        print(chunk.delta, end="", flush=True)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56from openai import OpenAI
import os

# 从环境变量获取API密钥，配置方法见：https://ark.volcengine.com/region:cn-beijing/docs/quick-start
api_key = os.getenv('ARK_API_KEY')

# 初始化客户端，配置API地址与工具启用头
client = OpenAI(
    base_url='https://ark.cn-beijing.volces.com/api/v3',
    api_key=api_key,
    default_headers={"ark-beta-image-process": "true"}
)

# 发起图像处理请求
response = client.responses.create(
    model="doubao-seed-2-0-lite-260215",
    tools=[
        {
            "type": "image_process",
            "point": {
                "type": "disabled"
            },
            "grounding": {
                "type": "disabled"
            },
            "zoom": {
                "type": "enabled" # 启用缩放（zoom）工具
            },
            "rotate": {
                "type": "disabled"
            }
        }
    ],
    input=[
        {
            "type": "message",
            "role": "user",
            "content": [
                {
                    "type": "input_image",
                    "image_url": "https://arkdocs.tos-cn-beijing.volces.com/images/advanced-usage/image_process_1.jpg"  # 输入图片 URL
                },
                {
                    "type": "input_text",
                    "text": "前方路牌写了什么？"  # 系统提示文本
                }
            ]
        }
    ],
    stream=True  # 启用流式响应，实时获取处理结果
)

# 打印流式响应结果
for chunk in response:
    if hasattr(chunk, 'delta'):
        print(chunk.delta, end="", flush=True)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59package main
import (
    "context"
    "fmt"
    "github.com/openai/openai-go/v3"
    "github.com/openai/openai-go/v3/option"
    "github.com/openai/openai-go/v3/responses"
)
func main() {
    client := openai.NewClient(option.WithBaseURL("https://ark.cn-beijing.volces.com/api/v3"))
    ctx := context.Background()
    tools := [] map[string] any {
        {
            "type": "image_process",
            "point": {
                "type": "disabled"
            },
            "grounding": {
                "type": "disabled"
            },
            "zoom": {
                "type": "enabled" // 启用缩放（zoom）工具
            },
            "rotate": {
                "type": "disabled"
            }
        }
    }
    stream := client.Responses.NewStreaming(ctx, responses.ResponseNewParams {
        Input: responses.ResponseNewParamsInputUnion {
            OfInputItemList: [] responses.ResponseInputItemUnionParam {
                {
                    OfInputMessage: & responses.ResponseInputItemMessageParam {
                        Role: string(responses.ResponseInputMessageItemRoleUser),
                        Content: [] responses.ResponseInputContentUnionParam {
                            {
                                OfInputImage: & responses.ResponseInputImageParam {
                                    ImageURL: openai.String("https://arkdocs.tos-cn-beijing.volces.com/images/advanced-usage/image_process_1.jpg"),
                                },
                            }, {
                                OfInputText: & responses.ResponseInputTextParam {
                                    Text: "前方路牌写了什么？",
                                },
                            },
                        },
                    },
                },
            }
        },
        Model: "doubao-seed-2-0-lite-260215",
    }, option.WithJSONSet("tools", tools), option.WithHeaderAdd("ark-beta-image-process", "true"))
    for stream.Next() {
        data := stream.Current()
        fmt.Println(data.RawJSON())
    }
    if stream.Err() != nil {
        panic(stream.Err())
    }
}

示例二：画点（Point）工具#

以下代码展示了如何启用画点（point）工具，让模型在视觉问答中进行计数，并以流式方式返回答案。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41curl --location 'https://ark.cn-beijing.volces.com/api/v3/responses' \
--header 'Authorization: Bearer <ARK_API_KEY>' \
--header 'Content-Type: application/json' \
--header 'ark-beta-image-process: true' \
--data '{
    "model": "doubao-seed-2-0-lite-260215",
    "stream": true,
    "tools": [
        {
            "type": "image_process",
            "point": {
                "type": "enabled"
            },
            "grounding": {
                "type": "disabled"
            },
            "zoom": {
                "type": "disabled"
            },
            "rotate": {
                "type": "disabled"
            }
        }
    ],
    "input": [
        {
            "type": "message",
            "role": "user",
            "content": [
                {
                    "type": "input_image",
                    "image_url": "https://arkdocs.tos-cn-beijing.volces.com/images/advanced-usage/image_process_2.jpg"
                },
                {
                    "type": "input_text",
                    "text": "数一数有多少颗草莓？"
                }
            ]
        }
    ]
}'

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60from arkruntime import Ark
import os

# 从环境变量获取 API 密钥，配置方法见：https://ark.volcengine.com/region:cn-beijing/docs/quick-start

api_key = os.getenv('ARK_API_KEY')

# 初始化客户端，配置 API 地址与工具启用头

client = Ark(
    base_url='https://ark.cn-beijing.volces.com/api/v3',
    api_key=api_key,
)

# 发起图像处理请求

response = client.responses.create(
    model="doubao-seed-2-0-lite-260215",
    tools=[
        {
            "type": "image_process",
            "point": {
                "type": "enabled" # 启用画点（point）工具
            },
            "grounding": {
                "type": "disabled"
            },
            "zoom": {
                "type": "disabled"
            },
            "rotate": {
                "type": "disabled"
            }
        }
    ],
    input=[
        {
            "type": "message",
            "role": "user",
            "content": [
                {
                    "type": "input_image",
                    "image_url": "https://arkdocs.tos-cn-beijing.volces.com/images/tool-calling/image_process_2.jpg"  # 输入图片 URL
                },
                {
                    "type": "input_text",
                    "text": "数一数有多少颗草莓？"  # 系统提示文本
                }
            ]
        }
    ],
    extra_headers={"ark-beta-image-process": "true"},
    stream=True  # 启用流式响应，实时获取处理结果
)

# 打印流式响应结果

for chunk in response:
    if hasattr(chunk, 'delta'):
        print(chunk.delta, end="", flush=True)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56from openai import OpenAI
import os

# 从环境变量获取API密钥，配置方法见：https://ark.volcengine.com/region:cn-beijing/docs/quick-start
api_key = os.getenv('ARK_API_KEY')

# 初始化客户端，配置API地址与工具启用头
client = OpenAI(
    base_url='https://ark.cn-beijing.volces.com/api/v3',
    api_key=api_key,
    default_headers={"ark-beta-image-process": "true"}
)

# 发起图像处理请求
response = client.responses.create(
    model="doubao-seed-2-0-lite-260215",
    tools=[
        {
            "type": "image_process",
            "point": {
                "type": "enabled" # 启用画点（point）工具
            },
            "grounding": {
                "type": "disabled"
            },
            "zoom": {
                "type": "disabled"
            },
            "rotate": {
                "type": "disabled"
            }
        }
    ],
    input=[
        {
            "type": "message",
            "role": "user",
            "content": [
                {
                    "type": "input_image",
                    "image_url": "https://arkdocs.tos-cn-beijing.volces.com/images/advanced-usage/image_process_2.jpg"  # 输入图片 URL
                },
                {
                    "type": "input_text",
                    "text": "数一数有多少颗草莓？"  # 系统提示文本
                }
            ]
        }
    ],
    stream=True  # 启用流式响应，实时获取处理结果
)

# 打印流式响应结果
for chunk in response:
    if hasattr(chunk, 'delta'):
        print(chunk.delta, end="", flush=True)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59package main
import (
    "context"
    "fmt"
    "github.com/openai/openai-go/v3"
    "github.com/openai/openai-go/v3/option"
    "github.com/openai/openai-go/v3/responses"
)
func main() {
    client := openai.NewClient(option.WithBaseURL("https://ark.cn-beijing.volces.com/api/v3"))
    ctx := context.Background()
    tools := [] map[string] any {
        {
            "type": "image_process",
            "point": {
                "type": "enabled"  // 启用画点（point）工具
            },
            "grounding": {
                "type": "disabled"
            },
            "zoom": {
                "type": "disabled"
            },
            "rotate": {
                "type": "disabled"
            }
        }
    }
    stream := client.Responses.NewStreaming(ctx, responses.ResponseNewParams {
        Input: responses.ResponseNewParamsInputUnion {
            OfInputItemList: [] responses.ResponseInputItemUnionParam {
                {
                    OfInputMessage: & responses.ResponseInputItemMessageParam {
                        Role: string(responses.ResponseInputMessageItemRoleUser),
                        Content: [] responses.ResponseInputContentUnionParam {
                            {
                                OfInputImage: & responses.ResponseInputImageParam {
                                    ImageURL: openai.String("https://arkdocs.tos-cn-beijing.volces.com/images/advanced-usage/image_process_2.jpg"),
                                },
                            }, {
                                OfInputText: & responses.ResponseInputTextParam {
                                    Text: "数一数有多少颗草莓？",
                                },
                            },
                        },
                    },
                },
            }
        },
        Model: "doubao-seed-2-0-lite-260215",
    }, option.WithJSONSet("tools", tools), option.WithHeaderAdd("ark-beta-image-process", "true"))
    for stream.Next() {
        data := stream.Current()
        fmt.Println(data.RawJSON())
    }
    if stream.Err() != nil {
        panic(stream.Err())
    }
}

示例三：框选（Grounding）工具#

以下代码展示了如何启用框选（grounding）工具，让模型框选出图中的特定目标，并以流式方式返回答案。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41curl --location 'https://ark.cn-beijing.volces.com/api/v3/responses' \
--header 'Authorization: Bearer <ARK_API_KEY>' \
--header 'Content-Type: application/json' \
--header 'ark-beta-image-process: true' \
--data '{
    "model": "doubao-seed-2-0-lite-260215",
    "stream": true,
    "tools": [
        {
            "type": "image_process",
            "point": {
                "type": "disabled"
            },
            "grounding": {
                "type": "enabled"
            },
            "zoom": {
                "type": "disabled"
            },
            "rotate": {
                "type": "disabled"
            }
        }
    ],
    "input": [
        {
            "type": "message",
            "role": "user",
            "content": [
                {
                    "type": "input_image",
                    "image_url": "https://arkdocs.tos-cn-beijing.volces.com/images/advanced-usage/image_process_1.jpg"
                },
                {
                    "type": "input_text",
                    "text": "帮我框选出图中的路牌。"
                }
            ]
        }
    ]
}'

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60from arkruntime import Ark
import os

# 从环境变量获取 API 密钥，配置方法见：https://ark.volcengine.com/region:cn-beijing/docs/quick-start

api_key = os.getenv('ARK_API_KEY')

# 初始化客户端，配置 API 地址与工具启用头

client = Ark(
    base_url='https://ark.cn-beijing.volces.com/api/v3',
    api_key=api_key,
)

# 发起图像处理请求

response = client.responses.create(
    model="doubao-seed-2-0-lite-260215",
    tools=[
        {
            "type": "image_process",
            "point": {
                "type": "disabled"
            },
            "grounding": {
                "type": "enabled" # 启用框选（grounding）工具
            },
            "zoom": {
                "type": "disabled"
            },
            "rotate": {
                "type": "disabled"
            }
        }
    ],
    input=[
        {
            "type": "message",
            "role": "user",
            "content": [
                {
                    "type": "input_image",
                    "image_url": "https://arkdocs.tos-cn-beijing.volces.com/images/tool-calling/image_process_1.jpg"  # 输入图片 URL
                },
                {
                    "type": "input_text",
                    "text": "帮我框选出图中的路牌。"  # 系统提示文本
                }
            ]
        }
    ],
    extra_headers={"ark-beta-image-process": "true"},
    stream=True  # 启用流式响应，实时获取处理结果
)

# 打印流式响应结果

for chunk in response:
    if hasattr(chunk, 'delta'):
        print(chunk.delta, end="", flush=True)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56from openai import OpenAI
import os

# 从环境变量获取API密钥，配置方法见：https://ark.volcengine.com/region:cn-beijing/docs/quick-start
api_key = os.getenv('ARK_API_KEY')

# 初始化客户端，配置API地址与工具启用头
client = OpenAI(
    base_url='https://ark.cn-beijing.volces.com/api/v3',
    api_key=api_key,
    default_headers={"ark-beta-image-process": "true"}
)

# 发起图像处理请求
response = client.responses.create(
    model="doubao-seed-2-0-lite-260215",
    tools=[
        {
            "type": "image_process",
            "point": {
                "type": "disabled"
            },
            "grounding": {
                "type": "enabled" # 启用框选（grounding）工具
            },
            "zoom": {
                "type": "disabled"
            },
            "rotate": {
                "type": "disabled"
            }
        }
    ],
    input=[
        {
            "type": "message",
            "role": "user",
            "content": [
                {
                    "type": "input_image",
                    "image_url": "https://arkdocs.tos-cn-beijing.volces.com/images/advanced-usage/image_process_1.jpg"  # 输入图片 URL
                },
                {
                    "type": "input_text",
                    "text": "帮我框选出图中的路牌。"  # 系统提示文本
                }
            ]
        }
    ],
    stream=True  # 启用流式响应，实时获取处理结果
)

# 打印流式响应结果
for chunk in response:
    if hasattr(chunk, 'delta'):
        print(chunk.delta, end="", flush=True)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59package main
import (
    "context"
    "fmt"
    "github.com/openai/openai-go/v3"
    "github.com/openai/openai-go/v3/option"
    "github.com/openai/openai-go/v3/responses"
)
func main() {
    client := openai.NewClient(option.WithBaseURL("https://ark.cn-beijing.volces.com/api/v3"))
    ctx := context.Background()
    tools := []map[string]any{
        {
            "type": "image_process",
            "point": map[string]string{
                "type": "disabled",
            },
            "grounding": map[string]string{
                "type": "enabled", // 启用框选（grounding）工具
            },
            "zoom": map[string]string{
                "type": "disabled",
            },
            "rotate": map[string]string{
                "type": "disabled",
            },
        },
    }
    stream := client.Responses.NewStreaming(ctx, responses.ResponseNewParams{
        Input: responses.ResponseNewParamsInputUnion{
            OfInputItemList: []responses.ResponseInputItemUnionParam{
                {
                    OfInputMessage: &responses.ResponseInputItemMessageParam{
                        Role: openai.F(responses.ResponseInputMessageItemRoleUser),
                        Content: openai.F([]responses.ResponseInputContentUnionParam{
                            {
                                OfInputImage: &responses.ResponseInputImageParam{
                                    ImageURL: openai.String("https://arkdocs.tos-cn-beijing.volces.com/images/advanced-usage/image_process_1.jpg"),
                                },
                            }, {
                                OfInputText: &responses.ResponseInputTextParam{
                                    Text: openai.F("帮我框选出图中的路牌。"),
                                },
                            },
                        }),
                    },
                },
            },
        },
        Model: openai.F("doubao-seed-2-0-lite-260215"),
    }, option.WithJSONSet("tools", tools), option.WithHeaderAdd("ark-beta-image-process", "true"))
    for stream.Next() {
        data := stream.Current()
        fmt.Println(data.RawJSON())
    }
    if stream.Err() != nil {
        panic(stream.Err())
    }
}

示例四：旋转（Rotate）工具#

以下代码展示了如何启用旋转（rotate）工具，让模型对图像进行旋转操作，并以流式方式返回答案。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41curl --location 'https://ark.cn-beijing.volces.com/api/v3/responses' \
--header 'Authorization: Bearer <ARK_API_KEY>' \
--header 'Content-Type: application/json' \
--header 'ark-beta-image-process: true' \
--data '{
    "model": "doubao-seed-2-0-lite-260215",
    "stream": true,
    "tools": [
        {
            "type": "image_process",
            "point": {
                "type": "disabled"
            },
            "grounding": {
                "type": "disabled"
            },
            "zoom": {
                "type": "disabled"
            },
            "rotate": {
                "type": "enabled"
            }
        }
    ],
    "input": [
        {
            "type": "message",
            "role": "user",
            "content": [
                {
                    "type": "input_image",
                    "image_url": "https://arkdocs.tos-cn-beijing.volces.com/images/advanced-usage/image_process_1.jpg"
                },
                {
                    "type": "input_text",
                    "text": "帮我把图片顺时针旋转90度。"
                }
            ]
        }
    ]
}'

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60from arkruntime import Ark
import os

# 从环境变量获取 API 密钥，配置方法见：https://ark.volcengine.com/region:cn-beijing/docs/quick-start

api_key = os.getenv('ARK_API_KEY')

# 初始化客户端，配置 API 地址与工具启用头

client = Ark(
    base_url='https://ark.cn-beijing.volces.com/api/v3',
    api_key=api_key,
)

# 发起图像处理请求

response = client.responses.create(
    model="doubao-seed-2-0-lite-260215",
    tools=[
        {
            "type": "image_process",
            "point": {
                "type": "disabled"
            },
            "grounding": {
                "type": "disabled"
            },
            "zoom": {
                "type": "disabled"
            },
            "rotate": {
                "type": "enabled" # 启用旋转（rotate）工具
            }
        }
    ],
    input=[
        {
            "type": "message",
            "role": "user",
            "content": [
                {
                    "type": "input_image",
                    "image_url": "https://arkdocs.tos-cn-beijing.volces.com/images/tool-calling/image_process_1.jpg"  # 输入图片 URL
                },
                {
                    "type": "input_text",
                    "text": "帮我把图片顺时针旋转 90 度。"  # 系统提示文本
                }
            ]
        }
    ],
    extra_headers={"ark-beta-image-process": "true"},
    stream=True  # 启用流式响应，实时获取处理结果
)

# 打印流式响应结果

for chunk in response:
    if hasattr(chunk, 'delta'):
        print(chunk.delta, end="", flush=True)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56from openai import OpenAI
import os

# 从环境变量获取API密钥，配置方法见：https://ark.volcengine.com/region:cn-beijing/docs/quick-start
api_key = os.getenv('ARK_API_KEY')

# 初始化客户端，配置API地址与工具启用头
client = OpenAI(
    base_url='https://ark.cn-beijing.volces.com/api/v3',
    api_key=api_key,
    default_headers={"ark-beta-image-process": "true"}
)

# 发起图像处理请求
response = client.responses.create(
    model="doubao-seed-2-0-lite-260215",
    tools=[
        {
            "type": "image_process",
            "point": {
                "type": "disabled"
            },
            "grounding": {
                "type": "disabled"
            },
            "zoom": {
                "type": "disabled"
            },
            "rotate": {
                "type": "enabled" # 启用旋转（rotate）工具
            }
        }
    ],
    input=[
        {
            "type": "message",
            "role": "user",
            "content": [
                {
                    "type": "input_image",
                    "image_url": "https://arkdocs.tos-cn-beijing.volces.com/images/advanced-usage/image_process_1.jpg"  # 输入图片 URL
                },
                {
                    "type": "input_text",
                    "text": "帮我把图片顺时针旋转90度。"  # 系统提示文本
                }
            ]
        }
    ],
    stream=True  # 启用流式响应，实时获取处理结果
)

# 打印流式响应结果
for chunk in response:
    if hasattr(chunk, 'delta'):
        print(chunk.delta, end="", flush=True)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59package main
import (
    "context"
    "fmt"
    "github.com/openai/openai-go/v3"
    "github.com/openai/openai-go/v3/option"
    "github.com/openai/openai-go/v3/responses"
)
func main() {
    client := openai.NewClient(option.WithBaseURL("https://ark.cn-beijing.volces.com/api/v3"))
    ctx := context.Background()
    tools := []map[string]any{
        {
            "type": "image_process",
            "point": map[string]string{
                "type": "disabled",
            },
            "grounding": map[string]string{
                "type": "disabled",
            },
            "zoom": map[string]string{
                "type": "disabled",
            },
            "rotate": map[string]string{
                "type": "enabled", // 启用旋转（rotate）工具
            },
        },
    }
    stream := client.Responses.NewStreaming(ctx, responses.ResponseNewParams{
        Input: responses.ResponseNewParamsInputUnion{
            OfInputItemList: []responses.ResponseInputItemUnionParam{
                {
                    OfInputMessage: &responses.ResponseInputItemMessageParam{
                        Role: openai.F(responses.ResponseInputMessageItemRoleUser),
                        Content: openai.F([]responses.ResponseInputContentUnionParam{
                            {
                                OfInputImage: &responses.ResponseInputImageParam{
                                    ImageURL: openai.String("https://arkdocs.tos-cn-beijing.volces.com/images/advanced-usage/image_process_1.jpg"),
                                },
                            }, {
                                OfInputText: &responses.ResponseInputTextParam{
                                    Text: openai.F("帮我把图片顺时针旋转 90 度。"),
                                },
                            },
                        }),
                    },
                },
            },
        },
        Model: openai.F("doubao-seed-2-0-lite-260215"),
    }, option.WithJSONSet("tools", tools), option.WithHeaderAdd("ark-beta-image-process", "true"))
    for stream.Next() {
        data := stream.Current()
        fmt.Println(data.RawJSON())
    }
    if stream.Err() != nil {
        panic(stream.Err())
    }
}

支持的模型#

参见工具调用能力。

计费说明#

公测期间：暂时免费使用，无额外收费。
我们将会提前 2 周通过官方渠道告知具体收费标准，保障您的使用权益。

注意事项#

函数命名冲突：若用户自定义函数与 image_process 重名，由模型自行判断调用何种工具（无需额外配置）。
图片规格限制（超出规格将导致处理失败）：
- 文件体积 ≤ 10MB
- 总像素 ≤ 36000000 像素
- 图片宽和高的长度 > 14 像素
- 长宽比 < 150:1
文件格式支持情况：
- 支持 .gif、.jpg、.jpeg、.png、.webp、.bmp、.tiff、.ico、.icns、.jp2 格式。
- 不支持 .dib、.sgi、.heic、.heif 格式。
功能限制：
- 当前不支持与 Web Search（联网内容插件）工具混合使用，也不支持通过 tool_choice 参数指定调用 image_process。
- 暂不支持 caching 参数，使用该参数会返回 400 错误信息。
Tokens 消费提示：多轮图像处理会增加 Tokens 消费（上一轮图片输入会计入下一轮 Tokens），需注意调用成本。