#601

Global Rank · of 601 Skills

byted-bytehouse-multimodal-search AI Agent Skill

View Source: bytedance/agentkit-samples

Medium

Installation

npx skills add bytedance/agentkit-samples --skill byted-bytehouse-multimodal-search

12

Installs

ByteHouse 多模态检索 Skill

🚀 快速开始

环境准备

pip install clickhouse-connect volcengine-python-sdk[ark] numpy

环境变量配置

优先从环境变量读取配置，禁止硬编码明文敏感信息：

# ByteHouse 配置
export BYTEHOUSE_HOST="<你的ByteHouse连接地址>"
export BYTEHOUSE_PORT="<ByteHouse端口>"
export BYTEHOUSE_USER="<ByteHouse用户名>"
export BYTEHOUSE_PASSWORD="<ByteHouse密码>"
export BYTEHOUSE_DATABASE="<默认数据库，可选，默认default>"
export BYTEHOUSE_SECURE="<是否启用加密，可选，默认true>"

# 火山引擎方舟 API 配置
export ARK_API_KEY="<火山引擎方舟API密钥>"
export ARK_BASE_URL="https://ark.cn-beijing.volces.com/api/v3"
export EMBEDDING_MODEL="doubao-embedding-vision-251215"
export EMBEDDING_DIMENSIONS="1536"  # 可选，默认1536

如果环境变量未配置，会自动提示用户输入。

📚 核心能力

1. 多模态向量化

基于豆包多模态向量化模型 doubao-embedding-vision-251215：

输入类型 支持格式 最大限制
文本 纯文本字符串 无长度限制
图片 JPG/PNG/GIF/WEBP/BMP <10MB，宽高>14px
视频 MP4/AVI/MOV <50MB

关键约束：

  • 多模态向量化必须调用 /embeddings/multimodal 接口
  • 图片/视频输入格式：{"type": "image_url", "image_url": {"url": "xxx"}}
  • 部分模型不支持 dimensions 参数

2. 向量检索功能

功能 方法 说明
纯向量检索 vector_search() 基于向量相似度检索
混合检索 hybrid_search() 向量+全文检索融合
以文搜图 text_search_image() 文本搜索图片
以图搜图 image_search_image() 图片搜索相似图片
以文搜视频 text_search_video() 文本搜索视频

📖 代码实现

完整示例代码实现位于 scripts/ 目录：

快速使用

from scripts import ByteHouseMultimodalSearch

# 初始化客户端
search = ByteHouseMultimodalSearch(connection_type="http")

# 创建表
search.create_multimodal_table("my_index")

# 插入文档
search.insert_document("my_index", doc_id=1, content_type="text", 
                      content="ByteHouse 多模态检索", title="介绍")

# 向量检索
results = search.vector_search("my_index", query_embedding=embedding, top_k=10)

⚙️ 最佳实践

索引选择

数据规模 索引类型 适用场景
<100万 HNSW 中小规模，低延迟
100万-1亿 HNSW_SQ 大规模，平衡性能成本
>1亿 IVF_PQ_FS 超大规模

性能优化

SETTINGS 
    index_granularity = 1024,
    index_granularity_bytes = 0,
    enable_vector_index_preload = 1

指令优化

场景 Query 侧指令
通用文搜图 Target_modality: image. Instruction:根据文本描述找到对应的图片.
电商商品检索 Target_modality: image. Instruction:找到和描述匹配的同款商品图片.
原图检索 Target_modality: image. Instruction:查找和本图完全相同的图片.

❓ 常见问题

Q1: 向量维度怎么选？

  • 推荐 1536 维作为通用值
  • 维度越高精度越高，但成本也越高

Q2: 如何处理低召回问题？

  1. 增大 hnsw_ef_s 参数

Q3: API 调用失败排查

  • 404: 检查路径是否为 /embeddings/multimodal
  • 400: 检查输入格式，部分模型不支持 dimensions
  • 401: 检查 ARK_API_KEY 是否正确
  • 429: 降低请求频率

🔗 参考文档

Installs

Installs 12
Global Rank #601 of 601

Security Audit

ath Medium
socket Safe
Alerts: 0 Score: 90
snyk Medium
EU EU-Hosted Inference API

Power your AI Agents with the best open-source models.

Drop-in OpenAI-compatible API. No data leaves Europe.

Explore Inference API

GLM

GLM 5

$1.00 / $3.20

per M tokens

Kimi

Kimi K2.5

$0.60 / $2.80

per M tokens

MiniMax

MiniMax M2.5

$0.30 / $1.20

per M tokens

Qwen

Qwen3.5 122B

$0.40 / $3.00

per M tokens

How to use this skill

1

Install byted-bytehouse-multimodal-search by running npx skills add bytedance/agentkit-samples --skill byted-bytehouse-multimodal-search in your project directory. Run the install command above in your project directory. The skill file will be downloaded from GitHub and placed in your project.

2

No configuration needed. Your AI agent (Claude Code, Cursor, Windsurf, etc.) automatically detects installed skills and uses them as context when generating code.

3

The skill enhances your agent's understanding of byted-bytehouse-multimodal-search, helping it follow established patterns, avoid common mistakes, and produce production-ready output.

What you get

Skills are plain-text instruction files — not executable code. They encode expert knowledge about frameworks, languages, or tools that your AI agent reads to improve its output. This means zero runtime overhead, no dependency conflicts, and full transparency: you can read and review every instruction before installing.

Compatibility

This skill works with any AI coding agent that supports the skills.sh format, including Claude Code (Anthropic), Cursor, Windsurf, Cline, Aider, and other tools that read project-level context files. Skills are framework-agnostic at the transport level — the content inside determines which language or framework it applies to.

Data sourced from the skills.sh registry and GitHub. Install counts and security audits are updated regularly.

EU Made in Europe

Chat with 100+ AI Models in one App.

Use Claude, ChatGPT, Gemini alongside with EU-Hosted Models like Deepseek, GLM-5, Kimi K2.5 and many more.

Start for free View pricing
Customer Support