# Reusable AI tools

Use the tools API for product features such as a lyrics generator, image-to-prompt analysis, background removal, or style transfer. A frontend integrates the stable tool contract; Star Task owns the private implementation, validation, charging, and output normalization.

## Discover tools at runtime

Request `GET https://task.starhk.uk/v1/tools?site_id=<SITE_ID>`. Catalog and individual tool-detail reads are public, but `site_id` is required because availability and pricing are site-scoped. Quote, upload, create, and poll requests require the current user's `Authorization: Bearer <ACCESS_TOKEN>`. Render only returned tools. Each item includes:

- `id`, `display_name`, and `category` for catalog display;
- `execution_type`: `text-generation`, `image-understanding`, or `image-generation`;
- `cost_points`, the effective site price;
- `ui_schema`, an ordered list of frontend fields;
- `constraints`, including accepted image count, sizes, or text limits;
- `version`, which changes when an administrator updates the tool.

Do not hard-code the backend model, provider, system prompt, or price. A site may hide any returned tool from its interface, but hiding is presentation logic and does not alter Star Task's server-side availability.

## Build the interface

Generate form controls from `ui_schema`. Supported field types are currently:

- `text`: use `label`, `placeholder`, `maxLength`, and `multiline`;
- `select`: use `label`, `options`, and `default`. When `allowCustom` is true, render an editable combobox (for example a labeled input with a datalist) rather than a closed enum; preserve keyboard access and allow values outside `options`.

Fields may include `group: "basic" | "advanced"`, `description`, and `required`. Show basic fields immediately and place advanced fields in a collapsed, keyboard-accessible disclosure. Mark required fields from `ui_schema[].required`; do not infer required state from visual order. Older catalogs may omit grouping, in which case render fields in the returned order.

Use `execution_type` and `constraints` for media input:

- `text-generation`: normally sends no `image_urls`;
- `image-understanding`: requires the configured number of image URLs;
- `image-generation`: may require reference images and may expose `sizes`.

When media input is enabled, build the file picker from `constraints.accepted_image_types`,
enforce `constraints.max_image_bytes` before uploading, and enforce the combined image count
from `min_ref_images` / `max_ref_images`. Show a local preview and upload progress. Upload each
selected file through `POST /v1/upload`; do not send base64 data in a tool task or expose a local
browser object URL to Star Task.

Uploaded input images are temporary assets. Star Task stores new image uploads under the
`uploads/images/` R2 prefix, which production config expires after two hours. Never present an
uploaded input URL as permanent user storage. Generated task results use a separate prefix and
are not covered by this temporary-input lifecycle rule.

Refresh the catalog after a tool version changes. Do not attempt to interpret or display the private prompt template.

## Quote and create

Quote with `POST /v1/tools/:toolId/quote?site_id=<SITE_ID>` using the same body shape as creation. Create with `POST /v1/tools/:toolId/tasks?site_id=<SITE_ID>` and a user bearer token:

```js
const body = {
  params: {
    theme: '在香港重逢',
    keywords: '渡轮、晚风、霓虹',
    style: 'Cantopop',
    mood: '怀旧、克制',
    language: '繁体中文',
    duration: '中等（约3分钟）',
    structure: '主歌-预副歌-副歌-主歌-预副歌-副歌-桥段-副歌',
    rhyme: '自然押韵',
  },
  image_urls: [],
  idempotency_key: crypto.randomUUID(),
};

const created = await fetch(
  'https://task.starhk.uk/v1/tools/lyrics-generator/tasks?site_id=site-xx',
  {
    method: 'POST',
    headers: {
      Authorization: `Bearer ${accessToken}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify(body),
  },
).then(response => response.json());
```

For `lyrics-generator`, only `theme` is required. Optional parameters are `keywords`, `style`, `mood`, `language`, `title`, `duration`, `structure`, `rhyme`, `reference_artist`, and `extra`. Style, mood, and language are editable selects; duration, structure, and rhyme are closed selects. Preserve these stable parameter names, but obtain labels, options, defaults, limits, price, and version from the runtime catalog. Never expose a model or provider selector for this tool.

For `image-to-prompt`, upload exactly one JPEG, PNG, or WEBP image through Star Task, then pass the returned reachable URL:

```js
const form = new FormData();
form.append('file', file);
const uploaded = await fetch(
  `https://task.starhk.uk/v1/upload?site_id=${encodeURIComponent(siteId)}`,
  {
  method: 'POST',
  headers: { Authorization: `Bearer ${accessToken}` },
  body: form,
  },
).then(response => response.json());
```

```js
{
  params: {
    target_platform: '通用生图模型',
    output_language: '中文',
    detail_level: '专业详细',
    include_negative_prompt: '是',
    focus: '构图、光线和镜头语言',
  },
  image_urls: [uploaded.url],
  idempotency_key: crypto.randomUUID()
}
```

Obtain option labels and defaults from the live `ui_schema`; do not hard-code the example values. The implementation is private and may change without changing the frontend contract. Any execution failure makes the whole task fail under the normal refund policy.

When structured output is valid, `data` can include `prompt`, `negative_prompt`, `summary`, `subject`, `style`, `composition`, `lighting`, `camera`, `color_palette`, and `aspect_ratio`. Always render `text` as a fallback if `data` is absent.

Poll the returned `poll_url` exactly like a normal Star Task task. Successful text and image-understanding tools return a normalized output:

```json
{
  "type": "tool",
  "tool_id": "lyrics-generator",
  "tool_version": 3,
  "execution_type": "text-generation",
  "text": "...",
  "data": {}
}
```

The version above is illustrative. Always use the runtime `tool_version`; never branch on or hard-code a specific catalog version.

`data` appears only when the tool has a structured output schema and the result passes server-side schema validation. Always support `text` as a fallback. Image-generation tools retain the normal media output shape and stable Star Task/R2 URLs.

Star Task normalizes structured output inside the backend. Frontend sites must not send or select implementation-specific output modes and do not need a release when the private implementation changes.

For diagnostics, a text tool result may include `structured_output.mode` and `structured_output.valid`. Treat `data` as authoritative only when present. If `valid` is false, render or otherwise handle `text`; do not parse it into trusted structured data on the frontend.

## General image understanding

The stable tool ID is `image-understanding`. It accepts exactly one JPEG, PNG, or WEBP image. Always read `constraints.max_image_bytes` and `constraints.accepted_image_types` from the runtime catalog instead of copying current limits into frontend policy.

Upload the image with `/v1/upload`, then quote and create with this shape:

```json
{
  "params": {
    "question": "识别图片中的文字并说明界面状态",
    "detail_level": "标准",
    "output_language": "中文"
  },
  "image_urls": ["https://task.starhk.uk/v1/assets/<upload-id>.png"],
  "idempotency_key": "<fresh UUID v4>"
}
```

`question` is optional. Current supported values for `detail_level` and `output_language` come from `ui_schema`; do not hard-code the example options. When structured output validation succeeds, `task.output.data` contains `summary`, `scene`, `objects`, `ocr_text`, `composition`, `colors`, optional `chart_or_ui_data`, and `uncertainties`. Other text-only models should consume this structured result or the `text` fallback, not the original image.

## Web-search chat

The stable frontend-facing tool ID is `web-search-chat`. Create it through
`POST /v1/tools/web-search-chat/tasks?site_id=<SITE_ID>`. Frontend sites only use the Star Task
tool contract and must not select or submit a backend model, provider, or private implementation
identifier.

```json
{
  "params": {
    "query": "香港今天有哪些重要的人工智能新闻？"
  },
  "image_urls": [],
  "idempotency_key": "<fresh UUID v4>"
}
```

`query` is required and limited to 128 characters. Obtain the current limits from the live
`ui_schema`; do not hard-code the example as a frontend-owned contract.

On success, `task.output.data` has this contract:

```ts
type WebSearchResult = {
  schema_version: string;
  provider: string;
  sources: Array<{
    url: string;
    title?: string;
    summary?: string;
    site_name?: string;
    publish_time?: string;
    logo_url?: string;
    content?: string;
    authority?: number;
    rerank_score?: number;
  }>;
};
```

Treat every source URL, title, and summary as untrusted external content: render text escaped,
allow only safe `http:`/`https:` destinations, use `target="_blank"` with
`rel="noopener noreferrer"`, and never inject returned HTML. Always support `task.output.text`
if `data` is absent. The returned `provider` is diagnostic metadata only and must not drive
frontend routing. This tool returns search results and citations; it does not synthesize an answer.

## End-to-end browser example

This example intentionally obtains the current price from quote, uses the upload response's top-level `url`, and appends `site_id` when polling the returned relative URL:

```ts
const TASK_BASE_URL = 'https://task.starhk.uk';

type ToolTask = {
  id: string;
  status: 'pending' | 'running' | 'succeeded' | 'failed' | 'canceled';
  cost_points: number;
  billing_status: 'charged' | 'not_charged' | 'refunded' | 'refund_pending';
  output?: {
    type: 'tool';
    tool_id: string;
    text: string;
    data?: Record<string, unknown>;
  } | null;
  error?: string | null;
};

async function starRequest<T>(path: string, accessToken: string, init: RequestInit = {}): Promise<T> {
  const response = await fetch(`${TASK_BASE_URL}${path}`, {
    ...init,
    headers: {
      Authorization: `Bearer ${accessToken}`,
      ...(init.body instanceof FormData ? {} : { 'Content-Type': 'application/json' }),
      ...init.headers,
    },
  });
  const payload = await response.json();
  if (!response.ok || payload.ok === false) {
    const error = new Error(payload.error?.message || `Star Task HTTP ${response.status}`);
    Object.assign(error, { code: payload.error?.code, status: response.status, details: payload.error?.details });
    throw error;
  }
  return payload as T;
}

async function uploadToolImage(siteId: string, accessToken: string, file: File) {
  const form = new FormData();
  form.append('file', file);
  return starRequest<{ ok: true; url: string; content_type: string; size: number }>(
    `/v1/upload?site_id=${encodeURIComponent(siteId)}`,
    accessToken,
    { method: 'POST', body: form },
  );
}

async function runImageUnderstanding(siteId: string, accessToken: string, file: File) {
  const uploaded = await uploadToolImage(siteId, accessToken, file);
  const input = {
    params: { question: '请提取文字并概括图片', detail_level: '标准', output_language: '中文' },
    image_urls: [uploaded.url],
  };

  const quote = await starRequest<{ ok: true; tool_id: string; cost_points: number }>(
    `/v1/tools/image-understanding/quote?site_id=${encodeURIComponent(siteId)}`,
    accessToken,
    { method: 'POST', body: JSON.stringify(input) },
  );
  // Display quote.cost_points before the user confirms submission.

  const created = await starRequest<{ ok: true; task_id: string; poll_url: string; cost_points: number }>(
    `/v1/tools/image-understanding/tasks?site_id=${encodeURIComponent(siteId)}`,
    accessToken,
    { method: 'POST', body: JSON.stringify({ ...input, idempotency_key: crypto.randomUUID() }) },
  );

  for (;;) {
    const result = await starRequest<{ ok: true; task: ToolTask }>(
      `${created.poll_url}?site_id=${encodeURIComponent(siteId)}`,
      accessToken,
    );
    if (result.task.status === 'succeeded') return result.task.output;
    if (result.task.status === 'failed' || result.task.status === 'canceled') {
      throw new Error(result.task.error || '工具执行失败');
    }
    await new Promise(resolve => setTimeout(resolve, 1500));
  }
}
```

Use the same quote/create/poll helpers for `web-search-chat`, omit upload, send `image_urls: []`, and replace the `params` object with the search parameters above. Stop polling when the page is hidden or the component unmounts, and resume from the saved `poll_url`; do not create a replacement task merely because polling was interrupted.

## Charging and failure behavior

- Quote validates the same tool inputs and returns the current estimate without charging. The create response's `cost_points` is authoritative.
- Tool tasks use the administrator-configured fixed tool price, not the selected text model's token price.
- `image-understanding` and `web-search-chat` currently each cost 0.10 points globally. A site override can change that value, so fetch catalog/quote at runtime and never hard-code `0.10` as authorization or billing logic.
- Creation immediately debits the quoted fixed amount. Reuse an idempotency key only when retrying the same user action; generate a new UUID for a new user action.
- Failed or canceled tasks are refunded by Star Task. Poll until terminal state and use `task.billing_status`: `charged`, `not_charged`, `refunded`, or `refund_pending`. Do not call a points consume/refund endpoint from the frontend.
- Public charges and configured prices use 0.01-point increments. Internal final settlement may retain three decimals. User-facing balances, quotes, charges, and refunds must display exactly two decimals, for example with `Intl.NumberFormat(locale, { minimumFractionDigits: 2, maximumFractionDigits: 2 })`.
- JSON numbers do not preserve trailing zeros: the API can return `0.1`; display it as `0.10`, but send and compare it as a number.
- Backend selection and routing are private implementation details. Handle only the documented Star Task task states and error contract.