update：更新skill脚本
This commit is contained in:
parent
0deb3cf57b
commit
e1b0967cb3
@ -6,3 +6,6 @@ feishu:
|
||||
hisense_geo:
|
||||
base_url: "https://geo-api.hisense.com"
|
||||
token: "" # 生产环境 x-token，勿提交仓库
|
||||
haoxian:
|
||||
base_url: "https://resource-api.cemeta.cn"
|
||||
token: "" # 好现 x-token，勿提交仓库
|
||||
|
||||
119
haoxian/skills/temp0923/SKILL.md
Normal file
119
haoxian/skills/temp0923/SKILL.md
Normal file
@ -0,0 +1,119 @@
|
||||
---
|
||||
name: hisense-sept-qa-raw-query
|
||||
description: >-
|
||||
Query Hisense September agentic projects for qa_raw and qa_product_raw
|
||||
(HISENSE-HIST). Before any API call, must confirm BOTH batch date and project
|
||||
id together. Returns id, project_name, total, qa_raw_list, qa_product_raw_list.
|
||||
Use when user asks for 9月项目、问答底表、商品卡、qa_raw、qa_product_raw、
|
||||
HISENSE-HIST、temp0923, or 批次日期 / 项目id.
|
||||
---
|
||||
|
||||
# 海信 9 月 qa_raw / qa_product_raw 查询
|
||||
|
||||
从 `https://resource-api.cemeta.cn` 拉 9 月项目，再按批次查问答底表与商品卡表，组装返回 JSON。
|
||||
|
||||
配置：`config/config.dev.yaml` → `haoxian.base_url` / `haoxian.token`（Header `x-token`）。本机可用 `http://localhost:9999` 代替 base（路径相同）。
|
||||
|
||||
## 必选前置输入（缺一不可，须同时确认）
|
||||
|
||||
**使用本 skill 时，执行任何接口调用之前，必须先同时拿到下面 1 和 2。缺任意一项都停下来，两项齐了再继续后续流程。不要猜、不要沿用上次对话默认值、不要只拿到一个就先跑一半。**
|
||||
|
||||
| # | 必选项 | 示例 |
|
||||
|---|--------|------|
|
||||
| 1 | 查询的批次日期 | `20260919`（`YYYYMMDD`） |
|
||||
| 2 | 项目 id | `1096` |
|
||||
|
||||
**一次性原文提示用户（同一条消息里同时确认两项）：**
|
||||
|
||||
> 请同时确认后回复：
|
||||
> 1. 查询的批次日期（示例：20260919）
|
||||
> 2. 项目id（示例：1096）
|
||||
|
||||
用户在同一轮或连续回复中给出 1 和 2 后，再执行下方「执行流程」。若只给了一项，用同样提示补要缺失项，仍不要开查。
|
||||
|
||||
## 执行流程
|
||||
|
||||
### 1. 拉 9 月项目列表
|
||||
|
||||
```
|
||||
GET {base_url}/api/agentic-project/project?page=1&page_size=999&listed=all
|
||||
Header: x-token
|
||||
```
|
||||
|
||||
过滤（必须同时满足）：
|
||||
|
||||
- `listed = 1`
|
||||
- `status = 'running'`
|
||||
- `end_month = '2026-09'`
|
||||
|
||||
取出全部 `id`、`project_name`。用用户输入的项目 id 在列表中解析 `project_name`（找不到也继续查表，name 可为空并注明）。
|
||||
|
||||
### 2. 拼接 crawl_batch_id
|
||||
|
||||
```
|
||||
HISENSE-HIST-{project_id}-{批次日期}
|
||||
```
|
||||
|
||||
例：`HISENSE-HIST-1096-20260919`
|
||||
|
||||
### 3. 查问答底表 qa_raw
|
||||
|
||||
```
|
||||
GET {base_url}/api/geo_label/l1/qa_raw?project_id={id}&crawl_batch_id={crawl_batch_id}&page=1&page_size=50
|
||||
Header: x-token
|
||||
```
|
||||
|
||||
翻页直到收齐 `data.total`。接口列表字段为 `data.items`。
|
||||
|
||||
### 4. 查商品卡表 qa_product_raw
|
||||
|
||||
```
|
||||
GET {base_url}/api/geo_label/l1/qa_product_raw?project_id={id}&crawl_batch_id={crawl_batch_id}&page=1&page_size=50
|
||||
Header: x-token
|
||||
```
|
||||
|
||||
同样翻页收齐；接口列表字段为 `data.items`。
|
||||
|
||||
### 5. 组装返回 JSON
|
||||
|
||||
**对外不要输出 `items`。** `data.items` 分别映射为：
|
||||
|
||||
- qa_raw → `qa_raw_list`
|
||||
- qa_product_raw → `qa_product_raw_list`
|
||||
|
||||
`total` 取 **qa_raw** 的 `data.total`。
|
||||
|
||||
```json
|
||||
{
|
||||
"id": 1096,
|
||||
"project_name": "500L左右冰箱推荐",
|
||||
"total": 227,
|
||||
"code": 0,
|
||||
"ok": true,
|
||||
"crawl_batch_id": "HISENSE-HIST-1096-20260919",
|
||||
"qa_raw_list": [],
|
||||
"qa_product_raw_list": []
|
||||
}
|
||||
```
|
||||
|
||||
必须包含：`id`、`project_name`、`total`、`qa_raw_list`（全量明细）、`qa_product_raw_list`（全量明细）。体积大时落盘并告知路径，例如：
|
||||
|
||||
`haoxian/skills/temp0923/data/qa_raw_{批次}/{id}_response.json`
|
||||
|
||||
## 推荐脚本
|
||||
|
||||
```bash
|
||||
# 可选：刷新 9 月项目缓存
|
||||
python3 haoxian/skills/temp0923/scripts/query_qa_raw.py list-projects
|
||||
|
||||
# 必选参数：--batch、--id
|
||||
python3 haoxian/skills/temp0923/scripts/query_qa_raw.py project --batch 20260919 --id 1096
|
||||
```
|
||||
|
||||
脚本 stdout 即为上述返回结构。本地缓存目录：`haoxian/skills/temp0923/data/`。
|
||||
|
||||
## 注意
|
||||
|
||||
- Token 只读配置，勿写入 skill、勿提交仓库。
|
||||
- `total=0` 或空列表均为有效结果。
|
||||
- 翻页建议 `page_size=50`，失败重试，避免一次过大导致断连。
|
||||
348
haoxian/skills/temp0923/scripts/query_qa_raw.py
Normal file
348
haoxian/skills/temp0923/scripts/query_qa_raw.py
Normal file
@ -0,0 +1,348 @@
|
||||
#!/usr/bin/env python3
|
||||
"""海信 9 月项目 + qa_raw / qa_product_raw 查询（resource-api / localhost）。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import sys
|
||||
import time
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[4] # ai_script
|
||||
SKILL_DATA = Path(__file__).resolve().parents[1] / "data"
|
||||
DEFAULT_BASE = "https://resource-api.cemeta.cn"
|
||||
REQUIRED_END_MONTH = "2026-09"
|
||||
|
||||
|
||||
def load_haoxian_config() -> dict:
|
||||
cfg_path = ROOT / "config" / "config.dev.yaml"
|
||||
if not cfg_path.exists():
|
||||
raise SystemExit(f"缺少配置: {cfg_path}")
|
||||
section = None
|
||||
hao: dict[str, str] = {}
|
||||
for line in cfg_path.read_text().splitlines():
|
||||
if line.startswith("haoxian:"):
|
||||
section = "hao"
|
||||
continue
|
||||
if line and not line.startswith(" ") and line.endswith(":"):
|
||||
section = None
|
||||
continue
|
||||
if section == "hao" and ":" in line and line.startswith(" "):
|
||||
k, v = line.split(":", 1)
|
||||
hao[k.strip()] = v.strip().strip('"').strip("'")
|
||||
if not hao.get("token"):
|
||||
raise SystemExit("config.dev.yaml 缺少 haoxian.token")
|
||||
if not hao.get("base_url"):
|
||||
hao["base_url"] = DEFAULT_BASE
|
||||
return hao
|
||||
|
||||
|
||||
def http_get(url: str, token: str, retries: int = 4) -> dict:
|
||||
last: Exception | None = None
|
||||
for i in range(retries):
|
||||
try:
|
||||
req = urllib.request.Request(url, headers={"x-token": token})
|
||||
with urllib.request.urlopen(req, timeout=180) as resp:
|
||||
return json.loads(resp.read())
|
||||
except Exception as e: # noqa: BLE001
|
||||
last = e
|
||||
time.sleep(1.5 * (i + 1))
|
||||
raise RuntimeError(f"GET failed after retries: {url}: {last}")
|
||||
|
||||
|
||||
def list_sept_projects(base: str, token: str) -> list[dict]:
|
||||
url = f"{base.rstrip('/')}/api/agentic-project/project?page=1&page_size=999&listed=all"
|
||||
data = http_get(url, token)
|
||||
items = (data.get("data") or {}).get("list") or []
|
||||
out = []
|
||||
for p in items:
|
||||
if (
|
||||
p.get("listed") == 1
|
||||
and str(p.get("status") or "").lower() == "running"
|
||||
and p.get("end_month") == REQUIRED_END_MONTH
|
||||
):
|
||||
out.append(
|
||||
{
|
||||
"id": p.get("id"),
|
||||
"project_name": p.get("project_name"),
|
||||
"brand": p.get("brand"),
|
||||
"category": p.get("category"),
|
||||
"end_month": p.get("end_month"),
|
||||
"status": p.get("status"),
|
||||
"listed": p.get("listed"),
|
||||
}
|
||||
)
|
||||
out.sort(key=lambda x: int(x["id"]) if str(x["id"]).isdigit() else 0, reverse=True)
|
||||
return out
|
||||
|
||||
|
||||
def fetch_l1_table(
|
||||
base: str,
|
||||
token: str,
|
||||
table: str,
|
||||
project_id: int | str,
|
||||
batch: str,
|
||||
page_size: int = 50,
|
||||
) -> dict:
|
||||
"""分页拉取 /api/geo_label/l1/{table}，返回 {total, code, ok, crawl_batch_id, items}。"""
|
||||
crawl = f"HISENSE-HIST-{project_id}-{batch}"
|
||||
all_items: list = []
|
||||
total = None
|
||||
code = 0
|
||||
page = 1
|
||||
while True:
|
||||
url = (
|
||||
f"{base.rstrip('/')}/api/geo_label/l1/{table}"
|
||||
f"?project_id={project_id}&crawl_batch_id={crawl}"
|
||||
f"&page={page}&page_size={page_size}"
|
||||
)
|
||||
data = http_get(url, token)
|
||||
code = data.get("code", 0)
|
||||
if code != 0:
|
||||
return {
|
||||
"id": int(project_id) if str(project_id).isdigit() else project_id,
|
||||
"total": 0,
|
||||
"code": code,
|
||||
"ok": False,
|
||||
"crawl_batch_id": crawl,
|
||||
"items": [],
|
||||
"message": data.get("message"),
|
||||
}
|
||||
d = data.get("data") or {}
|
||||
if total is None:
|
||||
total = d.get("total", 0) or 0
|
||||
page_items = d.get("items") or []
|
||||
all_items.extend(page_items)
|
||||
if len(all_items) >= total or not page_items:
|
||||
break
|
||||
page += 1
|
||||
if page > 100:
|
||||
break
|
||||
return {
|
||||
"id": int(project_id) if str(project_id).isdigit() else project_id,
|
||||
"total": total,
|
||||
"code": code,
|
||||
"ok": True,
|
||||
"crawl_batch_id": crawl,
|
||||
"fetched": len(all_items),
|
||||
"items": all_items,
|
||||
}
|
||||
|
||||
|
||||
def fetch_qa_raw(base: str, token: str, project_id: int | str, batch: str, page_size: int = 50) -> dict:
|
||||
return fetch_l1_table(base, token, "qa_raw", project_id, batch, page_size)
|
||||
|
||||
|
||||
def fetch_qa_product_raw(base: str, token: str, project_id: int | str, batch: str, page_size: int = 50) -> dict:
|
||||
return fetch_l1_table(base, token, "qa_product_raw", project_id, batch, page_size)
|
||||
|
||||
|
||||
def cmd_list_projects(args: argparse.Namespace) -> None:
|
||||
cfg = load_haoxian_config()
|
||||
base = args.base or cfg["base_url"]
|
||||
projects = list_sept_projects(base, cfg["token"])
|
||||
SKILL_DATA.mkdir(parents=True, exist_ok=True)
|
||||
out = SKILL_DATA / "sept_projects_202609.json"
|
||||
out.write_text(json.dumps(projects, ensure_ascii=False, indent=2))
|
||||
print(json.dumps({"count": len(projects), "path": str(out), "projects": projects}, ensure_ascii=False, indent=2))
|
||||
|
||||
|
||||
def cmd_batch_totals(args: argparse.Namespace) -> None:
|
||||
cfg = load_haoxian_config()
|
||||
base = args.base or cfg["base_url"]
|
||||
batch = args.batch
|
||||
projects = list_sept_projects(base, cfg["token"])
|
||||
cache_dir = SKILL_DATA / f"qa_raw_{batch}"
|
||||
cache_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
rows = []
|
||||
for i, p in enumerate(projects, 1):
|
||||
pid = p["id"]
|
||||
cache = cache_dir / f"{pid}.json"
|
||||
if cache.exists() and not args.refresh:
|
||||
payload = json.loads(cache.read_text())
|
||||
total = payload.get("total", 0)
|
||||
items = payload.get("items") or []
|
||||
row = {
|
||||
"id": pid,
|
||||
"project_name": p["project_name"],
|
||||
"total": total,
|
||||
"code": 0,
|
||||
"ok": True,
|
||||
"crawl_batch_id": payload.get("crawl_batch_id") or f"HISENSE-HIST-{pid}-{batch}",
|
||||
"fetched": len(items),
|
||||
}
|
||||
else:
|
||||
try:
|
||||
raw = fetch_qa_raw(base, cfg["token"], pid, batch, page_size=args.page_size)
|
||||
except Exception as e: # noqa: BLE001
|
||||
row = {
|
||||
"id": pid,
|
||||
"project_name": p["project_name"],
|
||||
"total": -1,
|
||||
"code": None,
|
||||
"ok": False,
|
||||
"err": str(e),
|
||||
}
|
||||
rows.append(row)
|
||||
print(f"[{i}/{len(projects)}] FAIL id={pid}: {e}", file=sys.stderr)
|
||||
continue
|
||||
items = raw.get("items") or []
|
||||
payload = {
|
||||
"project_id": pid,
|
||||
"project_name": p["project_name"],
|
||||
"crawl_batch_id": raw["crawl_batch_id"],
|
||||
"batch_date": batch,
|
||||
"total": raw["total"],
|
||||
"fetched": raw.get("fetched", len(items)),
|
||||
"items": items,
|
||||
}
|
||||
cache.write_text(json.dumps(payload, ensure_ascii=False))
|
||||
row = {
|
||||
"id": pid,
|
||||
"project_name": p["project_name"],
|
||||
"total": raw["total"],
|
||||
"code": raw.get("code", 0),
|
||||
"ok": raw.get("ok", True),
|
||||
"crawl_batch_id": raw["crawl_batch_id"],
|
||||
"fetched": payload["fetched"],
|
||||
}
|
||||
rows.append(row)
|
||||
if i % 20 == 0 or i == len(projects):
|
||||
print(f"[{i}/{len(projects)}] ...", file=sys.stderr)
|
||||
|
||||
summary = {
|
||||
"batch_date": batch,
|
||||
"base_url": base,
|
||||
"count": len(rows),
|
||||
"ok": sum(1 for r in rows if r.get("ok")),
|
||||
"total_sum": sum(r["total"] for r in rows if r.get("ok") and r["total"] >= 0),
|
||||
"projects": rows,
|
||||
}
|
||||
(SKILL_DATA / f"qa_raw_{batch}_summary.json").write_text(json.dumps(summary, ensure_ascii=False, indent=2))
|
||||
tsv = ["project_id\tproject_name\ttotal\tok\tcrawl_batch_id"]
|
||||
for r in rows:
|
||||
tsv.append(
|
||||
f"{r.get('id')}\t{r.get('project_name')}\t{r.get('total')}\t{r.get('ok')}\t{r.get('crawl_batch_id','')}"
|
||||
)
|
||||
(SKILL_DATA / f"qa_raw_{batch}_totals.tsv").write_text("\n".join(tsv) + "\n")
|
||||
print(json.dumps({"batch_date": batch, "count": len(rows), "projects": rows}, ensure_ascii=False, indent=2))
|
||||
|
||||
|
||||
def cmd_project(args: argparse.Namespace) -> None:
|
||||
cfg = load_haoxian_config()
|
||||
base = args.base or cfg["base_url"]
|
||||
batch = args.batch
|
||||
pid = args.id
|
||||
cache_dir = SKILL_DATA / f"qa_raw_{batch}"
|
||||
cache = cache_dir / f"{pid}.json"
|
||||
product_cache = cache_dir / f"{pid}_product_raw.json"
|
||||
|
||||
name = None
|
||||
projects_path = SKILL_DATA / "sept_projects_202609.json"
|
||||
if projects_path.exists():
|
||||
for p in json.loads(projects_path.read_text()):
|
||||
if str(p.get("id")) == str(pid):
|
||||
name = p.get("project_name")
|
||||
break
|
||||
|
||||
if cache.exists() and not args.refresh:
|
||||
payload = json.loads(cache.read_text())
|
||||
qa_raw_list = payload.get("items") or payload.get("qa_raw_list") or []
|
||||
crawl = payload.get("crawl_batch_id") or f"HISENSE-HIST-{pid}-{batch}"
|
||||
total = payload.get("total")
|
||||
code = 0
|
||||
ok = True
|
||||
if not name:
|
||||
name = payload.get("project_name")
|
||||
else:
|
||||
raw = fetch_qa_raw(base, cfg["token"], pid, batch, page_size=args.page_size)
|
||||
qa_raw_list = raw.get("items") or []
|
||||
crawl = raw["crawl_batch_id"]
|
||||
total = raw["total"]
|
||||
code = raw.get("code", 0)
|
||||
ok = raw.get("ok", True)
|
||||
cache_dir.mkdir(parents=True, exist_ok=True)
|
||||
cache.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"project_id": raw["id"],
|
||||
"project_name": name,
|
||||
"crawl_batch_id": crawl,
|
||||
"batch_date": batch,
|
||||
"total": total,
|
||||
"fetched": len(qa_raw_list),
|
||||
"items": qa_raw_list,
|
||||
},
|
||||
ensure_ascii=False,
|
||||
)
|
||||
)
|
||||
|
||||
if product_cache.exists() and not args.refresh:
|
||||
prod_payload = json.loads(product_cache.read_text())
|
||||
qa_product_raw_list = (
|
||||
prod_payload.get("items") or prod_payload.get("qa_product_raw_list") or []
|
||||
)
|
||||
else:
|
||||
prod = fetch_qa_product_raw(base, cfg["token"], pid, batch, page_size=args.page_size)
|
||||
qa_product_raw_list = prod.get("items") or []
|
||||
if not prod.get("ok", True):
|
||||
ok = False
|
||||
code = prod.get("code", code)
|
||||
cache_dir.mkdir(parents=True, exist_ok=True)
|
||||
product_cache.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"project_id": int(pid) if str(pid).isdigit() else pid,
|
||||
"project_name": name,
|
||||
"crawl_batch_id": crawl,
|
||||
"batch_date": batch,
|
||||
"total": prod.get("total", len(qa_product_raw_list)),
|
||||
"fetched": len(qa_product_raw_list),
|
||||
"items": qa_product_raw_list,
|
||||
},
|
||||
ensure_ascii=False,
|
||||
)
|
||||
)
|
||||
|
||||
out = {
|
||||
"id": int(pid) if str(pid).isdigit() else pid,
|
||||
"project_name": name,
|
||||
"total": total,
|
||||
"code": code,
|
||||
"ok": ok,
|
||||
"crawl_batch_id": crawl,
|
||||
"qa_raw_list": qa_raw_list,
|
||||
"qa_product_raw_list": qa_product_raw_list,
|
||||
}
|
||||
print(json.dumps(out, ensure_ascii=False, indent=2))
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Hisense Sept qa_raw query")
|
||||
parser.add_argument("--base", default=None, help="API base, default from config / resource-api")
|
||||
parser.add_argument("--page-size", type=int, default=50)
|
||||
sub = parser.add_subparsers(dest="cmd", required=True)
|
||||
|
||||
p1 = sub.add_parser("list-projects", help="List Sept running projects")
|
||||
p1.set_defaults(func=cmd_list_projects)
|
||||
|
||||
p2 = sub.add_parser("batch-totals", help="Fetch/totals for all Sept projects in a batch")
|
||||
p2.add_argument("--batch", required=True, help="Batch date YYYYMMDD, e.g. 20260919")
|
||||
p2.add_argument("--refresh", action="store_true")
|
||||
p2.set_defaults(func=cmd_batch_totals)
|
||||
|
||||
p3 = sub.add_parser("project", help="Fetch qa_raw + qa_product_raw; stdout uses qa_raw_list")
|
||||
p3.add_argument("--batch", required=True)
|
||||
p3.add_argument("--id", required=True)
|
||||
p3.add_argument("--refresh", action="store_true")
|
||||
p3.set_defaults(func=cmd_project)
|
||||
|
||||
args = parser.parse_args()
|
||||
args.func(args)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Loading…
x
Reference in New Issue
Block a user