{"skill":{"slug":"authoring-data-cli-source","displayName":"authoring-data-cli-source","summary":"Use when the user needs to add or redesign an agent-data-cli source for RSS, news, social media, finance, APIs, scraping, browser automation, authentication,...","description":"---\nname: authoring-data-cli-source\ndescription: Use when the user needs to add or redesign an agent-data-cli source for RSS, news, social media, finance, APIs, scraping, browser automation, authentication, or remote content sync.\n---\n\n# Authoring agent-data-cli source\n\n## Overview\n\nUse this skill to design and implement an `agent-data-cli` source with stable behavior and clear project fit.\n\nThis skill is deliberately stricter than normal feature work because a weak source design causes protocol drift, command confusion, and unreliable sync behavior.\n\nIt is the source-authoring path for RSS feeds, HTTP APIs, HTML scraping, browser-driven sites, finance data, news content, and other remote content systems that must fit the `source/channel/content` model.\n\nCurrent core contract to keep in mind:\n\n- `source/channel` is still the only core resource model\n- `content update` sources return `ContentSyncBatch`, not flat per-row persistence instructions\n- shared persistence is now `content_nodes`, `content_channel_links`, and `content_relations`\n- structural relations in core use abstract `parent`; source-specific meaning belongs in `relation_semantic`\n\n## Hard Gate\n\nDo not start implementation immediately.\n\nThe required sequence is:\n\n1. research\n2. spec\n3. plan\n4. approval\n5. implement\n6. verify\n\nIf the user explicitly wants to skip a stage, say what risk that creates before proceeding.\n\n## When to Use\n\nUse this skill when the user wants to:\n\n- add a new source\n- redesign an existing source\n- add source capabilities such as `channel search`, `content search`, `content update`, or `content interact`\n- add support for RSS, APIs, scraping, browser automation, authentication, cookies, or remote side effects\n\nDo not use this skill for:\n\n- ordinary content operations against an existing source\n- unrelated CLI or store changes with no source work\n\n## Install From skills.sh\n\nInstall this skill directly from `skills.sh`:\n\n```bash\nnpx skills add https://github.com/severinzhong/agent-data-cli --skill authoring-data-cli-source\n```\n\n## Install\n\nIf `agent-data-cli` is not present locally, install it first:\n\n```bash\ngit clone https://github.com/severinzhong/agent-data-cli\ncd agent-data-cli\nuv sync\n```\n\nThen load the bundled skills from this repository's `skills/` directory and work from the repo root.\n\nImportant boundary:\n\n- source code belongs in the source workspace repo, typically `agent-data-hub`\n- keep `agent-data-cli` focused on core/cli/store/protocol work\n- do not install source runtime dependencies into the core project with `uv add`\n- use `uv pip install` or `init.sh` inside the source workspace instead\n\n## Workflow\n\n### 1. Research\n\nIdentify the source type before making architecture decisions.\n\nClassify it as one or more of:\n\n- RSS\n- API\n- HTML scraping\n- browser-driven\n- auth or session driven\n- interact capable\n\nResearch must confirm:\n\n- whether the source has a real `channel` concept\n- whether remote discovery and remote sync are distinct\n- how to identify unique content\n- whether the source has hierarchical or container-like content that should become `content_relations`\n- what time field is available\n- how pagination or incremental fetch works\n- what config is required\n- whether interact is actually possible\n\nUse available web research, local fetch tools, and the repo's `fetchers/` where appropriate.\n\n### 2. Spec\n\nWrite a source-specific spec before implementation.\n\nIt must define:\n\n- source to resource mapping\n- supported capabilities\n- config fields and mode if needed\n- content normalization and dedup strategy\n- `content_key` strategy\n- whether update returns only direct content, or also context nodes and `content_relations`\n- whether the source needs `relation_semantic` values such as `reply`, `contains`, or `list_item`\n- storage requirements\n- error boundaries\n- CLI-visible semantics\n- testing scope\n\nFor native search/query views:\n\n- treat column names as a soft compatibility surface because multi-source and multi-channel aggregation merges by column header\n- prefer explicit names such as `published_at`, `publisher`, `author`, `price`, `volume`\n- avoid vague names such as `time`, `source`, `value` unless that meaning is genuinely exact\n- column order is mainly for readability; header naming is what determines merge behavior\n\n### 3. Plan\n\nTurn the approved spec into an implementation plan.\n\nThe plan must break work into:\n\n- failing tests to add first\n- source code units to implement\n- `ContentSyncBatch` construction path\n- CLI verification steps\n- persistence and audit verification\n\n### 4. Approval\n\nWait for user approval after the spec and plan.\n\nDo not jump from research straight to code.\n\n### 5. Implement\n\nImplement with TDD.\n\n- write failing tests first\n- verify the failure is correct\n- write minimal code\n- rerun focused tests\n\n### 6. Verify\n\nBefore claiming completion, verify:\n\n- unit tests\n- CLI simulation tests\n- help output\n- capability and config behavior\n- persistence side effects\n- `content_nodes` / `content_channel_links` / `content_relations` side effects when update is involved\n- interact audit behavior when applicable\n\n## Read Next\n\n- `references/source-contract.md` for repository rules\n- `references/source-type-rss.md` for feed-like sources\n- `references/source-type-api.md` for JSON or HTTP API sources\n- `references/source-type-browser.md` for browser-driven sources\n- `references/source-type-interact.md` for remote side effects\n- `references/source-testing.md` for test matrix\n- `references/source-review-checklist.md` before final verification\n","topics":["Browser Automation","News","RSS","Sync"],"tags":{"latest":"1.0.1"},"stats":{"comments":0,"downloads":598,"installsAllTime":22,"installsCurrent":0,"stars":1,"versions":2},"createdAt":1773368180000,"updatedAt":1778491872769},"latestVersion":{"version":"1.0.1","createdAt":1773973432476,"changelog":"authoring-data-cli-source 1.0.1\n\n- Expanded scope and clarified usage for broader source types, including finance, news, and social media.\n- Added guidance for installing the skill directly from skills.sh.\n- Updated documentation to clarify the separation of source and core/cli/store dependencies and workflows.\n- Detailed the current resource model and emphasized new contract points (e.g., use of content_nodes, content_relations, and relation_semantic).\n- Improved research and spec phases to cover hierarchical content and naming conventions for data columns.\n- Provided updated workflow and verification steps reflecting current project architecture.","license":"MIT-0"},"metadata":null,"owner":{"handle":"severinzhong","userId":"s17e4txqn31ew031ywbnadxbqx842t0w","displayName":"Severin Zhong","image":"https://avatars.githubusercontent.com/u/14969913?v=4"},"moderation":null}