Skip to content

feat(rf-core): add rf-run-digest skill - #12

Open
ai94iq wants to merge 2 commits into
MarketSquare:mainfrom
ai94iq:feat/digest-runs
Open

ai94iq wants to merge 2 commits into
MarketSquare:mainfrom
ai94iq:feat/digest-runs

Conversation

@ai94iq

@ai94iq ai94iq commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

What this changes

Adds the rf-run-digest skill to rf-core: reading a finished run with robotcode results and grouping failures by cause. Rebased on main without #11. Parked until robotcode has a grouping command: robotcodedev/robotcode#697.

AI assistance

Claude Code (Claude Opus) made the edits and ran the checks. I reviewed the diff and the check output, and I have the right to submit it.

Before you open it

  • Robot Framework capabilities go through robotcode; gaps are reported upstream.
  • Generated files were regenerated with their command, not edited by hand: python scripts/build-agent-variants.py and python scripts/build-catalog.py.

For the reviewer

Skill metadata follows 5.0.0 (no metadata.version; author ai94iq). Installer test counts and REQUIRED_ROWS include rf-run-digest. Maintainers per catalog.yaml: @manykarim.

Local checks: pytest tests --ignore=tests/eval green on the installer, description, structure and command suites; catalogue, governance, skill validation, plugin references and docs voice pass.

@ai94iq
ai94iq requested a review from manykarim as a code owner November 15, 2026 07:47
@ai94iq ai94iq changed the title Feat/digest runs feat(rf-core): add rf-run-digest skill Nov 15, 2026
@ai94iq

ai94iq commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

Merge chain: #11, then #12, then #13, then #14, then #15, then #16.

All six merge into main cleanly in this order (dry run on 2026-11-15).

Apart from #13 and #15, the PRs do not depend on each other.

@manykarim

Copy link
Copy Markdown
Collaborator

Thanks for reworking this so quickly, @ai94iq! Dropping the output.xml parser, the listener and the in-place shrink in favour of robotcode results is exactly the direction we want, and we've since written it down as "RobotCode first" in AI_POLICY.md. With that change, the remaining text mostly overlaps with what already ships. The part that is really new, grouping failures, deserves a better home, and I'd love your help getting it there.

What already covers result reading

Since content 3.0.0, these teach reading results with robotcode results:

  • the robotcode skill of the robotcode plugin, which rf-core points to;
  • rf-debug-expert (plugins/rf-core/agents/rf-debug-expert.md, lines 18-29), which uses results summary --failed, show --failed and log -bl and groups failures by cause;
  • the rf-core hooks, which steer the agent to robotcode results summary --failed;
  • step 5 of the rf-requests and rf-restinstance skills.

A separate skill that triggers after every run would also use part of the skill-listing budget (about 8,000 characters in Claude Code).

Where your grouping idea could go

1. Upstream in robotcode. Grouping failures that differ only by ids, hosts or timestamps would help every robotcode user, not just agents. An issue at https://github.com/robotcodedev/robotcode could look like this:

Title: results: group failed tests by normalised message

`robotcode results show --failed` lists each failure on its own. In large runs many failures
share one cause and differ only by ids, hosts or timestamps:

  Order 4711 not found (GET https://api-1.example.com/orders/4711)
  Order 4712 not found (GET https://api-2.example.com/orders/4712)

Proposal: `robotcode results show --failed --group-by message`, which replaces numbers, UUIDs,
hosts and timestamps with placeholders and prints each group once with its count and test names:

  2x  Order {id} not found (GET {host}/orders/{id})
      Orders.Get Order By Id, Orders.Get Archived Order

Your earlier digest code already shows the normalisation rules, so a PR to robotcode could be within reach too.

2. Here, once robotcode has it. The manual version is already on its way: your #14 adds fingerprints (ids replaced, hosts and timestamps dropped) to rf-debug-expert. When robotcode gets a grouping command, this PR could become the small follow-up that swaps those manual steps for the command, in rf-debug-expert and in step 5 of rf-requests and rf-restinstance. Could you link the robotcode issue here? We'll keep this PR open in the meantime.

When you update the branch, please base it on main without #11, regenerate with python scripts/build-agent-variants.py and python scripts/build-catalog.py, and name those commands in the PR.

AI assistance

We published AI_POLICY.md after you opened this PR, so you couldn't have known about it. It now applies to all open PRs, including this one. When you update, could you add the "AI assistance" section from the current PR template to the description (one line is enough, or "none" if no tool was involved)?

Thanks again for taking the feedback on board so well, and I'm looking forward to where this goes! 🙂

PS: I merged #13, #14, #15 and #16 today. I made a few small adjustments myself so they could go in right away; each PR's last comment lists them. Thanks for those! 🎉

@manykarim

Copy link
Copy Markdown
Collaborator

Hi @ai94iq, a short update: 5.0.0 is out and removed the rf-agentskills bundle, so bundles/ no longer exists on main. When you update the branch, please leave out the bundles/rf-agentskills/ files; python scripts/build-catalog.py no longer writes them.

Everything else from my last comment stays the same: we keep this PR open while the grouping idea goes to robotcode, so a link to that issue here would be great. Thanks! 🙂

Rebased on main without the rf-test-quality change.
@ai94iq

ai94iq commented Nov 20, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @manykarim. Rebased on main without #11, regenerated with build-agent-variants.py and build-catalog.py, and added the AI assistance section. robotcode issue for the grouping: robotcodedev/robotcode#697. Once it lands I'll turn this into the small follow-up for rf-debug-expert and step 5 of rf-requests and rf-restinstance.

Two harness tests list rf-core's staged skills by name; the new skill is now one of them.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants