The Skill That Never Fired

A skill can fail in two ways. Its instructions can be wrong, so it does the job badly. Or Claude can decide never to load it, so the instructions never run at all. The first failure is obvious when you test the skill by name. The second only shows up when you test whether Claude chooses it on its own.

The second failure is the quiet one. You write a skill, you invoke it by name to check it, and it works. Then in normal use it just sits there. Claude answers without it. Nothing errors, nothing warns you, and the skill still shows as installed. It was never wrong. It was never chosen.

That choice is a routing decision, and Claude makes it by matching the request against your skill's name and description, before it reads a word of the body. The Claude Code docs say it directly: the description is what helps Claude decide when to load a skill. Anthropic tells you to test that decision, separate from the skill's output, and ships a tool that does it. Its skill-creator scores one target skill over repeated runs: does this skill fire on the prompts it should, and stay quiet on the ones it should not?

What that score does not tell you is what happened when another plausible skill was there too: whether the neighbour took the request, both fired, or neither did. That is the failure this piece is interested in, where your skill sits beside one that could answer it and the winner is not guaranteed to be yours. This walks through building a skill, watching that decision for yourself, and grading it against the neighbour it can lose to. You can run a first pass in about 15 minutes at a terminal.

What a skill is

At its simplest, a skill is a folder with one required file, SKILL.md. It can also hold scripts and reference files that load only when needed, but the minimum is the one file:

---
name: customer-date
description: "Format a date for customer-facing UK correspondence (emails, letters, messages to customers) as D Month YYYY. For CSV or data exports, use export-date."
---

Rewrite the date the user gives in UK long form, for example 30 August 2026. Reply with only the formatted date.

For a normal auto-invocable skill, the name and description sit in Claude's discovery context so it can decide whether the skill is relevant. The body below the frontmatter loads only when the skill is invoked, whether Claude chooses it or you type its name. Claude sees both the name and the description, and the description is the main field Anthropic gives you for saying when the skill should run. Write it for the router, not as a note to yourself. Claude Code also accepts a when_to_use field, appended to the description; these skills use only a name and a description.

Three common ways to use a skill. In claude.ai, turn on code execution, open Customize then Skills, and upload the folder as a zip. In Claude Code, put the folder in .claude/skills/ for one project or ~/.claude/skills/ for all of them. Through the Claude API, you upload it and reference its skill_id. The core SKILL.md format travels across all three, though installation differs and some frontmatter, including the disable-model-invocation used later, is specific to Claude Code.

Watching the routing decision

The mistake to avoid is judging a skill by its output. Ask Claude to format a date and you might get 30 August 2026 whether your skill ran or not, because the model can format a date on its own. The output tells you nothing about routing.

You want the decision itself. In Claude Code, a skill runs through a Skill tool that appears in the event stream. Run a prompt non-interactively and filter the stream down to the skill call:

$ claude -p "Rewrite this date for the customer email: 2026-08-30" \
    --output-format stream-json --verbose \
  | jq -c 'select(.type=="assistant") | .message.content[]?
           | select(.type=="tool_use" and .name=="Skill") | .input'

{"skill":"customer-date","args":"2026-08-30"}

That last line is the filtered result, not the raw stream, which wraps each event in more metadata. If the reader prints nothing, dump a raw event and look for a Skill call by hand, because the stream's shape shifts between versions. It is the routing decision read from the tool call, not guessed from the output. /skills shows which skills are available to Claude and /context shows the discovery listing's context cost, but neither proves this prompt invoked one. In claude.ai there is no equivalent machine-readable event. Anthropic's guidance is to review Claude's thinking to confirm a skill loaded, which works for checking by eye but not for building the kind of record above. And the event stream shows the skills Claude actually invoked, both of them when it invokes two, which is how a both shows up at all. What it does not expose is the candidate set: the other installed skills that were plausible but never invoked. You see what fired, not what it beat.

Routing is a decision you can grade

Whether a skill fires is a choice among whatever skills could plausibly answer the request. You can only grade that choice if you know what the right answer was before you run it.

So I built two skills with different jobs. customer-date, above, formats dates for customer emails in long form. export-date formats them for CSV exports as DD/MM/YYYY. Then I wrote a labelled prompt set: 4 requests that clearly want the customer skill, 4 that clearly want the export skill, and 4 date-adjacent requests that should fire neither. Every result gets one of four labels: right, wrong, none, or both.

Start with the failures, because they are where the method earns its keep.

Ambiguous request. Ask "Format this date: 2026-08-30" with both skills installed, and the results scatter: sometimes one fires, sometimes both, sometimes neither. That scatter is the expected result of an ambiguous request. The request never said whether it wanted the customer or the export format, so there is no correct answer to grade against. An ambiguous prompt is not a failed test, it is an ungradable one. If you cannot label the right skill before running it, the result cannot tell you whether Claude chose well.

Names and descriptions that draw no line. I named two skills for their output format, long-date and slash-date, and gave them the same vague description, "Format a date." Their bodies did different things, but their discovery metadata claimed the same job, so there was no boundary for the router to use and nothing told Claude which one fits a customer request. The grades went bad in the way that matters: one customer prompt fired nothing at all, and 2 export prompts fired both skills at once. Misses and double-fires, which is why "both" has to be one of your outcome labels.

Then the control, so you can see what clean looks like. Give the two skills distinct, use-case descriptions, and ask prompts whose wording matches those use cases, and routing is clean: 8 out of 8 to the right skill, and the neither-prompts correctly firing nothing. That is the model doing the keyword and intent matching you made easy for it. It is the case that should work, and it does. Note that the customer prompts contain words like "customer email" that are already in the customer skill's description. Clean routing here is a control condition, not proof that routing is robust. The evidence is in the failures above.

The fix

Routing runs on the discovery metadata Claude can see, which is the name and the description together, and my runs show either one can carry it. When the names were the vague part but the descriptions were sharp, routing was clean. When the descriptions were the vague part but the names said the use case, customer-date and export-date, routing was also clean, 8 out of 8 on the same prompts. That name-only run is an easy case, mind: the prompts carry the same words as the names, customer and export, so it shows a name helps when the request echoes it, not that a bare name is a strong signal on its own. It broke in one condition only: format-only names, long-date and slash-date, plus a shared vague description, where neither field told Claude what set the two skills apart.

That broken condition is the useful one. I left the weak names alone and rewrote only the descriptions around use cases, and the mess went to 8 out of 8. A good description rescued names that carried no signal. A good name had already done the same for descriptions that carried none. What you cannot do is leave both vague and expect Claude to find the line.

So write the description as a routing rule, not a summary. Put the use first. Include the words people actually type when they want this skill. Draw the boundary against the neighbour it might be confused with.

Some skills should not be auto-routed at all. Anything with a side effect or a real cost is safer as a skill you invoke by name, /customer-date, or one you lock with disable-model-invocation: true so only a person can trigger it. For those, manual invocation is the design, not a workaround. The rule underneath: how much routing error you can accept depends on what a wrong route costs. When the cost is high, the fix is often to stop routing automatically rather than to tune the description harder.

The failure that has nothing to do with your skill

There is one more way a skill stops firing, and no description work touches it. For skills still exposed to the model, Claude Code keeps every skill's name in the discovery listing, but the listing has a budget, around 1% of the model's context window, and once it runs over, Claude Code starts dropping descriptions, beginning with the skills you invoke least. That can strip out exactly the words that told two skills apart. So a skill whose description used to distinguish it cleanly can start missing once your catalogue grows large enough, with nobody editing it. /doctor reports the listing's cost. If a skill that used to route well starts slipping, check the size of your catalogue before you rewrite the skill: prune the skills you do not use, shorten the descriptions that survive so the distinguishing words fit, set low-priority skills to name-only so Claude keeps their names without their descriptions, or set rarely-used skills to disable-model-invocation: true, which takes them out of the router and its listing entirely; you still invoke those with /name.

Worth knowing before it bites a team: skills with the same name at different levels do not merge, one shadows the other. The order is enterprise, then personal, then project, so a /deploy skill in your ~/.claude/skills/ silently overrides the one your repo ships in .claude/skills/. If you commit skills for a team, give them names that will not collide, and do not rely on the project copy winning. This is also where overlap arrives for people who did not build it: a marketplace pack or an inherited folder drops in a skill whose description competes with one of yours, and the first you hear of it can be a skill that used to fire and now does not.

Which layer failed

The check separates two layers that a vague "it didn't work" runs together. Execution failure means Claude loaded the skill and the body did the wrong thing. Selection failure means the right body never got its chance to run at all. Everything else in this piece is a kind of selection failure: a discovery miss, interference from a neighbour, two definitions that overlap, a listing truncated at scale, a same-name skill shadowing yours. Only execution failure is about the instructions. The rest is why a skill can regress with nobody touching it, and why testing the body is only half the job.

The stakes climb once the skills matter. A date formatter losing to its twin costs you a wrong date format. A code-review skill that loses requests to a generic "help me with this file" skill costs you the review you thought ran on every change. Whether that happens turns on the same thing as the date skills: whether the two descriptions draw a line the router can use. The installed list will not tell you, so check it directly. Ask "look at this diff" with both installed and the route can go four ways: cleanly to the review skill, to a both, to the generic skill alone, or to neither. Give it requests whose correct skill you know, install it next to the neighbour you suspect, and read which one the Skill tool actually calls.

Run it yourself

The official skill-creator plugin measures a target skill's trigger rate for you. The manual version here adds the identity of the competing skill, so you can tell a miss from interference, a neighbour firing instead of the target or alongside it, reproduce a collision between two specific neighbours, and read the Skill call yourself. There is a second reason to read the calls rather than trust a score: as of the current evaluator (August 2026), it reads the first tool call in a run and counts the target as not fired if anything else, a neighbour skill included, gets there first, so the interference this piece is about can quietly lower the very rate meant to catch it. Here is the whole pack. Two skills, twelve prompts, four outcome labels, and the one-line reader from earlier. It is also a download, at durabilitycurve.com/tools/skill-routing-eval.

The skills, with descriptions that draw the boundary:

---
name: customer-date
description: Format a date for customer-facing UK correspondence (emails, letters, messages to customers) as D Month YYYY. For CSV or data exports, use export-date.
---
Rewrite the date the user gives in UK long form, for example 30 August 2026. Reply with only the formatted date.

---
name: export-date
description: Format a date for CSV or database exports (spreadsheets, data files) as DD/MM/YYYY. For customer emails and letters, use customer-date.
---
Rewrite the date the user gives in slashed form, for example 30/08/2026. Reply with only the formatted date.

The prompts, each with its known-correct skill:

should fire customer-date:
  Rewrite this date for the customer email: 2026-08-30
  Put this date in a letter to the client: 2026-08-30
  Format the date for a message to a customer: 2026-08-30
  Tidy the date in this customer-facing note: 2026-08-30

should fire export-date:
  Format this date for the CSV export: 2026-08-30
  Put this date into the spreadsheet export: 2026-08-30
  Format the date for the database file: 2026-08-30
  Prepare this date for a data export: 2026-08-30

should fire neither (date-adjacent work these formatters should refuse):
  What is today's date?
  When did the Second World War end?
  Parse this log timestamp: 2026-08-30T14:22Z
  What day of the week is 2026-08-30?

Install both skills, run each prompt through the jq reader above, and mark the result right, wrong, none, or both. Those labels roll up into three numbers worth watching. Recall: of the requests that should fire a skill, how many did. False triggers: of the requests that should not, how many fired it anyway. Interference: with a neighbour installed, how often that neighbour fires on a request meant for this skill, either instead of it or alongside it. Recall and false triggers are what the standard trigger-rate test measures for one skill, from its positive and negative cases. Interference is the number it cannot give you, because it only records whether the target fired, not which competing skill fired instead or alongside it. Score a both as a hit on recall and on interference at once: the intended skill ran, but so did a skill that should have stayed quiet. A skill that scores well alone and badly in company has a selection problem, and editing the body will not touch it.

What I measured on Claude Opus 5, arranged by what actually distinguished the two skills:

In this run, either signal alone held the line; only the condition where neither field distinguished the jobs produced misses and double-fires.

On the date-adjacent negatives above, run against the distinct-description pair, both skills stayed quiet: 0 false triggers in 4. Those negatives ran against the sharp descriptions only, so this does not show how false triggers rise as a description gets vaguer. Run the same set against the descriptions you plan to ship: a clean positive-routing score will not tell you whether a skill grabs adjacent work it should leave alone. Small numbers, one model, one surface, and single runs. The routing choice is a model decision that can scatter, so a clean 8 out of 8 is one draw, not a settled rate; run each prompt a few times and read how often the right skill wins, not a single mark. This is a diagnostic you run on your own skills, not a benchmark, and the caption matters more than the cells: this is the shape of the thing, not what Opus 5 does in general. Two skills is the floor, not necessarily the hard case. A real catalogue may have several plausible neighbours, so run the eval beside the skills your target actually competes with, not only against a clean pair.

The habit

A skill has two ways to fail. Its instructions can be wrong, and you probably test that already. Or Claude can never choose it, and that one leaves no mark: the skill sits installed, looking healthy, and quietly does nothing.

Test whether Claude chooses the skill. The output looking right does not prove the skill ran. The skills you never test that way are the ones you only think are working.

The pack here, two skills, twelve labelled prompts, and the run.sh reader, needs only the claude CLI and jq and is yours to keep: durabilitycurve.com/tools/skill-routing-eval.