Harness Score 1.6.4: Safer Scans, Broader CI, and Trustworthy Docs

AI coding agents do not work in a vacuum. Their results depend on the context files, rules, skills, hooks, tests, CI pipelines, and safety controls that surround the model. That surrounding system is the harness: the part of an agentic workflow that a team can actually design, version, test, and improve.

Harness Score turns that idea into a deterministic repository audit. It checks 36 concrete signals across six dimensions, assigns a maturity level from L0 to L4, and shows the next gaps to close. It runs with zero LLM calls, zero network access during a scan, and zero runtime dependencies.

Since version 1.5.3, the project has gone through a focused release train. The goal was not to inflate the score or add flashy checks. It was to make the result safer to trust across large repositories, monorepos, self-hosted CI systems, and five documentation languages.

TL;DR

Harness Score can now:

• audit the structural readiness of an AI coding harness across Cursor, Claude Code, Codex, Windsurf, Cline, Continue, OpenCode, Zed, and other supported conventions;
• fail closed instead of presenting a partial filesystem walk as an authoritative score;
• inspect complete, deeply nested repositories with a much larger safety fuse;
• recognize CI configuration inside nested workspaces, Google Cloud Build directories, and Forgejo Actions;
• produce terminal, JSON, Markdown, badge, diff, and GitHub Action outputs;
• keep its check catalog, points, dimension totals, and maturity thresholds synchronized across English, Brazilian Portuguese, Spanish, Simplified Chinese, and Hindi.

The latest version is 1.6.4. You can run the default audit without installing it in your project:

npx --yes harness-score

What Harness Score is actually measuring

The harness engineering discussion is broader than any single tool. Birgitta Böckeler describes an outer harness that combines guidance with feedback sensors so agents can prevent mistakes and self-correct earlier. LangChain's anatomy of an agent harness similarly separates the model from the surrounding prompts, tools, permissions, state, and execution infrastructure.

Harness Score focuses that broad idea on evidence that can be found in a software repository. Its six dimensions are:

• Context & Guides: whether the agent can understand the project and its conventions;
• Skills & Commands: whether repeated procedures are packaged and discoverable;
• Hooks & Guardrails: whether important behavior is enforced during agent execution;
• Sensors & Feedback: whether tests, linting, types, formatting, and test files can check the work;
• CI Feedback: whether those sensors run as part of the delivery workflow;
• Hygiene & Safety: whether secrets, environment files, lockfiles, licenses, and MCP configuration are handled safely.

The result is not just a percentage. The maturity ladder distinguishes a repository that is merely documented from one that has feedback loops and enforced guardrails. Each failed check includes evidence and a remediation path in the complete check catalog.

Version 1.5.3: do not trust an incomplete scan

The most important change in 1.5.3 was a safety decision: an incomplete repository walk must not look like a complete evaluation.

Before this release, filesystem limits or unreadable paths could produce a partial view while still leaving a maturity result visible. Version 1.5.3 made incomplete repository and effective-scope walks fail closed. Terminal output, Markdown reports, badges, diffs, the CLI exit status, and the GitHub Action all learned the difference between an authoritative result and provisional diagnostics.

This matters because a high score based on only the easy-to-read part of a repository is worse than no score. It creates false confidence.

The same release improved native tool-root support. If a supported configuration directory was itself the scan root, Harness Score became better at recognizing Claude Code, Cursor, Codex, Windsurf, Cline, Continue, Gemini, Zed, OpenCode, and GitHub harness layouts. Hook configuration selection also became deterministic, with updated Claude Code event and handler validation and forward-compatible warnings for structurally valid future events.

Version 1.6.0: complete repository scans at real-world scale

Version 1.6.0 extended that fail-closed foundation. The production depth cap was removed, and the file-count safety fuse increased to 1,000,000 files.

That change is especially useful for monorepos and deeply nested workspaces. A CI workflow, rule, test suite, or harness artifact should not disappear from the evaluation simply because it lives farther down the tree than expected.

The scanner also began treating discovered paths that cannot be inspected and symlinks that escape the scan root as explicit incomplete-scan conditions. The important product behavior stayed the same: Harness Score can still explain what it found, but it will not pretend that a partial snapshot is an authoritative maturity verdict.

No check IDs, points, thresholds, or public APIs changed in 1.6.0. The release improved the reliability of the measurement rather than changing the ruler.

Version 1.6.1: CI inside nested workspaces now counts

Many repositories do not keep every CI file at the top level. A monorepo may contain independently deployed applications, each with its own Jenkinsfile, GitLab pipeline, Azure Pipelines configuration, CircleCI setup, or Bitbucket pipeline.

Version 1.6.1 made non-GitHub CI discovery work at any depth below the scan root. It also stabilized evidence ordering and removed duplicate evidence when matcher patterns overlap.

For users, the practical improvement is simple: you should not need to rearrange a legitimate workspace structure just to make the scanner understand that CI exists and runs tests or static checks.

This release was also a good example of open-source feedback improving the product. Thanks to Felipe Contratres for reporting the nested-workspace problem and contributing the initial fix, and to dbtorrico for identifying a guide-locale parity gap and suggesting translations.

Version 1.6.2: Google Cloud Build support

Version 1.6.2 added deterministic discovery for Google Cloud Build configurations under cloudbuild/ using .yml or .yaml files.

Those files can now contribute evidence to the CI pipeline, test execution, and lint/type-check checks. The scanner remains filesystem-only: it confirms that the configuration is present and contains the relevant commands, but it does not call Google Cloud or claim that a remote pipeline actually ran.

Thanks to Gabriel Caiana for reporting the gap and supplying a reproducible Cloud Build pipeline.

Version 1.6.3: Forgejo Actions support

Not every team hosts its repositories and automation on GitHub. Version 1.6.3 added support for Forgejo Actions workflows in .forgejo/workflows/.

That makes the CI dimension more useful for teams choosing a self-hosted, open-source forge. The same CI checks can recognize the workflow and its test and lint/type-check steps without adding a provider API, credentials, telemetry, or network behavior to the scanner.

Taken together, versions 1.6.1 through 1.6.3 move Harness Score closer to its intended position: the repository structure and working controls should matter more than the vendor logo on the pipeline.

Version 1.6.4: documentation that cannot silently drift

The scanner already had the correct weights, but the compact check table in the documentation had five stale values. Version 1.6.4 corrected them in every locale:

• CI-03: 3 points;
• CI-04: 3 points;
• HYG-01: 2 points;
• HYG-06: 2 points;
• HYG-08: 3 points.

More importantly, the release expanded the synchronization tests. CI now verifies unique and complete table rows, the weights defined by the implementation, all six dimension totals, detailed catalog headings, the 108-point total, and the maturity thresholds across all five languages.

This patch did not change anyone's score. It aligned the public explanation with the implemented model and made the same class of drift fail CI in the future. The 1.6.4 release notes include the complete validation record.

Turn the score into a ratchet, not a one-time report

A local scan is the fastest way to understand the project, but the score becomes more useful when it is part of the engineering loop.

Save a machine-readable baseline:

npx --yes harness-score --json > harness-score.json

Compare a future scan against it:

npx --yes harness-score --diff harness-score.json

Or set a minimum maturity level in CI:

npx --yes harness-score --min-level 3

The GitHub Action can run the same audit on every push or pull request:

- uses: paladini/harness-score@v1
  with:
    min-level: '3'

For stricter supply-chain control, pin the Action to a full commit SHA. For a quick adoption path, the stable v1 tag follows compatible 1.x releases.

What the score does not prove

Harness Score is an objective measure of repository-level harness infrastructure, not a certification of final software quality.

It can verify that tests, linters, type checking, CI workflows, context files, hooks, and safety patterns exist and are structurally connected. It does not prove that the tests assert the right behavior, that an AGENTS.md is perfectly written, that a remote pipeline is healthy, or that an agent understood the product requirement.

Use it as a readiness map and an improvement backlog. Combine it with code review, runtime observability, product evaluation, security testing, and semantic checks appropriate to your system.

What is planned next

The public roadmap is a direction of travel, not a promise of dates or release numbers. The main planned areas are:

1. harness-score init: deterministically scaffold starter artifacts for high-value missing checks, turning diagnosis into an optional, reviewable first fix.
2. SARIF output: bring relevant findings into GitHub's code-scanning experience.
3. Larger real-world corpus studies: use evidence from public repositories to challenge and calibrate thresholds, weights, and missing checks, then publish the findings.
4. Broader ecosystem detection: recognize more test runners, linters, type checkers, and CI conventions as real repositories expose gaps.
5. More tool adapters: continue the multi-harness plugin plan beyond the current Cursor and Claude Code work, with lightweight integrations for Codex, OpenCode, Junie, Windsurf, Cline, Continue, and Zed before considering a heavier VS Code extension.

There are also narrower reliability improvements under discussion and community review, including generated environment directories that should not invalidate a scan and better parity when a tool configuration directory is itself the repository root. These will only become release claims after implementation, review, and the same cross-platform gates used by the current release train.

Try the latest release

Run this from the root of a repository:

npx --yes harness-score

Then look at the first failed checks and ask a practical question: which missing artifact would reduce the most repeated agent mistake in this project?

You can explore the guide, inspect the full changelog, or contribute a detector and reproducible fixture through the GitHub repository.

This article was prepared with AI assistance. Release claims, commands, roadmap language, tags, links, and the current 1.6.4 package metadata were checked against the public repository, documentation, DEV Community, and package registry before publication.

Which part of your AI coding harness is hardest to make visible today: context, feedback, guardrails, or CI?