Blog

mcpscore 1.12.0: configure the rules your CI scores on

· 5 min read

Some of the 105 rules will never apply to your server, and some matter more to you than their default weight says. mcpscore 1.12.0 adds a mcpscore.toml that turns rules off and re-ranks the rest for your own CI, the way you configure a linter. Your badge and your page on mcpscore.dev keep the canonical score.

One file, the linter way

Check the file in next to your server. Every entry is keyed on the rule’s rule_id, the same stable identifier the JSON report and the rules reference use, so there is nothing new to learn.

# mcpscore.toml, checked in next to your server
[rules]
server_websiteurl_present = "off"       # we don't publish a website
server_icons_present      = "off"
server_title_present      = "critical"  # our agent picks tools by title

[gate]
fail_on = "critical"   # any failed rule at or above CRITICAL fails the build

off means the rule does not run: it counts toward neither your score nor your maximum, and the report lists it under skipped rules with the reason disabled-by-config. A severity name re-ranks a rule to that weight, from one point for LOW to five for CRITICAL, and the result records the default it moved from. If you keep your project metadata in pyproject.toml, a [tool.mcpscore] table works the same way.

The score you gate on is the score under your policy

We considered reporting two numbers, the canonical score and a configured one, and rejected it. If you turned a rule off, you have already decided it is not your concern; a second number that keeps counting it is noise. So the configured score is the score for that run, and the score line says which file shaped it.

Config: mcpscore.toml — 2 rules off, 1 re-ranked, gate at CRITICAL
...
Audit finished. Final score: 78/92 (mcpscore.toml: 2 rules off, 1 re-ranked, gate at CRITICAL)
Gate failed — [gate] fail_on = "critical": 1 failed rule at or above CRITICAL: server_title_present

That is a real run against DeepWiki, which fails all three configured rules. Without the file it scores 78 of 91. The two LOW rules that are off take one point each out of the maximum, and the title rule re-ranked from MEDIUM to CRITICAL adds three, so the maximum lands on 92. The score holds at 78 because none of those rules was earning points. The qualifier travels with the number wherever it gets pasted, and the gate names the rule that failed the build.

A gate that names the rule

--fail-under already fails a build below a score. The new [gate] table fails it on severity instead: any failed rule counted in the main score at or above fail_on, after re-ranking, exits with code 3 and lists the offenders. Promote a rule to CRITICAL and it now blocks merges; a rule that is off never can. Both gates share exit code 3, and the report’s config block records which one tripped.

The file is forward-compatible on purpose. A rule_id this release does not know, including one it has retired, produces a warning and is ignored, so a configuration written for a newer mcpscore still runs on an older one. Any other value is a usage error, because a silently ignored typo is worse than a red build.

Your badge stays canonical

A configuration changes the score for the run that used it and nothing else. The web service never reads one: every audit on mcpscore.dev runs with every rule at its default weight, and only those audits feed the badge. That is an architectural property, not a scoring rule, which is why the public number stays comparable across servers while your CI number answers your own question.

Two teams with different files get different numbers for the same server, and that is the point. Compare configured scores only across runs with the same config.sha256, which the report carries beside the source file, the disabled rules, and every re-rank. To reproduce the canonical number inside a configured repository, pass --no-config.

In the GitHub Action

The mcpscore GitHub Action v1.1.0 picks the file up from your checkout, so min-score gates the configured score and the pull request comment names the file, what it changed, and the rules that tripped a configured gate. A gate exit now publishes the report, outputs, and job summary before failing the job, where it used to read as “could not audit”.

# The nearest mcpscore.toml up to the repository root applies
uvx mcpscore==1.12.0 ./server.py --fail-under 80

# The canonical score, ignoring every configuration source
uvx mcpscore==1.12.0 ./server.py --no-config

Try mcpscore 1.12.0

Read the complete release notes, inspect the package on PyPI, and follow the configure-rules guide for discovery order, the pyproject.toml form, and the report fields. Start by turning off the rules that cannot apply to you; promote the ones your agents depend on once the file is in.

The file composes with the rest of the CLI: smoke mode runs after the configured audit, and the canonical score is always one paste away on mcpscore.dev.