Method

How a body of writing becomes fifteen numbers, how two texts are compared, and why the comparison deserves belief. The terms are defined on the Concepts page and the commands are in Developers.

What a parser can measure

Ask a model to write like you and it copies your topics and your favourite words. Those copy easily, and a reader who knows your writing recognises something else: how long a sentence runs before its main verb arrives, how deep the clauses embed, whether ideas get joined or subordinated, how often a claim becomes a noun. A dependency parser produces each of those for any passage, so a style can be written down as a specification and the result checked against it afterwards.

The fifteen features

Six are measured once per sentence and kept as distributions: sentence length in tokens, embedding depth, finite clauses, subordinate clauses, coordinated items, and tokens before the main verb. Each is recorded as a mean, a standard deviation and the tenth, fiftieth and ninetieth percentiles.

Seven are rates over the whole corpus: fragments per sentence, passives and agentless passives per finite verb, colon and dash predications per sentence, pronouns per hundred tokens, and subordinate clauses and coordinated items per sentence.

Two are categorical distributions: the share of tokens in each part of speech, and the frequency of the twenty five most common function words.

One sentence's tree describes that sentence, the way one coin flip describes that flip. A style is a distribution over many sentences, so a profile needs a corpus. Below roughly a dozen sentences the spread is noise, and the profile carries a warning that says so.

Every target is a range, because a writer who alternates a long subordinated sentence with a short flat one has a rhythm, and a generator that hits the mean every time flattens it. Uniformity is a claim about variance, so the spread is scored as its own part of each feature, and sentence length variation below a quarter of the mean is reported as a finding. Where a corpus is itself uniform, the specification tells the generator to vary anyway.

Voice match

Each of the six distributions contributes a distance on mean and spread together, with the spread weighted so that a rewrite cannot score well by landing on the mean every sentence:

Each rate contributes its absolute difference divided by the target rate, with a floor of 0.5 on the divisor so that a rare feature cannot dominate. The two categorical distributions contribute total variation distance. Divergence between a target profile P and a candidate C is the mean of the fifteen, and the reported score squashes it onto a readable scale:

Identical profiles score 100. The second step is strictly decreasing in D, so the squashing changes how the number reads and never changes an ordering.

Content kept

A rewrite may fix structure and may not invent content. The second number checks the second half of that, because the promise was previously a line in a prompt plus the model's own account of whether it had obeyed, and a model that fabricates is not a reliable witness to its own fabrication.

Numbers, dates and named entities in the source are anchors, compared after normalisation, so two, 2 and $4.2M against 4,200,000 all match. The score is anchor recall, less fifteen points per introduced item to a cap of forty five, less a penalty for novel content words above a ten percent free allowance, less a penalty for source sentences the output does not cover, less ten points per unexplained change in negation count.

The pass or fail verdict is stricter than the score and does not average. It requires no introduced item, no unexplained dropped anchor, a novel word rate at or below twenty five percent, sentence coverage at or above eighty percent, and no change in negation count that a negative verb does not account for.

Where the analysis finds missing information, a superlative with no comparison set, a passive with a suppressed agent, a pronoun with no recoverable referent, that stays missing and is reported as a gap. Dropping can be legitimate: an anchor named in a reported gap is excused. Introducing never is, and a self reported inference does not overturn a measurement.

Why structure carries a voice

Three small experiments on two public domain authors, Austen and Twain, run while the measure was being built.

Sentence length alone

Welch's t = 0.19, Cohen's d = 0.07. By this feature the two authors are the same.

Same length, different parse

Two sentences carrying the same facts at almost the same length:

Pooling weak features

Take Austen's profile and score two held out passages against it, one of Austen's and one of Twain's. For each feature, the distance to Austen's own passage should be the smaller one.

Fourteen of fifteen point the right way and one ties. If structure carried nothing about authorship, each feature would be a coin flip:

A sign test assumes nothing about how the distances are distributed, and these are not normally distributed. Sentence length, which separated nothing as a mean, contributes +0.130 once it is measured as a distribution.

Averaging k roughly independent features keeps the signal and shrinks the noise by the square root of k:

So the metric averages all fifteen at equal weight, and a text that happens to be flat on two or three of them still scores on the rest.

Held out passages against each profile

The diagonal wins in both directions. If a writer's unseen prose sat closer to a stranger's profile than to their own, nothing built on the measure would hold, so this matrix is computed before anything else.

The diagonal is 73.1 and 65.7. A profile estimated from a few dozen sentences carries sampling error, and writers vary between passages, so an author's own unseen prose scores below 100 against their own profile. That figure is the ceiling a restyle can reach, and the benchmark reports its results against a ceiling computed the same way. The Concepts page repeats the matrix for three Howard Marks memos.

The parser learned on newswire

en_core_web_sm was trained on published news, and it misreads note register English in repeatable ways. Five repairs run before any diagnostic, and each one that fires appears in the report: ISO dates and hyphenated compounds kept as single tokens, a head noun misread as a verb, a subject arc landing on a nominal head, one as a pronoun against one as a numeral, an accusative pronoun parsed as a subject.

Each sentence also carries a list of structural reasons to distrust its parse. The rewrite may dispute a finding on the grounds that the parser, and not the prose, went wrong. One such dispute turned out to be right and is now a regression test.

One measurement, two uses

The machinery that finds defects also describes styles. Five nominalisations is a bug report when auditing a draft and a fingerprint when imitating an author. The profile records what is there.

Where a measured feature collides with a rule, the rule wins and the report prints the override. The profile measures dash predication, because it is a complete description, and marks it as not a target, because the rules ban it.

Limits

The features are not independent, so the square root of k is an upper bound on the gain. Fifteen features and a handful of passages is a small experiment. The measure says nothing about lexical choice, humour, argument, or the decision about what to leave out, and a passage can match every feature and read like nobody, so the interface shows the prose above the score.