Concepts
The vocabulary of dependency grammar, each term defined the way the code measures it, with an example run through the same parser the tool ships. Every arc and every number in the figures comes from that parser. The Method page then shows how the measurements become a score, and the page ends with three Howard Marks memos put through all of it.
Token
A token is the unit the parser counts. Most tokens are words. Punctuation marks
are tokens too, and a contraction splits into two, so didn't is did and
n't. Sentence length in the profile counts tokens that are neither
punctuation nor whitespace: "The board asked for a correction." is six.
A repair pass runs before the count, because the parser was trained on news and splits things a memo writer keeps whole. ISO dates and hyphenated compounds stay single tokens after it.
Part of speech
Every token carries two labels. The coarse one is a class from the Universal Dependencies set, seventeen in all: NOUN, VERB, ADJ, ADV, PRON, DET, ADP (preposition), AUX (auxiliary), CCONJ and SCONJ (coordinating and subordinating conjunction), PART (particle), NUM, PROPN (proper noun), PUNCT and a few rarer ones. The fine one is a Penn Treebank tag, and it carries grammar the coarse label drops: NN against NNS is singular against plural, VBD against VBN is a past tense against a past participle, JJ against JJS is an adjective against its superlative.
The profile uses the coarse labels for the part of speech mix and for pronoun counting, and the fine tags for deciding whether a verb is finite.
Head, dependent, arc
A dependency parse draws one arc for every token except one. The arc runs from
a head to a dependent, and its label names the relation. In "The board
asked for a correction", board depends on asked as its subject, The
depends on board as its determiner, for depends on asked as a
prepositional modifier and correction depends on for as the object of the
preposition. The token with no head is the root, usually the main verb. The
whole thing is a tree: one root, and every token reachable from it by following
arcs downward.
The labels that appear on this site:
| label | relation | example |
|---|---|---|
nsubj |
nominal subject | the board asked |
nsubjpass |
subject of a passive | the number was repeated |
dobj |
direct object | wrote the correction |
det |
determiner | the board |
amod |
adjectival modifier | other decks |
advmod |
adverbial modifier | read it twice |
aux, auxpass |
auxiliary, passive auxiliary | had corrected, was repeated |
neg |
negation | did not match |
prep, pobj |
preposition and its object | asked for a correction |
agent |
the by phrase of a passive | repeated by the team |
mark |
subordinator introducing a clause | because, that, if |
cc, conj |
conjunction and coordinated item | went out, and nobody corrected it |
advcl, ccomp, xcomp, relcl, acl, csubj, pcomp |
the subordinate clause types, below | |
compound, flat, appos |
parts of one name or noun phrase | Ann Lee |
punct |
punctuation |
Finite verb and clause
A verb is finite when it carries tense or is a modal. In Penn tags that is VBZ (asks), VBD (asked), VBP (ask, with a plural subject) and MD (will, could, might). Base forms after an auxiliary (did not match), participles (corrected, asking) and infinitives (to leave) are not finite.
The tool counts clauses by counting finite verbs, because a clause is one
assertion and a finite verb is what asserts. "The numbers did not match" has
one finite verb, did; match is a base form riding on it. "Pricing will hold"
has one, the modal will. Finite clauses per sentence is one of the six
sentence level features, and a sentence with no finite verb at all is a
fragment.
Coordination and subordination
Coordination joins two things of equal rank with and, but, or. The
conjunction is labelled cc and the second item hangs from the first on a
conj arc, at the same level. Subordination makes one clause part of
another, and the parser has a label for each way that can happen:
| label | kind of clause | example |
|---|---|---|
advcl |
adverbial: reason, time, condition, result | she read it because the numbers did not match |
ccomp |
clausal complement with its own subject | the view is that pricing will hold |
xcomp |
open complement, sharing a subject | wanted to leave |
relcl |
relative clause on a noun | the note which nobody had corrected |
acl |
other clausal modifier of a noun | the number quoted by the board |
csubj |
a clause as subject | what he said was wrong |
pcomp |
a clause as object of a preposition | by measuring first |
The profile counts subordinate clauses per sentence as the number of tokens
carrying one of those seven labels, and coordinated items per sentence as the
number of conj arcs, so "A, B and C" counts two.
The two read differently because of what the reader has to hold. A coordinated clause closes before the next opens. A subordinate clause stays open, and the sentence is unresolved until it closes. Austen's sentences in the sample corpus carry 2.6 subordinate clauses each; the draft that ships with the demo carries 1.8, and only a third as much coordination.
Embedding depth
Depth is the longest chain of arcs from the root down to a leaf, counting only
the arcs that nest. The root alone is depth 1. Arcs labelled conj, cc,
compound, flat, appos, punct and dep cost nothing, because they join
siblings or parts of a single name, and a reader holds nothing open while
crossing them. Without that rule, "with Ann Lee and Bob Ray" would score three
levels of nesting for four names.
The analyzer reports a finding when a sentence passes depth 8. The profile only records the distribution. The number is dependency depth, measured head to leaf, and a phrase structure tree of the same sentence would give a different one.
Tokens before the main verb
The count of tokens to the left of the root, punctuation excluded. A sentence that opens with a long subject, a fronted clause or a stack of modifiers makes the reader wait for the verb; rhetoric calls that shape periodic. A sentence whose verb arrives in the second or third word and trails its qualifications behind is loose. Both are legitimate, and writers lean one way.
Passive voice
In a passive clause the grammatical subject is the thing acted on, and the verb
takes a form of be plus a past participle. The parser labels the subject
nsubjpass and the auxiliary auxpass. When the writer names the actor, it arrives in
a by phrase labelled agent. Otherwise the clause is an agentless passive, and
the actor is absent from the sentence rather than merely placed late.
The profile records two rates, passives per finite verb and agentless passives per finite verb, and rates neither. The analyzer, which is looking for defects rather than describing a style, flags every agentless passive as a suppressed agent, and also catches the reduced form with no auxiliary, as in "the number quoted to the board".
Fragments and punctuation predication
A fragment is a sentence with no finite verb. It asserts nothing, so the parser roots it on whatever is left: a noun, an adjective, an adverb.
Punctuation predication is a particular way of producing one. A colon, semicolon or dash stands where a verb would go: "Pricing: unchanged since March." The colon is doing the work of is. Punctuation cannot head a clause, so the parser roots on the noun and hangs the rest off it as a modifier.
The profile counts colon and dash predications per sentence as the number of those punctuation tokens, and the specification marks the result as measured but not a target, because the writing rules the generator follows ban the construction. The fragment rate is a target: some writers use fragments on purpose, and Crane's prose in the benchmark is one example.
Pronouns
Pronouns per hundred tokens counts every token the parser classes as PRON: the personal ones, the relative which and that when they stand for a noun, and quantifier pronouns like nobody. Narrative prose that follows people runs high; memo prose that names things runs low. The sample corpus runs at 12.6 per hundred tokens and the draft at 4.2.
Nominalisation
A nominalisation turns a claim into a noun, so the writer can handle it without asserting it: "the view is that pricing will hold" against "we think pricing will hold". The analyzer keeps a list of container nouns (view, read, sense, concern, thesis, assessment and others) and flags one when it heads a subject or a predicate and nobody owns it. With an owner, "her view", it passes. The analyzer reports it as a finding; the profile does not measure it.
Function words and the part of speech mix
Words divide into open classes, which admit new members every year, and closed classes, which do not: determiners, prepositions, pronouns, conjunctions, auxiliaries and particles. The closed class words are the function words. They carry no topic, so their frequencies survive a change of subject, and most authorship attribution work rests on them, beginning with Mosteller and Wallace, who separated Hamilton from Madison in the Federalist Papers in 1964 on words like upon and whilst.
The profile keeps the twenty five most frequent function words per hundred tokens. For the sample corpus and the demo draft, the top of each list:
| sample corpus | per 100 | draft | per 100 |
|---|---|---|---|
| of | 4.55 | to | 3.16 |
| to | 3.74 | a | 3.16 |
| in | 3.21 | for | 3.16 |
| was | 3.21 | the | 3.16 |
| her | 3.21 | of | 3.16 |
| a | 2.94 | in | 2.11 |
| the | 2.94 | that | 2.11 |
| and | 2.67 | these | 2.11 |
Her in fifth place is a fact about a novel with a female protagonist. These in eighth place is a fact about a draft that keeps gesturing at things it has already said.
The part of speech mix is the share of tokens in each coarse class, kept for the fourteen largest classes. It records whether a writer builds sentences out of nouns and prepositions or out of verbs and adverbs.
Distributions
The profile measures each of the six sentence level features once per sentence and keeps the resulting distribution: mean, standard deviation and the tenth, fiftieth and ninetieth percentiles. The specification the generator receives quotes the range from the tenth to the ninetieth percentile and the standard deviation, and tells it to land inside the range rather than on the mean.
The coefficient of variation is the standard deviation divided by the mean. Below a quarter, the checker reports uniform rhythm, which is the pattern readers most often identify as machine written. The draft above sits at 0.22.
Distance between two texts
Three kinds of feature, three distance rules, all normalised so that 0 is identical and 1 is a large gap.
A distribution feature compares means and spreads together: 0.65 times the difference in means, divided by the target mean, plus 0.35 times the difference in standard deviations, divided by the target standard deviation. The weight on spread is deliberate: a rewrite that matches the mean and flattens the variance is the usual failure.
A rate compares two numbers: the absolute difference divided by the target rate, with a floor of 0.5 on the divisor so that a rare feature cannot dominate.
A categorical feature, the part of speech mix or the function word list, is compared by total variation distance: half the sum, over every category, of the absolute difference in share. Two identical distributions score 0; two that share no category score 1.
How the fifteen distances become the single 0 to 100 score is on the Method page.
The parser
The parser is spaCy's en_core_web_sm, a small statistical model trained on
annotated news and web text. It assigns tags and arcs by prediction, which
means it can be wrong, and it is wrong in repeatable ways on note register
English: a date read as a number, a head noun read as a verb, an accusative
pronoun read as a subject. The Method page lists the five repairs that run
before measurement. The parse is the measurement instrument, and every figure
on this page is what the instrument produced, corrections included.
Worked example: three Howard Marks memos
Howard Marks has written memos to Oaktree's clients since 1990. Three recent ones, Is It a Bubble?, What's Going On in Private Credit? and A Look Under the Hood, come to 744 sentences and 16,204 tokens of prose once headings, bullet lists and the legal footer are set aside. The memos are copyrighted, so the repository holds the numbers and two quoted sentences, and the script that produced them reads the text from wherever you saved it.
A sentence near the middle
Of the 744 sentences, this one from the private credit memo sits closest to the medians on length, depth, clause counts and verb position at once:
The shape recurs through all three memos: a main clause on a passive verb, a trailing participial clause that carries the point, and a coordinated pair inside it.
The deepest short sentence
Among sentences of 23 tokens or fewer, the deepest chain runs twelve arcs:
The distribution
Marks varies sentence length by 51 percent of its mean, Austen by 54, the draft by 22. On this axis an investor memo and an 1813 novel sit together and the machine draft sits apart.
The fifteen numbers
The six distributions as mean, standard deviation and the tenth to ninetieth percentile range, then five of the seven rates:
| feature | Marks | Austen sample | demo draft |
|---|---|---|---|
| sentence length, tokens | 21.8sd 11.1, 8 to 36 | 26.7sd 14.4, 8 to 48 | 15.8sd 3.5, 12 to 18 |
| embedding depth | 6.4sd 2.4, 4 to 9 | 6.6sd 2.5, 4 to 9 | 6.5sd 1.1, 5 to 7 |
| finite clauses per sentence | 2.2sd 1.3, 1 to 4 | 2.5sd 1.1, 1 to 4 | 1.8sd 0.7, 1 to 2 |
| subordinate clauses per sentence | 1.9sd 1.7, 0 to 4 | 2.6sd 1.4, 1 to 4 | 1.8sd 1.1, 0 to 3 |
| coordinated items per sentence | 0.8sd 1.0, 0 to 2 | 1.3sd 1.3, 0 to 3 | 0.3sd 0.8, 0 to 0 |
| tokens before the main verb | 6.2sd 6.3, 1 to 15 | 8.2sd 7.1, 1 to 20 | 5.2sd 3.5, 1 to 7 |
| fragments per sentence | 0.027 | 0 | 0 |
| passives per finite verb | 0.103 | 0.171 | 0 |
| agentless passives per finite verb | 0.091 | 0.143 | 0 |
| colon and dash predications per sentence | 0.176 | 0.429 | 0 |
| pronouns per 100 tokens | 7.4 | 12.6 | 4.2 |
Marks writes shorter sentences than Austen at nearly the same depth, subordinates less, coordinates far less, uses half her pronouns and reaches the verb two tokens sooner. Against the draft the comparison runs the other way: the mean depth is the same, and the draft's depth never leaves 5 to 7 while his ranges 4 to 9. The draft has no passives, no fragments and no colons. He has some of each, in small and steady amounts.
The function words:
| Marks | per 100 | Austen sample | per 100 | draft | per 100 |
|---|---|---|---|---|---|
| the | 6.26 | of | 4.55 | to | 3.16 |
| to | 3.12 | to | 3.74 | a | 3.16 |
| of | 2.84 | in | 3.21 | for | 3.16 |
| and | 2.62 | was | 3.21 | the | 3.16 |
| in | 2.38 | her | 3.21 | of | 3.16 |
| a | 1.91 | a | 2.94 | in | 2.11 |
| that | 1.78 | the | 2.94 | that | 2.11 |
| it | 1.20 | and | 2.67 | these | 2.11 |
The at 6.3 per hundred tokens, twice Austen's rate, is a writer naming things: the market, the Fed, the bubble, the lender. Austen's her, was and he are a writer following people. Further down Marks' list sit I at 0.84 and but at 0.65, and neither appears in the draft's top twenty five.
Does the profile know him?
Each memo scored against a profile built from the other two, and against the Austen profile. Then the Austen passages the other way round, and the draft against both.
| held out | vs Marks profile | vs Austen profile |
|---|---|---|
| Marks, A Look Under the Hood | 83.2 | 74.7 |
| Marks, Is It a Bubble? | 86.0 | 69.0 |
| Marks, What's Going On in Private Credit? | 86.2 | 72.2 |
| Austen passage, mean of four | 54.7 | 56.7 |
| the demo draft | 68.7 | 55.9 |
A memo the profile never saw scores 83 to 86 against the other two memos and 69 to 75 against Austen, and the gap holds for all three. The Austen row is close because her sample is fourteen sentences in four short passages, which is the noise the Method page warns about. The draft lands nearer Marks than Austen, as a business memo in any voice should, and eighteen points short of his own unseen prose.
The specification
What the generator receives in profile mode, built from the three memos. No sentence of his is in it.
SYNTACTIC TARGET PROFILE (marks: 3 documents, 744 sentences, 16204 tokens) LANGUAGE: English. The corpus is English prose and the output must be English. Hit the RANGES, not the means. Varying within the range is required -- landing on the mean every sentence is the main way this goes wrong. RHYTHM: vary sentence length by about 11.09 tokens (sd), matching the corpus at 51% of its mean. sentence length (tokens) 21.78 (typical range 8.0-36.0, sd 11.09) embedding depth 6.4 (typical range 4.0-9.0, sd 2.37) finite clauses per sentence 2.17 (typical range 1.0-4.0, sd 1.27) subordinate clauses/sentence 1.9 (typical range 0.0-4.0, sd 1.69) coordinated items/sentence 0.77 (typical range 0.0-2.0, sd 0.98) tokens before the main verb 6.17 (typical range 1.0-15.0, sd 6.3) RATES sentences with no finite verb 0.027 passives per finite verb 0.103 agentless passives per finite verb 0.091 colon/dash predications per sentence 0.176 <- MEASURED BUT NOT A TARGET: colon/dash predication is banned by remove-slop §1. Do not reproduce it. pronouns per 100 tokens 7.41 subordinate clauses per sentence 1.901 coordinated items per sentence 0.772 PART-OF-SPEECH MIX (% of tokens) NOUN 24.23%, VERB 12.88%, ADP 11.82%, DET 9.88%, ADJ 8.82%, PRON 7.41%, AUX 6.59%, ADV 4.55%, CCONJ 3.81%, PART 3.2%, PROPN 2.67%, SCONJ 2.57%, NUM 1.33%, SYM 0.12% FUNCTION-WORD FREQUENCY (per 100 tokens) -- the strongest authorship signal the 6.26, to 3.12, of 2.84, and 2.62, in 2.38, a 1.91, that 1.78, it 1.2, for 1.13, is 1.09, be 0.96, i 0.84, but 0.65, on 0.64, as 0.54, they 0.54, this 0.53, will 0.52
A restyle against this specification is scored the way the benchmark scores one. The demo accepts any corpus, so the three memos and a draft of your own will reproduce every number on this page.