JavaEye

A four-week title-and-links experiment where the untouched pages outperformed their forecast

Three numbers before the story:

The site is a US legal-forms reference property with 38 data pages. On 2026-10-18 I split 18 of them into three arms of six, matched so each arm started identically: two pages in Discovered, three unknown, one Crawled — not indexed. On 2026-11-15 I re-inspected all 22 URLs (the 18 plus 4 off-study pages) and read the results against the design I wrote before the experiment started.

This is a field report about one site in one window. Treat the method as the takeaway, not the percentages.

The title-edit arm and the control both finished at 5/6

Arm A got title rewrites and nothing else. The rewrite was structural: every title became Free {State} Will Form | Rules & PDF | statelegalforms — keyword, then a concrete promise the page actually keeps (a signing-requirements table and a fillable PDF), then the brand. Fifty-one to fifty-nine characters each — 51 for Ohio, 59 for Pennsylvania, per the change ledger. H1, body, description, JSON-LD: untouched.

After four weeks: 5 of 6 indexed. Arm C, which I never touched, also reached 5 of 6. The two arms finished with the same observed indexed count. This experiment did not establish an incremental benefit from title edits, and the endpoint counts do not establish whether indexing speed differed. My recorded prior for arm A was 5/6 — met, not falsified.

One nuance kept me honest here: arms A and B were submitted to Search Console in a single batch on 2026-10-18 at 20:41 (12 URLs), and arm C was never submitted. So arm A versus arm C compares title edits plus submission against neither. The matched outcome bounds what I can claim either way, but the clean zero-effect sentence is off the table.

The one unindexed page in each arm matters more than the totals. In arm A it was ohio-oh; in arm C it was arizona-az. Both were sitting at Crawled — currently not indexed before the experiment began, and both are still sitting there. Of the three pages that entered the experiment already stuck at Crawled, only one — michigan, in arm B — converted. The failure mode on this site is a pre-existing crawl-and-decline state that four weeks of treatment did not reliably flip.

Arm B went 6/6 — every page indexed, the only perfect arm. The treatment worked on inbound links, and the arrangement matters. Baseline pages each had 3 internal links by the deduped-source-page count, all coming from the same three places: the form-type page’s all-states grid and two blog posts’ state lists. The six arm-B pages formed a ring, each gaining a “Compare Will Requirements in Other States” section with links to the next three sibling pages in the ring — so every arm-B page gained exactly three new linking source pages, taking the count from 3 to 6. Anchors were the state name plus “will requirements.” No links were added to any arm-A or arm-C page, and the control pages were byte-locked for the whole window.

The gap over control is one page. With six pages per arm, I can’t tell whether that one page is an effect or noise, and the draft of this post originally said so in both directions at once — first that one page “isn’t a signal,” then that the link arm “did not hurt.” Both sentences exceed the evidence. What the retest actually shows: all six arm-B pages were indexed, and the arm’s one pre-existing Crawled page (michigan) converted where both pre-existing Crawled pages in the other arms stayed stuck. That’s enough to justify a small extension — 6 to 12 more pages — and the record next to it says one-page gap, n=6, confounded, not “links work.”

The trap would have been publishing “internal links 3→6: 100% indexed.” The 6/6 count is real. The causal story is what the off-study pages undercut: 4 untouched, unsubmitted pages also indexed — 4/4 — so a rising baseline could produce most of that perfect score without any link effect at all.

The control beat a subjective prior, and the cause is unresolved

Arm C delivered 5/6 against a recorded expectation of 2.2 of 6. That forecast deserves precision, because I got it wrong twice while writing this post. First draft said “3/6, from a baseline of 9 indexed out of 38 pages.” Neither number holds up: the design doc’s prior was 0.37 per page — 2.2 of 6 — and it was a subjective judgment written down before the experiment, not a calculation from the snapshot. The 9/38 figure was a separate static reading of the site at design time. Mixing them made the forecast sound more grounded than it was.

The miss itself is real and pointing the same way as the off-study pages: untouched cohorts indexed at a rate my pre-experiment judgment didn’t expect. The weekly inspection one day before the retest put the whole property at 37 of 39 known URLs indexed — 94.9% — a site-wide snapshot that includes the 22 retest URLs plus the 17 data pages not on study. What I cannot say — and the earlier draft of this post did say — is why. The results are consistent with a broad indexing increase on a maturing property that a static forecast missed. Whether that’s site age, a data-page batch quality pass, or something else, this experiment did not establish it. The forecast bias is one observation, not a systematic law, so the next round tests it rather than assumes it.

The control arm earned its keep anyway: it is what stops a 6/6 from reading as proof.

My next experiment will standardize submission and pre-register decision thresholds

Four fixes, all written into the next design doc before any page gets touched:

  1. Priors get a derivation and a time term. No more bare subjective numbers: each prior must state its inputs, and the control expectation must come from the site’s recent indexing trajectory, not a single historical count. If the trajectory can’t be estimated, the design must say so and treat the prior as a guess with an error bar.
  2. Submission becomes a designed factor, not a leftover. This round aligned A with B (same batch) but left C unsubmitted, so each treated-versus-control comparison also differs in submission status. Next round either submits all arms identically or drops submission from the treated arms entirely — one of the two, chosen in advance.
  3. The extension gets pre-registered thresholds. The 6–12 page link extension will state its comparison population, its submission handling, its outcome window, and what result upgrades the evidence label — before launch. “Reproduces, therefore true” is not a threshold; treated-versus-matched-control with defined margins is.
  4. Pre-existing states stay named, not averaged. Both failures across all arms were pages already stuck at Crawled before day one. Separating “converted from Crawled” from “stayed Crawled” told me more than the arm totals did, and it survives into the next report format.

One more number for the ledger: the retest itself cost one batch of API calls — 22 URL Inspection requests, zero failures — against weeks of not knowing. My day-one design doc kept my week-four interpretation anchored to what I had specified in advance.

Earlier experiments on this property’s indexing behavior are in my write-up of getting discovered pages indexed with real data. The method there and the method here share one rule: never report a number you can’t trace to an inspection response.