AnswerHub AEO/SEO/GEO + question corpus import — spec

What this covers: turning AnswerHub from a nine-row demo into the landing site for Taiwanese holiday-homework and reading-certification answer queries, by importing two question corpora the owner already holds, enriching every question with model-written working, and building the discovery surface (taxonomy pages, sitemap, robots, structured data) that the product currently has none of.

For what is built or deployed, see ClaudeDocs/HANDOFF.md — status deliberately does not live here.

Supersedes nothing. Builds on SPEC_ANSWERHUB_CRAFT.md (navigation and entry points), SPEC_ANSWERHUB_SHELL.md (page shell, CJK fonts) and SPEC_LWOPAN_MIGRATION.md (the .lpans batch format).

Problem Statement

AnswerHub is a working answer bank with no answers in it. Measured from the internet on 2026-08-11, apps.siao.ai/answerhub serves nine rows, every one of them a test fixture: title 第一題, description 說明, body print('hi'), author AH-000001. The homepage renders nine cards with identical titles.

It is also invisible. apps.siao.ai/sitemap.xml returns 404. apps.siao.ai/robots.txt contains only Cloudflare's injected content-signals comment block — not one User-agent: line, no Allow:, no Sitemap:. The root layout declares lang="en" on an entirely Chinese product and titles every page apps.siao.ai. Two routes in the whole application export generateMetadata, and both return a bare title and description — no canonical, no Open Graph, no structured data.

Meanwhile the queries this product exists to answer are being served by someone else. netholiday.reh.tw — "網路假期 - 答案共享資料庫 2026" — is a live community: its question numbering has reached 8271, its homepage shows contributions from seventeen minutes ago, it ships a browser extension, and it already emits two JSON-LD blocks per page. Its titles follow "{query}" 解答 | 網路假期 - 答案共享資料庫. Students find it by pasting a whole question into Google.

The owner holds two corpora that could contest those results and neither is in the product:

• ~/Desktop/NetHo_DATA/netholiday_data.csv, crawled from netholiday.reh.tw in January 2026, converted to ~/Downloads/netholiday_data.lpans: 8523 rows, 6258 distinct questions once question text is normalised (NFKC plus whitespace removal — raw string comparison finds only 6752, so 493 duplicates differ solely in punctuation width or spacing). Question and answer only — options, explanation and image_url are empty on every questions once the corpus reader strips hidden characters — it was 6259 before that, and the extra one was two rows differing only in a zero-width space, which the de-duplication could not see through. Measured 2026-08-15 by scripts/hidden-character-survey.mjs; see SPEC_ANSWERHUB_ANTIBOT.md for why the reader strips at all. Average question 90 characters, average answer 106; the extremes are a 36,864-character question and a 139,999-character answer, and some fields hold a single character.
• ~/Desktop/HappyRead-Answer-Database, crawled from happyread.kh.edu.tw/readerquiz/exam_sheet.php?id={book_id} — a Kaohsiung City Department of Education reading-certification system: 7308 distinct questions across 184 books, each with four options. Note that all_questions.csv in that directory is an incomplete aggregate (4838 rows) and is not the corpus; the 184 book_*.csv files are, holding 15,765 rows that reduce to 7308 distinct questions — the crawler re-drew each book's randomised quiz repeatedly, so the raw files carry roughly 3.5× duplication. This corpus has no answers: the answer column is empty on 4825 of the aggregate's 4838 rows and the per-book files carry no answer column at all. The crawler printed each book's page title but never stored it, so books exist only as numeric ids.

A third source, supplied by the owner, partially closes both gaps in the reading corpus, and has already been harvested — sites.google.com/view/love-books-hacker publishes the correct answer's option number and text per question, and names each book in its URL path. Its numbering is not the source system's: our book_135 (紅瓦房) is that site's 125紅瓦房, while low ids such as 4 and 5 do coincide, so books are identified by content fingerprint rather than by number.

Measured on the completed harvest of all 61 pages: 54 books matched (the winner scored 3–27 against a runner-up of 0 in every case; the seven misses are books this corpus does not contain, plus one duplicate page for a book already claimed), yielding 753 answers across the 1843 questions those books hold. Because the site prints the correct option number alongside the text, and identification reads only the text, the number is an independent witness: 684 answers agree with it, 69 carry no legible number, and 30 disagreed and were dropped rather than published.

The owner's happyread.kh.edu.tw account is locked pending a school transfer, so the source system cannot currently be re-crawled for the remaining 130 books' titles.

Publishing either corpus as-is would republish a competitor's database verbatim, which is what Google's spam policies call scraped content, and would give a searcher no reason to prefer the copy over the original.

Solution

Publish roughly 13,566 question pages — 6258 from the holiday-homework corpus, 7308 from the reading-certification corpus — where the question text is verbatim (that is what a pasted-question search matches against) but the page around it is substantially the product's own work: worked reasoning, the knowledge point being tested, a stated verification state, taxonomy breadcrumbs, and links to sibling questions.

Answers are not taken on trust. Every question with a source answer is independently re-answered by a model that has not been shown that answer, and the two are compared. Agreement publishes as VERIFIED; disagreement publishes as DISPUTED, showing both readings side by side and making no single claim. The 753 reading-certification questions the third source answered are eligible for VERIFIED/DISPUTED like any other; the remaining 6555 have nothing to compare against and publish as UNVERIFIED, saying so. Note the shape of that number — the third source names 54 of 184 books but answers only about a tenth of the reading corpus's questions, so book titles are the larger thing it bought us, not answers. A page's verification state, not the presence of an answer, decides whether the page leads with a direct answer block — which is the whole difference between this product and the site it intends to replace.

Around the corpus, build the discovery surface that does not exist: a canonical-path-and-metadata module every page reads from, a generated sitemap, a real robots.txt that admits search engines and AI retrieval crawlers while reserving training rights, a nested taxonomy of index pages (school level → subject for the homework corpus; source → book title for the reading corpus) giving both internal linking and a landing page for mid-tail Chinese queries, and QAPage structured data on every question page.

User Stories

Searching students

1. As a student who pasted a whole homework question into Google, I want the AnswerHub page for that exact question to appear, so that I do not have to open five tabs to find one answer.
2. As a student, I want the page to show the question exactly as it was worded in my assignment, so that I can be sure it is the same question and not a similar one.
3. As a student, I want the answer visible immediately without signing in, clicking to expand, or hitting a quota, so that I get what I came for.
4. As a student, I want to see why the answer is the answer, so that I can write it up in my own words instead of copying a bare result.
5. As a student, I want to know whether an answer has been checked, so that I can decide how much to trust it before I hand the work in.
6. As a student looking at a disputed answer, I want to see both candidate answers and the reasoning for each, so that I can judge which is right rather than being handed a coin flip presented as fact.
7. As a student looking at an unverified reading-certification answer, I want the page to say plainly that no second source confirmed it, so that I am not misled by a confident-looking page.
8. As a student, I want to see other questions from the same book or the same subject, so that I can answer the rest of my assignment in the same visit.
9. As a student on a phone, I want the answer readable without zooming or waiting on JavaScript, so that I can check it between classes.
10. As a student searching for "國中自然 網路假期 解答", I want to land on a subject index rather than a random single question, so that I can browse to the one I need.
11. As a student searching for a book title plus "閱讀認證 解答", I want the page for that book listing all of its questions, so that I can work through the whole certification quiz.
12. As a student who spots a wrong answer, I want an obvious way to report it, so that the next person does not get the same bad answer.

Search engines and AI assistants

1. As Googlebot, I want a sitemap listing every indexable page, so that I can discover 11,590 pages that are not linked from any external site.
2. As Googlebot, I want each question page to declare a canonical URL, so that a question reachable through more than one taxonomy path is not counted as duplicate content.
3. As Googlebot, I want the site to declare lang="zh-Hant", so that I serve it to the Traditional Chinese audience it is written for.
4. As Googlebot, I want internal-search result pages marked noindex, follow, so that I do not spend crawl budget on unbounded query parameters.
5. As Googlebot, I want taxonomy index pages that link to their children and up to their parents, so that link equity reaches deep question pages instead of leaving them orphaned.
6. As Googlebot, I want QAPage structured data naming the question, the answer and the answer's author, so that I can understand the page without inferring its shape from markup.
7. As Googlebot, I want breadcrumb structured data, so that the taxonomy path is shown in results instead of a raw URL.
8. As GPTBot / OAI-SearchBot / ClaudeBot / PerplexityBot / Google-Extended, I want robots.txt to permit retrieval, so that I can cite this site when answering a student's question.
9. As an AI crawler, I want the content-signal ai-train=no stated explicitly, so that the operator's reservation of training rights is unambiguous rather than merely unstated.
10. As an AI assistant, I want the answer and its verification state in the initial HTML rather than behind interaction, so that I can quote it accurately.
11. As an AI assistant, I want disputed answers labelled as disputed in the markup, so that I do not present a contested answer as settled.

The operator

1. As the operator, I want the whole enrichment pipeline to emit .lpans files, so that content enters the product through the one import door that already exists and is already tested.
2. As the operator, I want re-running an import to update rows rather than duplicate them, so that incremental syncing with the upstream site does not multiply the corpus.
3. As the operator, I want the 1771 duplicate rows collapsed on a stable source key, so that one question does not compete against itself for the same query.
4. As the operator, I want a question's source site and original question number recorded, so that I can re-sync, attribute, and audit where an answer came from.
5. As the operator, I want the model's independent answer stored alongside the source answer, so that a disagreement can be reviewed later without re-running the pipeline.
6. As the operator, I want reading-certification book titles recovered from the third source by content fingerprint rather than by its page numbering, so that a book is named correctly rather than confidently mislabelled — its 125紅瓦房 is our book_135. 29a. As the operator, I want books whose titles could not be recovered to ship numbered and noindex, so that 130 pages titled "Book 87" do not dilute the taxonomy. 29b. As the operator, I want the answers the third source publishes attached to the questions they belong to, so that a third of the reading corpus can be cross-verified instead of published on the model's word alone.
7. As the operator, I want the nine test-fixture rows removed before launch, so that the first thing a crawler sees is not print('hi').
8. As the operator, I want slugs shortened before the corpus is loaded, so that I am not issuing eleven thousand redirects later.
9. As the operator, I want a structural test that fails when a new page ships without going through the SEO seam, so that metadata coverage cannot rot the way visibility filtering did.
10. As the operator, I want the Google Search Console verification token configurable, so that I can verify the property without a code change.
11. As the operator, I want to measure indexed-versus-submitted pages, sampled long-tail rankings, and sampled AI citations monthly, so that "are we winning" has an answer that is not a feeling.
12. As the operator, I want the import to be resumable and batched, so that an 11,590-question load is not one transaction that fails at row 9000.

Contributors

1. As a signed-in contributor, I want to submit a corrected answer to an existing question, so that the corpus improves without the operator re-crawling.
2. As a contributor, I want my correction to move a page from UNVERIFIED toward VERIFIED when it agrees with the model's independent answer, so that my effort visibly improves the page.
3. As an administrator, I want to take down a question that should not be published, so that the existing moderation path covers imported content too.
4. As an administrator, I want taken-down questions dropped from the sitemap and from every taxonomy index, so that search engines are not pointed at pages that will 404.

Implementation Decisions

One new seam: the SEO module

Every canonical path, every Metadata object, and every JSON-LD document in AnswerHub is produced by a single new module. Pages do not assemble metadata themselves; sitemap and robots route handlers read their URLs from the same module, so a page's canonical URL and its sitemap entry cannot disagree. This is deliberately the only new seam in this spec — the visibility, import and slug seams already exist and are reused as-is.

The module exposes, for a content item or taxonomy node: its canonical path, its Metadata (title, description, canonical, Open Graph), and its structured data. It knows nothing about Prisma; callers pass it the already-loaded item, in the same style as display-name.

Structured data

QAPage on question pages, with Question and acceptedAnswer — except that a DISPUTED page emits suggestedAnswer entries and no acceptedAnswer, because the markup must not assert what the page declines to assert. BreadcrumbList on question and taxonomy pages. CollectionPage on taxonomy index pages. Deliberately not FAQPage (Google restricted its rich result to authoritative government and health sites in August 2023) and not Quiz.

Titles and canonical shape

Question pages lead with the question: the first ~40 characters of the verbatim question text, then a suffix. Taxonomy index pages lead with the mid-tail keyword ("國中自然 網路假期解答", "《書名》閱讀認證題庫解答"). Meta descriptions are derived from the answer summary, never from the generic 說明 placeholder.

URLs

slugify currently uses the entire title plus a six-character UUID fragment. On a 90-character Chinese question that produces an 800-plus-character percent-encoded URL. It changes to a bounded slug — a short prefix of the question plus a stable discriminator — before any corpus is loaded, while the only affected rows are nine test fixtures.

Question pages live under their taxonomy path, and that path is the canonical one. A question reachable from more than one node canonicalises to a single path.

Taxonomy

No new model. Category already supports nesting through parentId/children, and that tree carries both corpora:

• holiday homework: school level → subject
• reading certification: source → book title

The enrichment pipeline assigns each question a leaf category. Book titles were not stored by the crawler and the source system cannot currently be re-crawled, so they were recovered from the third source instead — 54 of 184 books are now named. The remaining 130 keep a numeric label and a noindex index page until the source system is reachable again: a book index page titled "Book 87" is not a page anyone searches for, and shipping 130 of them would dilute the taxonomy they are meant to build.

The fingerprint match is its own testable step, not a script side effect: for each source page, score every local book by how many of its questions have exactly one option appearing on that page, and accept the winner only when it clears the runner-up decisively. Where the runner-up scores zero — which was every successful match in the harvest — three agreeing questions suffice, because 183 other books had the same chance and took none of it. Matching on the number in the source path instead would have mislabelled most of the corpus.

Schema changes

Submission gains:

• verification — enum VERIFIED | DISPUTED | UNVERIFIED, defaulting to UNVERIFIED. Existing manual submissions are unaffected in meaning: nobody checked them either.
• answerText — the published answer, separate from the free-form body.
• explanation — the model-written working. .lpans already carries an explanation field that had nowhere to land.
• sourceAnswer — the answer as it arrived from the source, retained even when it loses a disagreement, so a dispute can be reviewed without re-running the pipeline.
• modelAnswer — the independent answer, retained for the same reason.
• sourceSite and sourceRef — where the question came from and its original identifier.
• sourceKey — a unique key derived from source site plus normalised question text, where normalisation is NFKC plus whitespace removal. This is what makes import idempotent and what collapses the duplicates, and the normalisation is load-bearing rather than cosmetic: raw string comparison leaves 493 holiday-homework duplicates standing, and the reading corpus's raw files carry roughly 3.5× duplication. It is a unique index, so a second import of the same question updates rather than inserts.

HANDOFF.md records that the container runs prisma db push on boot, so schema and code must deploy together and a code-only rollback will not roll the columns back. Take a database backup before deploying, as was done for hiddenByAdmin.

The enrichment pipeline

An offline pipeline, not a request path. It reads a corpus, and for each question:

1. Normalises the question text and derives sourceKey; duplicates collapse here.
2. Asks a model to answer the question without being shown the source answer, and to state the knowledge point and school level/subject.
3. Compares that answer with the source answer where one exists. Agreement → VERIFIED. Disagreement → DISPUTED, both answers kept. No source answer → UNVERIFIED.
4. Asks a model to write the working for the published answer.
5. Emits .lpans.

The comparison is a decision made from two independent answers, so it must be its own testable function rather than logic buried in a script. Its known limit is stated here so nobody rediscovers it: two runs of the same model on a book-plot question can agree and both be wrong, which is exactly why the reading corpus publishes as UNVERIFIED rather than borrowing the VERIFIED label from a comparison it never had.

Comparison needs three stages, not one, and the pilot is what proved it. The first fifty questions came back 30 disputed; reading them showed almost none were disagreements. Two causes, both structural:

• The holiday-homework corpus lost its options. The crawler kept question and answer, so a source answer is frequently the label of an option nobody preserved — 以上皆是。, or literally 2 — while a model answering freely produces a sentence. These can never match by string.
• Even where both sides say the same thing, they decorate it differently: 內門 against 內門區, 約翰湯姆生 against 約翰·湯姆生（John Thomson）.

So: the prompt asks for an answer rather than an explanation; string comparison sees through decoration, guarded so that a one-character answer inside a longer one can never count (是 inside 不是 is the inversion this must never wave through); and whatever still disagrees is asked plainly whether the two answers mean the same thing. That last stage is weaker evidence — the same model judging its own work — so the journal records it separately even though the page does not distinguish it. Result on the sample: 30 verified, 15 disputed, 5 unverified.

Not everything in the corpus can become a page. 8.6% of the holiday-homework questions are a bare ImageData.action?… link where the question should be, because the source rendered the question as a picture. At full size that is roughly 540 pages with no question, no checkable answer and no query that would ever reach them. They are dropped at import, and the count is reported rather than silently absorbed.

The source corpus contains vandalism. 自己想笨蛋, 我女朋友姓陳. These land in DISPUTED with both readings shown, which is the product demonstrating its own value rather than a case to suppress.

.lpans currently caps at MAX_QUESTIONS = 500 per file. The pipeline emits batched files under that cap rather than raising it — the cap bounds a request body, and an 11,590-question upload should not be one request. The import is resumable: re-running a batch is a no-op for rows already present, by sourceKey.

Rendering the verification state

• VERIFIED — answer block at the top of the page, working below, acceptedAnswer in JSON-LD.
• DISPUTED — both candidate answers presented as alternatives with their reasoning, no single answer block, suggestedAnswer in JSON-LD.
• UNVERIFIED — answer shown with an explicit statement that no second source confirmed it, suggestedAnswer in JSON-LD.

The label is rendered from the stored state, never inferred at the template, so a page cannot present a state the data does not hold.

robots and crawler policy

A real robots.txt replacing the comment-only file: User-agent: * with Allow: /, an explicit Sitemap: line, Disallow for the authenticated and administrative surfaces, and content signals search=yes, ai-input=yes, ai-train=no. GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot, CCBot and Google-Extended are named and allowed for retrieval.

/answerhub/search emits noindex, follow.

Because the site remains a subpath of apps.siao.ai, robots.txt and sitemap.xml are shared with the nav homepage and must cover both.

Sitemap

Generated from the database through the SEO module, honouring the visibility seam, and split into an index plus child sitemaps — 11,590 URLs plus taxonomy pages exceeds what belongs in one file. lastmod comes from the submission's updatedAt.

Site-wide fixes this depends on

Root layout lang becomes zh-Hant; the default title and description stop describing the site as an index of tools; a Search Console verification token is read from configuration; the nine test-fixture rows are deleted.

Not view counting, despite an earlier draft of this spec saying so: SPEC_ANSWERHUB_CRAFT.md recorded that nothing wrote to views, and that has since been fixed — the detail page increments it and two end-to-end tests cover it, including the case where a taken-down answer must not count. Verified by running the suite on 2026-08-11. This is the second stale claim inherited from an index rather than measured; the first was the corpus size.

Visibility

No change to the visibility seam. Every new taxonomy index page and the sitemap generator read through PUBLIC_SUBMISSION_FILTER, and seam-coverage.test.ts fails them if they do not. Taken-down questions therefore disappear from indexes and sitemap without special handling.

Testing Decisions

A good test here asserts what a visitor, a crawler or an importer can observe: the bytes of robots.txt, the URLs in the sitemap, whether a question page contains its question verbatim, whether a disputed page asserts an answer. It does not assert that a particular helper was called or that metadata was assembled in a particular order.

Unit tests (vitest, beside the module, matching the existing lib/answerhub/*.test.ts convention):

• The SEO module: canonical path for each content and taxonomy kind; metadata for a question, a taxonomy node and the home page; JSON-LD shape per verification state — in particular that DISPUTED emits no acceptedAnswer.
• slugify: bounded length, stability, collision behaviour, and that CJK input does not produce a pathological URL. Prior art: the module has no test today; display-name.test.ts is the closest shape.
• Source-key derivation: two rows differing only in whitespace or full-width punctuation collapse to one key; different questions do not.
• Book fingerprint matching: a page whose answers come from a known book matches that book; a page with no counterpart matches nothing rather than matching the least-bad candidate; and the numeric prefix in a source path is never used as the identifier. The 紅瓦房 pairing (local 135, source 125, 48 hits against 0 for controls) is the regression case.
• The agreement comparison: agreement, disagreement, and absent source answer each produce the right verification state.
• .lpans extension: files carrying the new fields parse; files without them still parse, because exports from the old application must keep working (lpans.test.ts is prior art and states this requirement).

Structural tests, in the style of seam-coverage.test.ts and for the same reason — that spec's own history is that behavioural tests all passed while a page skipped the seam entirely:

• Every file under app/answerhub exporting metadata or generateMetadata must consult the SEO seam or be named in an exemption list with a reason.
• Every indexable route must be reachable from the sitemap generator.

End-to-end tests (Playwright, tests/, prior art homepage.spec.ts, search.spec.ts, submissions.spec.ts):

• robots.txt responds with real directives and a sitemap line.
• sitemap.xml responds, is valid XML, and contains a seeded question's canonical URL; a taken-down question's URL is absent.
• A question page contains the question text verbatim, a canonical link, QAPage JSON-LD, and breadcrumbs.
• A DISPUTED page shows both answers and no single answer block.
• An UNVERIFIED page shows the unverified statement.
• The search page carries noindex.
• A taxonomy index page lists its questions and links to its parent.
• Fixtures must include at least one question per verification state.

Accessibility and visual suites already exist (accessibility.spec.ts, visual.spec.ts); new page types are added to them rather than given parallel suites.

Out of Scope

• Moving to a dedicated subdomain or a separate domain. Decided against for this round; the site stays at apps.siao.ai/answerhub.
• Search Console verification submission and data export. The agent cannot authenticate to Google; it prepares the token and the sitemap URL, and the owner performs the submission.
• Re-crawling happyread.kh.edu.tw for the 130 unnamed books' titles. The owner's account is locked pending a school transfer; those books ship numbered, UNVERIFIED, with noindex index pages, and are backfilled when access returns.
• Any answer source beyond the one the owner supplied. Harvesting further answer-sharing sites is not part of this round.
• Data cleaning beyond deduplication. The 36,864-character question, the 139,999-character answer and the single-character fields are knowingly published this round; deduplication is in scope because it is not cleanup, it is preventing one question from competing against itself.
• Incremental re-sync with the upstream site. The pipeline is built to be idempotent so this is cheap later, but no scheduled sync ships here.
• Contributor-facing correction workflow beyond what already exists. Stories 37–38 describe the intended direction; this round ships reporting through existing comments and moderation.
• Algolia. search.ts remains the unwired pure request builder that SPEC_LWOPAN_MIGRATION.md left it as.
• Browser extension, advertising, sponsorship, and any other monetisation.
• Blog and knowledge-base SEO. Those surfaces stay cut from nav per SPEC_ANSWERHUB_FRONTEND.md.
• The /answerhub route prefix rename, still deferred.

Further Notes

The timing. Written 2026-08-11. Taiwanese holiday-homework answer queries peak in the last days of August and again in early February. The owner's decision is to publish the full corpus for the August peak. It is recorded here that Google will not crawl 11,590 new pages in three weeks, so which few hundred pages get indexed for that peak is decided by Google rather than by us; the February window is where the corpus should be fully earning.

Three risks the owner has accepted after they were raised. Publishing in full rather than in a verified-first batch; publishing over a thousand possibly-wrong answers behind a disputed label; and publishing the reading-corpus questions no second source confirms — roughly two thirds of 7308, since the third source answered only 753 of them. Each is a deliberate decision, not an oversight — recorded so that a later reader does not "fix" them by accident.

Every corpus count in this document was measured, and the first set was wrong. An earlier draft cited 4838 reading questions and 6752 holiday-homework questions, both taken from the most obvious file rather than the real one: all_questions.csv is a partial aggregate, and raw string equality under-counts duplicates in Chinese text. Anyone revising these numbers should re-measure from the 184 book_*.csv files with NFKC normalisation rather than trusting either figure.

Where the differentiation actually lives. Not in the question text, which is verbatim and identical to the competitor's. It lives in the working, the taxonomy, and the verification state. If the enrichment is thin, this spec produces a copy of netholiday.reh.tw with worse authority, and the scraped-content exposure becomes real. The quality of the enrichment is the load-bearing element.

The reading corpus's provenance is different from the homework corpus's. It came from happyread.kh.edu.tw, a city education department system, and it carries no answers — so what gets published there is the product's own answer to someone else's question, which is a materially stronger position than republishing someone else's answer.