feat: AI search and chat

[anareyna]

Adds the indexer and query API for AI search and chat. UI in the next PR.

What's added

• scripts/index-ai-search.ts (run via npm run index:ai-search) walks the sitemap, fetches each page as markdown from the existing /api/content/*.md endpoint, uploads to Cloudflare AI Search.

• .github/workflows/index-ai-search.yml runs the indexer automatically after every successful production deploy. No manual work once secrets are set.

• POST /api/ai-search proxies the question to Cloudflare AI Search and hides the API token. Streams the answer back by default. Pass ?format=json for a single { answer, sources } object that's easy to read in curl.

• KV hash cache so re-runs skip pages that haven't changed. Optional. Leave the KV namespace ID empty to fall back to "re-upload everything."

  

• Docs: new "Enable AI Search" section in docs/getting-started.md.

How to test

1. For easy testing use the Cloudflare Playground. No setup. Dashboard → AI → AI Search → head-start-site → Playground → Chat tab. Type a question, read the answer. Best way to check if the AI's answers actually make sense.

The Playground has two tabs. Use Chat to test what our endpoint does (question in, AI answer out, plus the sources it used). The Search tab returns raw matching content with no AI answer. Useful for debugging retrieval, not what we ship.

2. Curl the JSON mode against a preview deploy (once env vars are added, see below):

curl -s -X POST 'https://<preview>.head-start.pages.dev/api/ai-search?format=json' \
  -H 'Content-Type: application/json' \
  -d '{"query": "How do I add a new locale?"}'

Returns:

{
  "answer": "To add a new locale, ...",
  "sources": [
    { "url": "...", "title": "Internationalization (i18n)" }
  ]
}

The default response (no ?format=json) is a streaming SSE response. That's what the UI will consume in a next PR. Hard to read by eye, so use JSON mode for manual testing.

3. Run it locally. Setup in docs/getting-started.md. Reuse the head-start-site instance and AI_SEARCH_HASHES KV namespace already on the Voorhoede CF account (don't create duplicates).

What still needs to happen (not code)

• GitHub repo secrets for the auto-indexer workflow. Four values listed in the docs. ✅
• Cloudflare Pages env vars (prod + preview) for the live proxy. Three values listed in the docs. ✅

Until these are done, the auto-indexer and the live preview proxy won't work. Local + Playground work today.

What's NOT in this PR (by design)

• No UI. Next PR.
• No Vercel AI SDK abstraction. Will revisit when the UI exists.
• No multi-turn chat. Same backend can power it later.

Decision log

docs/decision-log/2026-05-20-ai-search-prototype.md covers the why.

Checklist

• Self-reviewed
• Easy to review (small, commented)
• Docs updated (docs/getting-started.md)
• Decision log entry added
• Notified a reviewer

[cloudflare-workers-and-pages]

Deploying head-start with    Cloudflare Pages

Latest commit:	ebb690d
Status:	✅  Deploy successful!
Preview URL:	https://1232ca60.head-start.pages.dev
Branch Preview URL:	https://feat-ai-search.head-start.pages.dev

View logs

[Copilot]

Pull request overview

Adds an AI Search prototype to Head Start by introducing (1) a post-deploy indexer that uploads site markdown into Cloudflare AI Search (with optional KV hash caching) and (2) a server-side /api/ai-search proxy to query AI Search without exposing the API token.

Changes:

• Add /api/ai-search POST proxy with SSE streaming by default and optional ?format=json response reshaping.
• Add scripts/index-ai-search.ts + GitHub Actions workflow to crawl the sitemap and (re)index pages in Cloudflare AI Search, optionally skipping unchanged pages via KV.
• Add parseFrontmatter() (with tests) to extract metadata from the markdown frontmatter for AI Search item metadata.

Reviewed changes

Copilot reviewed 11 out of 12 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/pages/api/ai-search.ts	Adds the AI Search query proxy endpoint (streaming + JSON mode).
src/lib/frontmatter.ts	Adds parseFrontmatter() to read back fields emitted by buildFrontmatter().
src/lib/frontmatter.test.ts	Adds unit tests covering parseFrontmatter() round-trips/escaping.
scripts/index-ai-search.ts	Adds the sitemap crawler + uploader indexer with optional KV hash cache and pruning.
package.json	Adds index:ai-search script entry.
env.d.ts	Adds typings for new AI Search-related env vars.
docs/getting-started.md	Documents setup steps for enabling AI Search and the indexer workflow.
docs/decision-log/2026-05-20-ai-search-prototype.md	Records the rationale/architecture for the prototype.
astro.config.ts	Declares new server-side secret env schema for the runtime proxy.
.github/workflows/node-ci.yml	Adds empty placeholders for new env vars in CI jobs.
.github/workflows/index-ai-search.yml	Adds post-deploy indexing workflow triggered by deployment_status.
.env.example	Documents new optional env vars for AI Search.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Updated (UTC)
❌ Deployment failed View logs	head-start-worker	ebb690d	Jul 22 2026, 07:50 AM

[Copilot]

Pull request overview

Copilot reviewed 31 out of 33 changed files in this pull request and generated 5 comments.