Skip to content

How this site is built

This site is built to show an approach to AI delivery: decide by measurement, keep the model on a short lead, publish the failures. It was built with an AI coding agent under Karthick's direction. Every figure below was measured on this system on 13 November 2026 unless it says otherwise.

What happens when you paste a job description

  1. Clean. Control characters are removed and length is capped. Your text is treated as data, never as instructions.
  2. Extract. A small, fast model lists the requirements and the hard constraints. Each must quote your text exactly; anything that does not is dropped.
  3. Read the whole profile. The assessor is given all67 approved evidence records, plus a capability map that groups them by what Karthick can do. No search step decides what it sees. Those records are the only source of claims about him.
  4. Map. A stronger model works out what each requirement asks for, reads the records that bear on it together, labels it demonstrated, partial, not demonstrated or unknown, and cites the records. Requirements are mapped in parallel batches to cut waiting time.
  5. Check. Code, not a model, removes citations that are not in the approved catalogue, downgrades any supported claim left without a citation, and replaces any explanation containing a figure that is not in its sources.
  6. Score. The percentage is plain arithmetic in server code: essential requirements weigh 3, unspecified 2, desirable 1; demonstrated counts 1, partial 0.5. The result is capped at 95%: no profile is shown as a perfect match. The model never sees or sets the total.
  7. Sign. The result travels in a signed envelope, so the Q&A explains the same score you see and a tampered score is rejected.

Decisions made by measurement

Which model maps requirements to evidence

Thirty requirement rows with an unambiguous right answer were labelled, and four models were run on the same six cases. The labels were drafted with AI assistance and are pending owner review.

ModelRows correctMedian timeCost per assessment
anthropic/claude-haiku-5.521 / 309.7 sUS$0.004
openai/gpt-6.1-terra26 / 3011.0 sUS$0.018
anthropic/claude-sonnet-5.529 / 3010.8 sUS$0.052
google/gemini-4-hydrogen30 / 3017.2 sUS$0.053

The cheapest model hedged: it called clear matches and clear misses "partial" alike. Claude Sonnet 5.5 was chosen for mapping. Gemini 4 Hydrogen scored one row higher but took about six seconds longer, and one row in thirty is within noise. The cheap model still does extraction and Q&A, where it is good enough.

Whether this site needs a vector database

With only 30 evidence records at the time of the test, everything fit in one prompt. So the vector search was tested against simply sending all records.

RetrievalRows correctCases in expected bandCost per assessment
Pinecone top 429 / 304 / 6US$0.052
Pinecone top 829 / 306 / 6US$0.061
No vector search (all 30 records)30 / 306 / 6US$0.083

Honest answer: at that size, no. Sending everything was at least as accurate. Retrieving only four records per requirement was worse, because relevant evidence was left out and strong matches were under-scored. Eight records per requirement matched the no-search accuracy at about a quarter less cost, so the fit check first shipped with search.

That decision was reversed after real use. The owner reported requirements scored "partial" although several records, read together, plainly covered them: a search per requirement returns the closest records, not the full picture, and results varied from run to run. The fit check now reads the whole approved profile (67 records) with a capability map, and its instructions were rewritten to judge each requirement at the level it asks for. Across benchmark runs since, between 18 of 21 and 20 of 22 cases fell in the expected band (16 of 20 before) and all but one or two labelled rows were right (29 of 30 before), with unrelated roles still scoring near zero. Results vary a little from run to run; the evaluation page shows the latest. It costs about twice as much per assessment. The vector index still serves the Q&A and the video guide, where a single question needs a few records quickly.

Single runs on six benchmark cases (01, 02, 03, 09, 14, 15). Small sample: differences of one or two rows are within noise. Extraction model was anthropic/claude-haiku-5.5 in every run.

What it costs and how fast it is

Guardrails

Options rejected

Known limits

Stack

Astro with a Svelte island and Tailwind CSS; TypeScript throughout; Zod for every contract; serverless functions on Netlify; models through OpenRouter; Pinecone for Q&A retrieval with hosted embeddings; no application database. Unit and browser tests, type checks, a secret scan and a production build run on every push.