// 2026-02-04[ INSIGHTS ]7 min

Reading 80,000 employee survey responses in a weekend

How a small pipeline of classifiers, clustering, and LLM summarization turned a year-end HR survey into a 6-page brief leadership actually read.

Every year-end engagement survey ends the same way. The multiple-choice numbers land in a dashboard within a week. The free-text answers, the one place where employees actually said something, land in a spreadsheet tab that nobody opens.

We got handed one of those tabs: 80,000 free-text responses from a year-end HR survey. The ask was simple. Tell leadership what is in here, in a form they will read, before the planning cycle closes. We had a weekend of machine time and a few days of human time. This piece is the method we used, end to end.

Why nobody reads the free-text box

It is not laziness. Three things kill these datasets before anyone gets to the content.

Volume. At a reading pace of ten responses a minute, 80,000 responses is over 130 hours of undivided attention. No HR business partner has that, and splitting the pile across ten readers means ten inconsistent mental models of what the data says.

Duplication. A large fraction of any survey corpus is near-identical. Empty strings, "n/a", "no comment", the same sentence about parking copy-pasted by an entire floor. Raw row counts wildly overstate how much distinct signal exists, but you cannot know that until you deduplicate, and nobody deduplicates a spreadsheet by hand.

Dread. Free-text answers contain names, grievances, and occasionally allegations. Every person who opens the file becomes a person who has read the allegation about a specific manager and now has to decide what to do with it. Legal teams know this, which is why the file often stays closed on purpose.

The consequence is that organizations pay for the most honest data they will collect all year and then act only on the Likert scores.

The pipeline

Our design goal was a funnel: each stage reduces the volume the next stage has to handle, and the expensive components only ever see distilled input.

The funnel. Each stage cuts volume so the next stage stays cheap. The expensive model appears once, at stage 04, and only ever reads clustered excerpts.

Normalize and dedupe

Lowercase, strip boilerplate, drop empties and one-word answers, then collapse near-duplicates with MinHash over character shingles. This alone removed about 19% of the corpus, taking 80,000 rows down to 64,912. We keep the duplicate counts as weights, because 400 people pasting the same complaint about the shift-swap tool is itself a finding.

The cheap classifier pass

Every surviving row goes through three small-model calls: a sentiment label, a coarse topic tag from a fixed taxonomy of 14 buckets, and a PII scrub that replaces names, employee IDs, and team identifiers with typed placeholders. These are single-purpose prompts on the cheapest model tier, run in parallel batches overnight. Nothing downstream ever sees an unscrubbed row.

field note: the PII scrub runs before embedding, not after clustering. If a name gets into an embedding, it gets into a cluster centroid, and then a quote with that name in it is one careless copy-paste away from a board deck. Scrub at the mouth of the funnel, audit at the exit.

Embedding and clustering

We embed each scrubbed response and cluster with HDBSCAN, then merge clusters whose centroids sit closer than a tuned threshold. The topic tags from the classifier pass act as a sanity check here: a cluster that spans five topic buckets is usually two clusters that the density threshold glued together, and we split it. We landed on 47 themes, plus a noise bucket holding roughly 8% of responses that we sample manually.

Summaries with receipts

Now, and only now, the expensive model comes in. For each of the 47 themes it receives a stratified sample of member responses and writes a structured summary card. The contract is strict: every card must carry verbatim quotes, copied character for character from the source rows, with response IDs so a human can trace any quote back.

{
  "cluster_id": 31,
  "label": "on-call load falls on the same senior engineers",
  "n_responses": 412,
  "weighted_n": 517,
  "sentiment_split": { "negative": 0.71, "mixed": 0.22, "positive": 0.07 },
  "summary": "Platform and infra respondents describe an on-call
    rotation that has quietly shrunk to six or seven people.
    The complaint is not the pager itself; it is that opting
    out carries no visible cost.",
  "quotes": [
    { "id": "r_48211", "text": "I have been on call 14 of the last
      20 weekends. My manager knows. Nothing changes." },
    { "id": "r_09772", "text": "New joiners are 'not ready' for
      two years, apparently." }
  ],
  "confidence": "high",
  "flags": []
}

The flags field matters more than it looks. The model is instructed to flag any cluster containing possible legal or safety issues rather than summarize them. Those go to a named human, unsummarized, on day one.

The editor pass

A human editor spent two days turning 47 cards into 6 pages. They merged overlapping themes, cut anything the quotes could not support, and ordered the brief by what leadership could act on in the next two quarters rather than by cluster size. This pass is not optional. A machine-written 47-section document is just a smaller spreadsheet tab that nobody opens.

Quotes beat percentages

The first draft of the brief led every section with numbers: 71% negative sentiment on on-call, 517 weighted mentions. Leadership skimmed it politely. The revision led with the quotes and moved the numbers to a supporting line underneath. That version got forwarded, discussed in the exec staff meeting, and quoted back to us weeks later.

The reason is simple. Executives have seen a thousand percentages and have a discount rate for all of them. A verbatim sentence from an employee, with its specific detail intact, cannot be discounted the same way. "14 of the last 20 weekends" does work in a reader's head that "71% negative" never will. The percentages earn the quote its context; the quote earns the section its reader.

A theme without a verbatim quote attached did not make the brief. If we could not find a quote that carried the theme, we treated the theme as unproven.

How we validate the cluster labels

Clustering fails quietly. A label like "compensation concerns" can sit on top of a cluster that is actually about a bonus-timing bug in payroll software. So we audit before anything ships.

Sample audits. For every cluster, two reviewers each read 20 randomly sampled member responses and answer one question: does the label describe this response? Clusters scoring under 85% agreement get relabeled or split.
Inter-rater checks. The two reviewers rate independently and we compute agreement between them. Where the humans disagree with each other, the problem is usually an ambiguous label, and we rewrite the label until raters converge.
Quote traceback. Every quote in the final brief is matched by exact string search against the scrubbed corpus. Any quote the search cannot find is treated as a hallucination and removed, and its summary card goes back for rework. Out of 141 quotes in the draft cards, 3 failed traceback.

What it cost

This is a cheap-model workload, and the funnel shape is why. Roughly 97% of tokens flowed through the small classifier tier at stage 02, where per-token cost is a fraction of the frontier price. Embeddings are cheaper still. The expensive model wrote 47 summary cards from sampled excerpts, a few hundred thousand tokens in total. The whole machine run cost less than one hour of the consulting time it replaced, and the binding constraint was the two days of human editing, not compute.

What changed because the brief got read

The honest answer is that the pipeline changed nothing directly; the reading did. The on-call theme became a staffed rotation policy in the next planning cycle. The shift-swap tool complaint, invisible in the Likert data because it was concentrated in two facilities, got a fix scheduled within a month. And the survey itself changed: knowing the free-text would actually be read, the HR team cut four multiple-choice questions and added two open ones for the following year.

That last effect is the one we care about. A survey that gets read alters what people are willing to write in it. The pipeline is a weekend of work; the trust compounds annually.

If you have a corpus like this sitting in a spreadsheet tab, start small this week: dedupe it, run a sentiment and topic pass with a cheap model, and count what is actually in there. The funnel above is a weekend once the plumbing exists, and the first stage takes an afternoon.