Installing Elasticsearch, loading your data, dropping a search box on the page — honestly, getting the engine running doesn't take that long. There are plenty of docs for it.

The real work starts right after. You run a search and the wrong thing shows up at the top. A product that's definitely in your data doesn't show up at all. One typo and you get zero results. This is usually the point where people start googling "relevance tuning" — but half of this problem was decided long before you ever touched Elasticsearch. At the design stage.

I started out in search doing maintenance on a commercial search solution. The thing that confused me most back then wasn't query syntax, it was "which analyzer do I put on which field?" One time I'd set an analyzer on a field, and a support ticket came in: why isn't this searchable? It took me a while to figure out. The analyzer was tokenizing the word differently than I expected, so what the user typed didn't match the indexed terms, and nothing came back. Obvious in hindsight, but as a newbie, "it's clearly in the data, so why won't it show up?" was a genuine mystery.

That's when it clicked: half the reason search doesn't work isn't the query, it's a design decision you made way earlier. This post is about that "way earlier." No code. Just four questions to ask yourself before you create an index, before you pick an analyzer.

1. What is the user actually searching for? (search intent)

The starting point isn't "what fields do I have," it's "what is the user trying to do." And users don't search the way you expect. Almost never.

When I worked on a shopping site, I deliberately disabled search by product number. Figured a customer would never search by product number. Then the tickets came in, and it turned out to be our own internal staff. Customers search by keyword, but staff type a product number to pull up a specific item and check it. Same search box, completely different intent depending on who's using it.

It gets more extreme when the domain changes. Look at search for court rulings or patents: people use exact-match search far more than throwing in a few keywords. Design it with your general e-commerce search instincts and you'll build the exact opposite of what they need.

So the first question is always the same: who is searching, and what for? Customers and internal staff are different. A shopping site and legal search are different. Build without knowing this and you'll tune for one group while breaking it for the other.

2. Define what "good results" even means

If you judge "is this search good or bad" by gut feel, you'll never fix it. You need a concrete standard. The easiest one is a golden query list: grab the 10–20 queries people search most, and for each, write down "this is what should rank #1." That's your test sheet. Every time you change something later, you check pass/fail against it. Tuning without it is like grading with no answer key.

But go one level deeper and you find that "good results" itself means different things in different domains.

For general document search, term similarity like BM25 is what matters — how closely the query matches the body text, highest score floats to the top. But product search, like travel or shopping, is different. It's less about how well the terms match and more about a score built from product features like sales numbers. You don't want "the thing most similar to the query," you want "the thing this person is likely to buy."

And product search has a variable the textbooks don't mention: people. A pushy vendor asks "why isn't our product at the top?" and you end up boosting it. This "business pressure vs. search quality" tug-of-war has enough material for its own post (coming later), but the thing to know at the design stage is this: what counts as a "good result" depends on your domain and your business, and you have to define it first.

3. Look your data straight in the eye

Eighty percent of search quality is decided in the data, before you index anything. Before you fire up Elasticsearch, take an honest look at it. Are the titles clean or messy? Korean, English, mixed? Is the formatting consistent? Any empty or untrustworthy fields?

I've seen abuse, too. A seller had stuffed a pile of keywords totally unrelated to the title into the product title through the CMS. So that product would catch on the wrong queries and float to the top. Eventually another product's manager came to complain: why does this thing catch on this query and rank above us? The algorithm was fine. But when the data is dirty, the search gets dirty right along with it.

Special characters are a pain too. In legal search, a query like 제주 4·3 사건 ("Jeju April 3 incident") has a special symbol in it — that middle dot. To make those searchable I had to do a fair amount of work at the ingestion stage. On top of that, there turned out to be several kinds of whitespace: it looks like the same space to your eye, but it's actually a different character. I had to replace all of it with a regular space and normalize it, before indexing.

Feed in dirty data as-is, and your search comes out exactly that dirty.

4. The gap between the user's words and the data's words

Here's the problem search is really solving: the words the user types ≠ the words written in your data.

One time someone searched for 캐리어 (a suitcase) and a 캐리어 냉장고 (a Carrier-brand refrigerator) floated to the top — that one gave me trouble, heh. The user means luggage, but in the data "캐리어" was sitting there as an appliance brand name. Same characters, completely different intent. The machine can't tell them apart and just matches "캐리어 = 캐리어."

This kind of gap shows up especially often, and especially nastily, in Korean: loanword spelling variations, homonyms, brand names that overlap with common nouns. So if you gather up "where in our domain do the user's words and the data's words diverge" at the design stage, it becomes clear where you'll later need to build bridges with synonyms, analyzers, or boosting. Miss this, and you're stuck repeating "why won't this search work?" forever.

So what's the output of this stage?

Not code. A short one-page memo:

• The main search intents (who's looking for what)
• Golden queries and their ideal answers, plus your standard for "good results"
• A list of data-quality issues
• The gaps between the user's words and the data's words

This one page drives everything that follows: index design, analyzers, relevance. Thirty minutes spent here saves you days of rework. What I learned the hard way taking support tickets as a newbie, you can just lay out on one page before you ever build the index.

Next post

With this memo in hand, we finally go into Elasticsearch for real. Next up: index design — how to set up your fields, and how to split text vs keyword. That "which analyzer goes on which field" problem that tormented me as a newbie? I'll dig into it properly there.

Tell me in the comments which stage trips you up the most, and I'll write that one first.