# Elasticsearch: A Practical Guide to Search That Finds Things

A customer types "runing shoes" into your shop's search box. Your database runs `WHERE name LIKE '%runing shoes%'` and finds nothing, because nobody spelled it that way in the catalogue. The customer leaves.

What they wanted was obvious to a person: running shoes, with the best matches first, even with a typo. Doing that for millions of documents in a few milliseconds is a different problem from storing rows, and it is the problem Elasticsearch was built for.

## What Elasticsearch is

Elasticsearch is a distributed search and analytics engine built on the Apache Lucene library. You send it JSON documents over a REST API, it indexes them, and it answers searches and aggregations very quickly.

The core idea is the **inverted index**. Like the index at the back of a book, it maps each word to the documents that contain it. Finding every document with "running" is a lookup, not a scan.

Some vocabulary:

| Term | Meaning |
|---|---|
| Document | One JSON record, such as a product or a log line |
| Index | A collection of documents, roughly like a table |
| Mapping | The definition of each field and its type, roughly like a schema |
| Shard | A piece of an index. An index is split into shards so it can spread across servers |
| Replica | A copy of a shard on another node, for safety and extra read capacity |
| Node, cluster | One Elasticsearch server, and the group of servers working together |

Elasticsearch is made by Elastic. Its licensing has changed over the years: in 2021 it moved from the Apache licence to source-available licences, which led Amazon to create the **OpenSearch** fork, and in 2024 Elastic added the AGPL as an open-source option. The basics in this guide apply to both, though newer features differ.

The examples below use the request format of the Kibana Dev Tools console, which is the easiest place to try them. The same requests work with `curl`. Current versions switch security on by default, so you will need HTTPS and a username and password.

## Why teams choose it

### Search that understands text

When you index a text field, Elasticsearch runs it through an **analyzer**: it splits the text into words, lower-cases them and can reduce them to a root form, so "running" and "runs" both match "run". Results come back ranked by relevance, and fuzzy matching forgives typos.

### Fast aggregations

The same engine can count, bucket and average across huge numbers of documents: errors per minute, sales by category, the 95th percentile of response time. That is why it sits behind so many dashboards.

### It scales sideways

Add nodes and the cluster spreads shards across them. Lose a node and replicas take over.

### A full toolkit for logs

With Kibana for dashboards and shippers such as Beats or Logstash for collection, Elasticsearch is a well-trodden path for centralised logging and observability.

## Where Elasticsearch fits, and where it does not

**A strong choice for:**

- Search boxes on websites and in applications
- Log and event analysis
- Dashboards over large volumes of time-stamped data
- Filtering and faceting ("brand", "size", "price range") over catalogues

**Look elsewhere when:**

- **You need a system of record.** Elasticsearch has no multi-document transactions and no joins in the relational sense. Keep the truth in a database such as PostgreSQL and copy what needs searching into Elasticsearch.
- **You must read your write instantly.** Search is "near real time": a new document appears in search results after a refresh, which happens about once a second by default.
- **Your search needs are small.** PostgreSQL full-text search, or SQLite's FTS5, may be enough, and it is one less system to run.
- **You only store logs and rarely search them in depth.** A lighter system such as Grafana Loki can cost much less to operate.
- **The data changes constantly.** Every update rewrites the whole document internally. Heavy update workloads are a poor fit.

## Using it the right way

### Define the mapping yourself

If you index a document into an index that does not exist, Elasticsearch guesses the field types. The guess is often wasteful and sometimes wrong, and a field's type cannot be changed afterwards. Create the index with an explicit mapping:

```http
PUT /products
{
  "settings": { "number_of_shards": 1, "number_of_replicas": 1 },
  "mappings": {
    "dynamic": "strict",
    "properties": {
      "name":        { "type": "text" },
      "description": { "type": "text" },
      "brand":       { "type": "keyword" },
      "category":    { "type": "keyword" },
      "price":       { "type": "scaled_float", "scaling_factor": 100 },
      "in_stock":    { "type": "boolean" },
      "created_at":  { "type": "date" }
    }
  }
}
```

The most important distinction is between two string types:

- **`text`** is analyzed into words, for full-text search. You cannot sort or aggregate on it efficiently.
- **`keyword`** is stored as one exact value, for filters, sorting and aggregations. Use it for ids, status values, tags and brand names.

`"dynamic": "strict"` makes Elasticsearch reject documents with fields you did not define, instead of silently adding them.

### Index documents, in bulk when there are many

```http
PUT /products/_doc/1001
{
  "name": "Trail running shoes",
  "description": "Lightweight shoes with a firm grip for wet ground",
  "brand": "Northpeak",
  "category": "footwear",
  "price": 89.5,
  "in_stock": true,
  "created_at": "2026-03-01T10:15:00Z"
}
```

For anything more than a handful of documents, use the `_bulk` API. One request with a thousand documents is far cheaper than a thousand requests.

```http
POST /_bulk
{ "index": { "_index": "products", "_id": "1002" } }
{ "name": "Road running shoes", "brand": "Northpeak", "category": "footwear", "price": 74.0, "in_stock": true }
{ "index": { "_index": "products", "_id": "1003" } }
{ "name": "Running socks", "brand": "Loop", "category": "accessories", "price": 9.5, "in_stock": false }
```

### Search: score what matters, filter the rest

```http
GET /products/_search
{
  "query": {
    "bool": {
      "must": [
        { "match": { "name": { "query": "runing shoes", "fuzziness": "AUTO" } } }
      ],
      "filter": [
        { "term":  { "in_stock": true } },
        { "range": { "price": { "lte": 100 } } }
      ]
    }
  },
  "size": 10
}
```

Inside a `bool` query, clauses under `must` affect the relevance score. Clauses under `filter` are yes-or-no tests: they do not affect the score and Elasticsearch can cache them. Put exact conditions (status, date range, category) in `filter`. Use `match` for `text` fields and `term` for `keyword`, number and boolean fields.

### Aggregate

```http
GET /products/_search
{
  "size": 0,
  "query": { "term": { "in_stock": true } },
  "aggs": {
    "by_category": {
      "terms": { "field": "category" },
      "aggs": { "average_price": { "avg": { "field": "price" } } }
    }
  }
}
```

`"size": 0` says "no documents, just the numbers". The result is a count and an average price for each category.

### Plan for change with aliases

Because a field's type is fixed, changing a mapping means building a new index and copying the data across with the `_reindex` API. Make that painless from the start: have your application talk to an **alias**, a second name that points at the real index.

```http
POST /_aliases
{
  "actions": [
    { "add": { "index": "products_v1", "alias": "products_live" } }
  ]
}
```

When `products_v2` is ready, move the alias in a single request and the application never notices.

### Size shards sensibly

Each shard has a cost in memory and file handles. Thousands of tiny shards will hurt a cluster more than a few large ones. Elastic's general guidance is to aim for shards in the tens of gigabytes. Start with one primary shard for a small index and grow only when you have a reason. For logs, use data streams with index lifecycle management so old data is rolled over and deleted automatically.

### Page deep with `search_after`

`from` and `size` are fine for the first few pages, but by default you cannot go past 10,000 results that way, because every shard has to collect and sort all the earlier ones. For deep paging or exports, sort on a field with a unique tiebreaker and pass the last hit's sort values to `search_after`.

## Common mistakes

- **Using Elasticsearch as the only database.** Keep a source of truth elsewhere that you can rebuild the index from.
- **Accepting dynamic mappings.** Every string becomes both `text` and `keyword`, which doubles the work, and one odd document can lock in the wrong type.
- **Mapping explosion.** Indexing JSON with arbitrary keys (user-defined attributes, raw headers) creates thousands of fields and bloats the cluster. Use the `flattened` type for such objects.
- **Too many shards.** A daily index with five shards for a small log volume adds up quickly.
- **`term` queries on `text` fields.** The stored words were lower-cased and split, so an exact `term` for `"Trail running shoes"` matches nothing.
- **Leading wildcards.** A query such as `*shoes` must check every term. Model the search differently, for example with an n-gram analyzer.
- **Forcing a refresh after every write.** It makes documents visible immediately and indexing slow. Let the default refresh do its work.
- **An unsecured cluster on the internet.** Keep security on, and do not expose port 9200 publicly.

## Quick reference

| Task | Request |
|---|---|
| Cluster health | `GET /_cluster/health` |
| List indices with size | `GET /_cat/indices?v` |
| List nodes | `GET /_cat/nodes?v` |
| See a mapping | `GET /products/_mapping` |
| Count documents | `GET /products/_count` |
| Fetch one document | `GET /products/_doc/1001` |
| See how text is analyzed | `GET /products/_analyze` with `{"field": "name", "text": "Trail running shoes"}` |
| Why a document did or did not match | `GET /products/_explain/1001` with a query |
| Where shards live | `GET /_cat/shards?v` |

Elasticsearch gives its best results when you treat it as a specialist: explicit mappings, a database of record behind it, and filters kept apart from scored queries.

## Exploring Elasticsearch with CreamDB

Kibana is the full toolkit, and it is another service to run. For a quick look into a cluster, [CreamDB](https://www.creamdb.com) has an Elasticsearch explorer in the browser, with nothing to install.

- See the cluster's name, version and health.
- List indices with their health, document count and size, and filter them by name pattern.
- Open an index to read its mapping, settings and aliases.
- Search with Query DSL, including aggregations, and page through the hits.
- Ask in plain English, for example "errors per service in the last hour", and the assistant writes the query and shows the result.

The assistant reads on its own. Storing, updating or deleting a document waits as an approval card that you answer, and read-only mode on the server refuses every write.

CreamDB is a hosted service, so the cluster has to be reachable from the internet. Keep its security switched on and connect with a username and password or an API key.

It is free to start, and your SQL databases sit in the same tab.

