
A customer types "runing shoes" into your shop's search box. Your database runs WHERE name LIKE '%runing shoes%' and finds nothing, because nobody spelled it that way in the catalogue. The customer leaves.
What they wanted was obvious to a person: running shoes, with the best matches first, even with a typo. Doing that for millions of documents in a few milliseconds is a different problem from storing rows, and it is the problem Elasticsearch was built for.
What Elasticsearch is
Elasticsearch is a distributed search and analytics engine built on the Apache Lucene library. You send it JSON documents over a REST API, it indexes them, and it answers searches and aggregations very quickly.
The core idea is the inverted index. Like the index at the back of a book, it maps each word to the documents that contain it. Finding every document with "running" is a lookup, not a scan.
Some vocabulary:
| Term | Meaning |
|---|---|
| Document | One JSON record, such as a product or a log line |
| Index | A collection of documents, roughly like a table |
| Mapping | The definition of each field and its type, roughly like a schema |
| Shard | A piece of an index. An index is split into shards so it can spread across servers |
| Replica | A copy of a shard on another node, for safety and extra read capacity |
| Node, cluster | One Elasticsearch server, and the group of servers working together |
Elasticsearch is made by Elastic. Its licensing has changed over the years: in 2021 it moved from the Apache licence to source-available licences, which led Amazon to create the OpenSearch fork, and in 2024 Elastic added the AGPL as an open-source option. The basics in this guide apply to both, though newer features differ.
The examples below use the request format of the Kibana Dev Tools console, which is the easiest place to try them. The same requests work with curl. Current versions switch security on by default, so you will need HTTPS and a username and password.
Why teams choose it
Search that understands text
When you index a text field, Elasticsearch runs it through an analyzer: it splits the text into words, lower-cases them and can reduce them to a root form, so "running" and "runs" both match "run". Results come back ranked by relevance, and fuzzy matching forgives typos.
Fast aggregations
The same engine can count, bucket and average across huge numbers of documents: errors per minute, sales by category, the 95th percentile of response time. That is why it sits behind so many dashboards.
It scales sideways
Add nodes and the cluster spreads shards across them. Lose a node and replicas take over.
A full toolkit for logs
With Kibana for dashboards and shippers such as Beats or Logstash for collection, Elasticsearch is a well-trodden path for centralised logging and observability.
Where Elasticsearch fits, and where it does not
A strong choice for:
- Search boxes on websites and in applications
- Log and event analysis
- Dashboards over large volumes of time-stamped data
- Filtering and faceting ("brand", "size", "price range") over catalogues
Look elsewhere when:
- You need a system of record. Elasticsearch has no multi-document transactions and no joins in the relational sense. Keep the truth in a database such as PostgreSQL and copy what needs searching into Elasticsearch.
- You must read your write instantly. Search is "near real time": a new document appears in search results after a refresh, which happens about once a second by default.
- Your search needs are small. PostgreSQL full-text search, or SQLite's FTS5, may be enough, and it is one less system to run.
- You only store logs and rarely search them in depth. A lighter system such as Grafana Loki can cost much less to operate.
- The data changes constantly. Every update rewrites the whole document internally. Heavy update workloads are a poor fit.
Using it the right way
Define the mapping yourself
If you index a document into an index that does not exist, Elasticsearch guesses the field types. The guess is often wasteful and sometimes wrong, and a field's type cannot be changed afterwards. Create the index with an explicit mapping:
PUT /products
{
"settings": { "number_of_shards": 1, "number_of_replicas": 1 },
"mappings": {
"dynamic": "strict",
"properties": {
"name": { "type": "text" },
"description": { "type": "text" },
"brand": { "type": "keyword" },
"category": { "type": "keyword" },
"price": { "type": "scaled_float", "scaling_factor": 100 },
"in_stock": { "type": "boolean" },
"created_at": { "type": "date" }
}
}
}
The most important distinction is between two string types:
textis analyzed into words, for full-text search. You cannot sort or aggregate on it efficiently.keywordis stored as one exact value, for filters, sorting and aggregations. Use it for ids, status values, tags and brand names.
"dynamic": "strict" makes Elasticsearch reject documents with fields you did not define, instead of silently adding them.
Index documents, in bulk when there are many
PUT /products/_doc/1001
{
"name": "Trail running shoes",
"description": "Lightweight shoes with a firm grip for wet ground",
"brand": "Northpeak",
"category": "footwear",
"price": 89.5,
"in_stock": true,
"created_at": "2026-03-01T10:15:00Z"
}
For anything more than a handful of documents, use the _bulk API. One request with a thousand documents is far cheaper than a thousand requests.
POST /_bulk
{ "index": { "_index": "products", "_id": "1002" } }
{ "name": "Road running shoes", "brand": "Northpeak", "category": "footwear", "price": 74.0, "in_stock": true }
{ "index": { "_index": "products", "_id": "1003" } }
{ "name": "Running socks", "brand": "Loop", "category": "accessories", "price": 9.5, "in_stock": false }
Search: score what matters, filter the rest
GET /products/_search
{
"query": {
"bool": {
"must": [
{ "match": { "name": { "query": "runing shoes", "fuzziness": "AUTO" } } }
],
"filter": [
{ "term": { "in_stock": true } },
{ "range": { "price": { "lte": 100 } } }
]
}
},
"size": 10
}
Inside a bool query, clauses under must affect the relevance score. Clauses under filter are yes-or-no tests: they do not affect the score and Elasticsearch can cache them. Put exact conditions (status, date range, category) in filter. Use match for text fields and term for keyword, number and boolean fields.
Aggregate
GET /products/_search
{
"size": 0,
"query": { "term": { "in_stock": true } },
"aggs": {
"by_category": {
"terms": { "field": "category" },
"aggs": { "average_price": { "avg": { "field": "price" } } }
}
}
}
"size": 0 says "no documents, just the numbers". The result is a count and an average price for each category.
Plan for change with aliases
Because a field's type is fixed, changing a mapping means building a new index and copying the data across with the _reindex API. Make that painless from the start: have your application talk to an alias, a second name that points at the real index.
POST /_aliases
{
"actions": [
{ "add": { "index": "products_v1", "alias": "products_live" } }
]
}
When products_v2 is ready, move the alias in a single request and the application never notices.
Size shards sensibly
Each shard has a cost in memory and file handles. Thousands of tiny shards will hurt a cluster more than a few large ones. Elastic's general guidance is to aim for shards in the tens of gigabytes. Start with one primary shard for a small index and grow only when you have a reason. For logs, use data streams with index lifecycle management so old data is rolled over and deleted automatically.
Page deep with search_after
from and size are fine for the first few pages, but by default you cannot go past 10,000 results that way, because every shard has to collect and sort all the earlier ones. For deep paging or exports, sort on a field with a unique tiebreaker and pass the last hit's sort values to search_after.
Common mistakes
- Using Elasticsearch as the only database. Keep a source of truth elsewhere that you can rebuild the index from.
- Accepting dynamic mappings. Every string becomes both
textandkeyword, which doubles the work, and one odd document can lock in the wrong type. - Mapping explosion. Indexing JSON with arbitrary keys (user-defined attributes, raw headers) creates thousands of fields and bloats the cluster. Use the
flattenedtype for such objects. - Too many shards. A daily index with five shards for a small log volume adds up quickly.
termqueries ontextfields. The stored words were lower-cased and split, so an exacttermfor"Trail running shoes"matches nothing.- Leading wildcards. A query such as
*shoesmust check every term. Model the search differently, for example with an n-gram analyzer. - Forcing a refresh after every write. It makes documents visible immediately and indexing slow. Let the default refresh do its work.
- An unsecured cluster on the internet. Keep security on, and do not expose port 9200 publicly.
Quick reference
| Task | Request |
|---|---|
| Cluster health | GET /_cluster/health |
| List indices with size | GET /_cat/indices?v |
| List nodes | GET /_cat/nodes?v |
| See a mapping | GET /products/_mapping |
| Count documents | GET /products/_count |
| Fetch one document | GET /products/_doc/1001 |
| See how text is analyzed | GET /products/_analyze with {"field": "name", "text": "Trail running shoes"} |
| Why a document did or did not match | GET /products/_explain/1001 with a query |
| Where shards live | GET /_cat/shards?v |
Elasticsearch gives its best results when you treat it as a specialist: explicit mappings, a database of record behind it, and filters kept apart from scored queries.
Exploring Elasticsearch with CreamDB
Kibana is the full toolkit, and it is another service to run. For a quick look into a cluster, CreamDB has an Elasticsearch explorer in the browser, with nothing to install.
- See the cluster's name, version and health.
- List indices with their health, document count and size, and filter them by name pattern.
- Open an index to read its mapping, settings and aliases.
- Search with Query DSL, including aggregations, and page through the hits.
- Ask in plain English, for example "errors per service in the last hour", and the assistant writes the query and shows the result.
The assistant reads on its own. Storing, updating or deleting a document waits as an approval card that you answer, and read-only mode on the server refuses every write.
CreamDB is a hosted service, so the cluster has to be reachable from the internet. Keep its security switched on and connect with a username and password or an API key.
It is free to start, and your SQL databases sit in the same tab.
About the author
- CreamDB is a web-based database explorer with an AI assistant. Connect your databases, ask questions in plain English and get answers, tables and charts from any browser. Product news and tips here.




