Elasticsearch / ELK

Elasticsearch alone is an indexing and full-text search engine. Its real operational value reveals itself in the complete ELK stack: Elasticsearch (indexing and querying) + Logstash or Filebeat (collection and transformation) + Kibana (visualization, dashboards, alerting). For an SMB, two main use cases where Elasticsearch plays a different role: in application search, it serves as a fast read index alongside your transactional SQL database; in observability (logs, metrics, security events), it becomes the primary store fed by collection agents, with Kibana as the single interface for querying and visualization.

• →PostgreSQL full-text aloneFor simple search on a moderate catalog: no separate index needed as long as queries stay basicView page
• →Meilisearch / TypesenseModern lighter search engines, credible alternative for pure search needs without observability pipeline
• →AlgoliaSearch as SaaS, excellent UX: cost rises quickly with volume
• →OpenSearchOpen source fork of Elasticsearch under Apache license, defensible for new projects
• →Loki + GrafanaFor log centralization only: lighter than ELK, native Prometheus integration, but without Elasticsearch's search power

1. 01

   For application search: PostgreSQL as source of truth, Elasticsearch as read index fed by change-data-capture (Debezium) or application-level ingestion

2. 02

   For observability: Filebeat (collection), Logstash (transformation), Elasticsearch (primary store), Kibana (dashboards and alerting)

3. 03

   Minimal 3-node cluster in production: no single-node, high availability non-negotiable

4. 04

   Choice between Elasticsearch (SSPL) or OpenSearch (Apache 2.0) depending on your license sensitivity

5. 05

   Sized indices and mapping templates defined before massive ingestion: avoid mapping explosion

6. 06

   Automated index retention (ILM) to control storage, otherwise the bill skyrockets

• For application search: Elasticsearch or PostgreSQL full-text?

  Complementary in the vast majority of cases, not opposed. PostgreSQL stays source of truth with its transactional guarantees, constraints and relations. Elasticsearch comes as a read index, fed from Postgres, optimized for complex queries: advanced full-text search, facets, real-time aggregations. For simple search, keyword matching on a moderate catalog, PostgreSQL full-text largely suffices and avoids the overhead of a separate cluster. Moving to Elasticsearch is justified when your search needs exceed what SQL can do efficiently. Note that for observability (logs, SIEM, events), the question doesn't apply: Elasticsearch is itself the primary source.
• Why talk about the ELK stack and not just Elasticsearch?

  Because Elasticsearch alone is an engine without an interface or ingestion pipeline. It's useless in isolation. Operational value comes from the complete stack: Filebeat or Logstash to collect and transform data (logs, events, business data), Elasticsearch to index and query, Kibana to visualize, build dashboards and configure alerts. Deploying Elasticsearch without a pipeline or visualization is missing the essential.
• How much does a self-hosted ELK stack cost?

  For an SMB with a few servers to monitor or a medium catalog, expect an Elasticsearch cluster on 3 VPS with 8-16 GB RAM each, that is €80-200/month hosting. Initial setup (cluster + Filebeat/Logstash pipeline + Kibana + alerting) takes 4 to 10 days depending on complexity. Elastic Cloud (managed offering) starts around €100/month but climbs quickly with volume.
• Elasticsearch or OpenSearch?

  Since the Elasticsearch license change in 2021 (move to SSPL, non-OSI-open-source), OpenSearch is the fork maintained by Amazon with a pure Apache 2.0 license. Very wide compatibility but growing divergence. For new projects without strong dependency on recent Elastic features, OpenSearch is the defensible choice. For existing projects in production on Elasticsearch, migration isn't urgent.
• And ELK stack security?

  Critical. Many high-profile incidents involve Elasticsearch clusters exposed without authentication on the internet. Essential best practices: authentication enabled (free since 2020), TLS on all nodes, Kibana and admin access restricted to internal network or VPN, regular snapshot backups, index rotation. Properly deployed, the ELK stack is secure; poorly deployed, it's direct exposure of all your data and all your logs.