Skip to main content

What is LynxDB?

LynxDB is an open-source log analytics database built from scratch in Go. A single static binary that works as a pipe-mode CLI tool (like grep meets awk), a standalone server, or a distributed cluster -- same binary, same query language, same API at every scale.

The Problem

The log analytics market is broken:

  • On one end: grep and jq -- fast, scriptable, but no persistence, no indexing, no aggregation across time ranges.
  • On the other end: Splunk ($2,000+/GB/day), Elasticsearch (cluster of 6+ nodes), Datadog (opaque pricing and vendor lock-in).

There is nothing in between. LynxDB fills the gap.

Three Modes, One Binary

Developer laptop  →  cat app.log | lynxdb query '| stats count by level'
Single server → lynxdb server --data-dir /var/lib/lynxdb
3-node HA → lynxdb server --cluster.seeds node1:9400,node2:9400
1000-node fleet → lynxdb server --cluster.role query --cluster.seeds meta1:9400

Pipe Mode (No Server)

Query local files and stdin using the full SPL2 engine with zero network overhead:

cat /var/log/syslog | lynxdb query '| where level="ERROR" | stats count by service'
kubectl logs deploy/api | lynxdb query '| stats avg(duration_ms), p99(duration_ms) by endpoint'

No daemon, no config file, no data directory. The binary creates an ephemeral in-memory engine, ingests input, runs the SPL2 pipeline, prints results, and exits.

Server Mode (Single Node)

Persistent storage with a full REST API, materialized views, alerts, and dashboards:

lynxdb server
lynxdb query 'source=nginx status>=500 | stats count by uri | sort -count | head 10'

Cluster Mode (Distributed)

Scale to 1000+ nodes with S3-backed shared storage, Raft consensus, and distributed query execution:

lynxdb server --cluster.seeds node1:9400,node2:9400,node3:9400

How LynxDB Compares

LynxDBSplunkElasticsearchLokiClickHouse
DeploymentSingle binaryStandalone or distributedSingle node or clusterSingle binary or microservicesSingle binary or cluster
DependenciesNone-Bundled JVMObject storage (prod)Keeper (for replication)
Query languageSPL2SPLLucene DSL / ES|QLLogQLSQL
Pipe modeYesNoNoNoYes (clickhouse-local)
SchemaOn-readOn-readOn-writeLabels + lineOn-write
Full-text indexFST + roaring bitmapstsidxLuceneLabel index onlyToken bloom filter
Memory (idle)~50 MB~12 GB (min spec)~1 GB+~256 MB~1 GB
LicenseApache 2.0CommercialELv2 / AGPLAGPLApache 2.0

Key Features

  • SPL2 Query Language -- Splunk-inspired, works everywhere (CLI, API, alerts, dashboards)
  • Columnar Storage -- Custom .lsg segment format with delta-varint timestamps, dictionary encoding, LZ4 compression
  • Full-Text Search -- FST-based inverted index with roaring bitmap posting lists and bloom filters
  • Zero-Allocation VM -- 22ns/op bytecode evaluation, 2.1M events/sec pipeline throughput
  • Materialized Views -- Precomputed aggregations with ~400x query acceleration
  • Schema-on-Read -- No upfront schema, fields discovered and indexed automatically
  • Drop-in Compatibility -- Elasticsearch _bulk, OpenTelemetry OTLP, Splunk HEC
  • Alerts & Dashboards -- SPL2-powered alerting with 8 notification channels

Next Steps