What JSON Lines is and why it is everywhere
JSON Lines (also called JSONL, and nearly identical to NDJSON) stores one complete JSON value per line, separated by a newline. There is no enclosing array and no commas between records. That small change makes a big difference in practice:
- Streaming. A program can read, process and discard one line at a time, so a 20 GB file never has to fit in memory.
- Appending. Writers add a record by appending a line, without rewriting a closing bracket. That is why structured logs from tools such as Docker, Bunyan, Pino and many cloud logging exports use it.
- Parallel work. Files can be split on newlines and processed on many machines, as BigQuery, Spark and Hadoop do for newline-delimited JSON imports.
- Machine learning datasets. Fine-tuning files for OpenAI, Anthropic Bedrock and Hugging Face datasets are JSONL, with each line holding one training example such as a
messagesarray.