Linux Tutorial: Logs to CSV to JSON

Source: Dev.to

Setup Directory

mkdir -p tutorial
cd tutorial

Generate Sample Logs

# Generate logs inside the tutorial folder
echo "2026-01-18 05:42:09 | INFO | system initialized" >> myapp.log
echo "2026-01-18 05:42:09 | ERROR | disk space low" >> myapp.log
echo "2026-01-18 05:42:09 | INFO | user logged in" >> myapp.log

File: myapp.log

2026-01-18 05:42:09 | INFO | system initialized
2026-01-18 05:42:09 | ERROR | disk space low
2026-01-18 05:42:09 | INFO | user logged in

Convert Logs to CSV

sed 's/ | /","/g; s/^/"/; s/$/"/' myapp.log > myapp.csv

File: myapp.csv

"2026-01-18 05:42:09","INFO","system initialized"
"2026-01-18 05:42:09","ERROR","disk space low"
"2026-01-18 05:42:09","INFO","user logged in"

CSV‑to‑JSON Python Script

Create csv_to_json.py and paste the following code:

import csv
import json
import sys
from pathlib import Path

def convert(csv_file, json_file):
    csv_path = Path(csv_file)
    json_path = Path(json_file)

    if not csv_path.exists():
        print(f"Error: Input file '{csv_file}' not found.")
        print("Make sure you are running this from the project root and have generated the CSV.")
        sys.exit(1)

    data = []
    with open(csv_path, mode='r', encoding='utf-8') as f:
        reader = csv.DictReader(f, fieldnames=["timestamp", "level", "message"])
        for row in reader:
            data.append(row)

    # Ensure parent directory for output exists
    json_path.parent.mkdir(parents=True, exist_ok=True)

    with open(json_path, mode='w', encoding='utf-8') as f:
        json.dump(data, f, indent=4)

    print(f"Successfully converted {csv_file} to {json_file}!")

if __name__ == "__main__":
    # Find files relative to the script's own directory
    script_dir = Path(__file__).parent
    csv_file = script_dir / 'myapp.csv'
    json_file = script_dir / 'myapp.json'

    convert(csv_file, json_file)

Run the Conversion

python3 csv_to_json.py

File: myapp.json

[
    {
        "timestamp": "2026-01-18 05:42:09",
        "level": "INFO",
        "message": "system initialized"
    },
    {
        "timestamp": "2026-01-18 05:42:09",
        "level": "ERROR",
        "message": "disk space low"
    },
    {
        "timestamp": "2026-01-18 05:42:09",
        "level": "INFO",
        "message": "user logged in"
    }
]

The resulting JSON file can be consumed by web dashboards, data‑analysis tools, or used to generate test and benchmark data for language‑model experiments.