# ETL Pipeline Documentation

## 1. L1A (Raw Data Ingestion)
**Status**: ✅ Supports Incremental Update

This script ingests raw JSON files from `output_arena/` into `database/L1A/L1A.sqlite`.

### Usage
```bash
# Standard Run (Incremental)
# Only processes new files that are not yet in the database.
python ETL/L1A.py

# Force Refresh
# Reprocesses ALL files, overwriting existing records.
python ETL/L1A.py --force
```

L1B demoparser2 -> L1B.sqlite

L2 L1A.sqlite (+L1b.sqlite) -> L2.sqlite

L3 Deep Dive