Streaming pipeline: Kafka to curated tables
Synthetic events stream into Kafka, land as raw JSON on S3, get cleaned and aggregated in stage and curated layers, then load into a warehouse table with data quality checks at each hop.
- Event generator
- Kafka
- S3 raw
- Stage
- Curated
- Postgres / Redshift