Every system that runs long enough grows a second life as data. Orders become revenue dashboards, clicks become billing, logs become fraud models. The work splits three ways: stream processing (results update as events arrive), batch processing (big jobs over all the data, on a schedule), and the warehouse (SQL over everything, for people). Stream processing, Time and windows and Batch processing teach the ideas; MapReduce shows where they came from.
This page is the map of what each cloud sells for those three jobs, and how they charge, because in analytics the pricing model shapes the design more than anywhere else. A query engine that charges per byte scanned rewards different table layouts from one that charges per hour of capacity.
Side by side
| AWS | Google Cloud | Azure | Open source or multi-cloud | |
|---|---|---|---|---|
| Stream processing | Managed Service for Apache Flink | Dataflow (Apache Beam) | Fabric Real-Time Intelligence, Azure Stream Analytics | Apache Flink, Kafka Streams |
| Batch with Spark | EMR, Glue | Managed Service for Apache Spark (formerly Dataproc) | Fabric Data Engineering, Azure Databricks | Apache Spark, Databricks |
| Warehouse | Redshift | BigQuery | Fabric Data Warehouse (Azure Synapse Analytics for existing workloads) | Snowflake, ClickHouse |
| SQL over files in object storage | Athena | BigQuery external and BigLake tables | Fabric's SQL analytics endpoint, over OneLake shortcuts to ADLS, S3 and Cloud Storage | Trino, DuckDB |
| How queries are billed | Athena: $5 per TB scanned. Redshift Serverless: RPU-hours | On-demand: $6.25 per TiB scanned, first 1 TiB a month free; or slot-hours | Fabric capacity units, shared by every workload | Your clusters, or the vendor's credits |
| Stream engine billed per | $0.11 per KPU-hour (1 vCPU, 4 GB) | Worker vCPU and memory, plus Streaming Engine units, per second | Capacity units | Your clusters |