Kinesis Data Streams vs Amazon Data Firehose
Use Amazon Data Firehose when the goal is to land streaming data in S3, Redshift, OpenSearch, Iceberg tables or an observability vendor with no consumer code: it buffers, optionally transforms and converts, and delivers. Use Kinesis Data Streams when you need custom real-time consumers, several readers, millisecond-level processing, ordering per key or replay.
Data Streams vs Firehose compared
| Factor | Kinesis Data Streams | Amazon Data Firehose |
|---|---|---|
| Purpose | Durable, ordered stream for your own consumers | Managed delivery to destinations |
| Consumers | Lambda, KCL apps, Flink, Firehose; many at once | None to write; Firehose is the consumer |
| Latency | Records readable almost immediately | Buffered by size and time before delivery |
| Retention and replay | 24 hours to 365 days, replayable | No retention for replay; failed deliveries go to an S3 backup |
| Destinations | Whatever your consumer writes to | S3, Redshift, OpenSearch, Splunk, Iceberg, HTTP endpoints and partners |
| Transformation | In your consumer | Lambda transform, format conversion to Parquet or ORC, dynamic partitioning |
| Capacity | On-demand or provisioned shards | Scales automatically |
| Pricing model | Stream hours plus GB written and read, or shard hours | Per GB ingested (Direct PUT records rounded up to 5 KB), plus optional features |
When Firehose alone is enough
- Shipping logs, CloudWatch Logs subscriptions or events to S3 for Athena queries.
- Feeding OpenSearch or a third-party observability tool from AWS sources.
- Converting JSON to Parquet and partitioning by date or field on the way into a data lake.
- You do not need to read the same data twice or in strict order.
Mind the 5 KB rounding on Direct PUT and Data Streams sources: very small records are billed as 5 KB each, so batch tiny events together where you can.
When you need Data Streams
- Real-time reactions: fraud checks, alerting, live dashboards that cannot wait for a buffer.
- Multiple independent applications on the same feed, with enhanced fan-out for dedicated read throughput.
- Replay to backfill a new consumer or recover from a bad deploy.
- Per-key ordering for state machines or aggregations.
Using both
The common pattern is producers → Kinesis Data Streams → (a) Lambda or Flink for real-time processing and (b) a Firehose stream reading the same data stream to archive everything in S3. Firehose with a Data Streams source does not add a second write for producers, and gives you a cheap durable archive alongside your real-time consumers.
Data Streams pricing has two on-demand flavours (Standard and Advantage) plus provisioned mode; check which fits the volume before committing to a design.
Common wrong reasons
- "Firehose is a stream I can read from." It has no consumer API; it only delivers to configured destinations.
- "Data Streams is needed to get data into S3." Firehose Direct PUT does that on its own.
- "Firehose is real time." It is near real time; buffering trades latency for fewer, larger objects.
Questions
What is the difference between Kinesis Data Streams and Firehose?
Data Streams stores ordered records for your own consumers to read and replay. Firehose is a managed pipe that buffers and delivers data to destinations such as S3, Redshift and OpenSearch without consumer code.
Can Firehose read from Kinesis Data Streams?
Yes. A Firehose stream can use a Kinesis data stream as its source, alongside other consumers of the same stream.
Is Kinesis Firehose renamed?
Yes. It is now called Amazon Data Firehose; the service is the same.
Delivery failures in Firehose land quietly in a backup bucket unless you alarm on them; stream consumers fall behind silently unless you watch iterator age. CloudConsole is being built to show those alarms on a phone with production guardrails.
About the app → · Download on the App Store