aws logs put-subscription-filter
aws logs put-subscription-filter sends matching log events from a log group to Lambda, Kinesis Data Streams, Amazon Data Firehose or a cross-account destination in near real time. Lambda targets need lambda add-permission for logs.amazonaws.com; Kinesis and Firehose targets need --role-arn. A log group can have only two subscription filters.
What the command does
A subscription filter attaches a filter pattern and a destination to a log group. Each new event that matches is delivered to the destination, batched, gzip-compressed and base64-encoded. Only events ingested after the filter is created are sent; existing data is not replayed.
Common uses: shipping logs to a third-party observability tool through Firehose, triggering a Lambda function on specific errors, and central logging accounts that collect from many accounts.
Syntax
aws logs put-subscription-filter \
--log-group-name /ecs/orders-api \
--filter-name to-firehose \
--filter-pattern '' \
--destination-arn arn:aws:firehose:eu-west-2:123456789012:deliverystream/logs \
[--role-arn arn:aws:iam::123456789012:role/CWLtoFirehose] \
[--distribution ByLogStream|Random]
--filter-pattern '' (empty string) sends every event. Calling the command again with the same --filter-name updates the filter in place.
Worked examples
Trigger a Lambda function on errors. First let CloudWatch Logs invoke it:
aws lambda add-permission --function-name log-alerter \
--statement-id cwl-orders-api \
--action lambda:InvokeFunction \
--principal logs.amazonaws.com \
--source-arn "arn:aws:logs:eu-west-2:123456789012:log-group:/ecs/orders-api:*" \
--source-account 123456789012
aws logs put-subscription-filter \
--log-group-name /ecs/orders-api \
--filter-name errors-to-alerter \
--filter-pattern '?ERROR ?FATAL' \
--destination-arn arn:aws:lambda:eu-west-2:123456789012:function:log-alerter
Send everything to Kinesis Data Streams, spreading load evenly over shards:
aws logs put-subscription-filter \
--log-group-name /aws/lambda/my-function \
--filter-name all-to-kinesis \
--filter-pattern '' \
--destination-arn arn:aws:kinesis:eu-west-2:123456789012:stream/central-logs \
--role-arn arn:aws:iam::123456789012:role/CWLtoKinesis \
--distribution Random
The role must trust logs.amazonaws.com and allow kinesis:PutRecord on the stream (or firehose:PutRecord and firehose:PutRecordBatch for Firehose).
List and remove filters:
aws logs describe-subscription-filters --log-group-name /ecs/orders-api \
--query "subscriptionFilters[].[filterName,filterPattern,destinationArn]" --output table
aws logs delete-subscription-filter --log-group-name /ecs/orders-api --filter-name errors-to-alerter
Decode a payload when testing a Lambda consumer (the event carries it under awslogs.data):
echo "$DATA" | base64 -d | gunzip | jq '.logEvents[].message'Destinations
| Destination | Needs | Good for |
|---|---|---|
| Lambda | Resource policy for logs.amazonaws.com | Alerting, light transformation |
| Kinesis Data Streams | --role-arn | High volume, multiple consumers |
| Amazon Data Firehose | --role-arn | Delivery to S3, OpenSearch or third-party HTTP endpoints |
| Logs destination (cross-account) | A destination policy in the receiving account | Central logging accounts |
Gotchas
- Two filters per log group. A third fails with
LimitExceededException. Use one filter to Kinesis or Firehose and fan out from there. - Lambda can be overwhelmed. A chatty group sends a steady stream of invocations; errors and throttles in the consumer mean lost or delayed delivery. Filter tightly.
- Recursive loops. Never subscribe a function's own log group to that function: every invocation writes logs, which invoke it again.
- Retroactive delivery does not exist. To process history, export to S3 or run a Logs Insights query instead.
Questions
Does put-subscription-filter send old log events?
No. Only events ingested after the filter exists are delivered.
How many subscription filters can a log group have?
Two at the time of writing. Use Kinesis or Firehose as a single hub if more consumers need the data.
How do I put a metric filter instead of a subscription?
Use aws logs put-metric-filter, which turns matching events into a CloudWatch metric rather than streaming them.
Turning specific log lines into alerts is how many teams first hear about problems on their phones. CloudConsole, a phone app for AWS, is built to show those alerts and the surrounding logs together.
About the app → · Download on the App Store