aws lambda create-event-source-mapping
aws lambda create-event-source-mapping --function-name my-function --event-source-arn <queue or stream ARN> makes Lambda poll that source and invoke the function with batches of records. For SQS add --function-response-types ReportBatchItemFailures; for Kinesis and DynamoDB Streams set --starting-position and an on-failure destination.
What the command does
An event source mapping is a Lambda-managed poller. It reads from the source, groups records into batches, invokes the function synchronously and, depending on the source, deletes messages or advances the stream position on success. It runs with the function's execution role, which needs read permissions on the source (for SQS: sqs:ReceiveMessage, sqs:DeleteMessage, sqs:GetQueueAttributes).
The command returns a UUID that identifies the mapping for later updates, and a State that moves from Creating to Enabled.
Syntax
aws lambda create-event-source-mapping \
--function-name my-function \
--event-source-arn arn:aws:sqs:eu-west-2:123456789012:orders \
[--batch-size 10] [--maximum-batching-window-in-seconds 5] \
[--function-response-types ReportBatchItemFailures] \
[--starting-position LATEST|TRIM_HORIZON|AT_TIMESTAMP] \
[--filter-criteria ...] [--scaling-config MaximumConcurrency=10] \
[--destination-config ...] [--no-enabled]Worked examples
SQS queue with partial batch failure reporting and a concurrency cap:
aws lambda create-event-source-mapping \
--function-name orders-worker \
--event-source-arn arn:aws:sqs:eu-west-2:123456789012:orders \
--batch-size 10 \
--maximum-batching-window-in-seconds 2 \
--function-response-types ReportBatchItemFailures \
--scaling-config MaximumConcurrency=20
With ReportBatchItemFailures, the function returns the IDs of messages that failed and only those are retried, instead of the whole batch.
Kinesis stream with bounded retries and a failure destination:
aws lambda create-event-source-mapping \
--function-name clickstream \
--event-source-arn arn:aws:kinesis:eu-west-2:123456789012:stream/clicks \
--starting-position LATEST \
--batch-size 500 \
--bisect-batch-on-function-error \
--maximum-retry-attempts 3 \
--maximum-record-age-in-seconds 3600 \
--destination-config '{"OnFailure":{"Destination":"arn:aws:sqs:eu-west-2:123456789012:clicks-dlq"}}'
Only invoke for some messages, using a filter on the SQS body:
aws lambda create-event-source-mapping \
--function-name refunds \
--event-source-arn arn:aws:sqs:eu-west-2:123456789012:payments \
--filter-criteria '{"Filters":[{"Pattern":"{\"body\":{\"type\":[\"refund\"]}}"}]}'
Check it is running:
aws lambda list-event-source-mappings --function-name orders-worker \
--query "EventSourceMappings[].[UUID,State,BatchSize,LastProcessingResult]" --output tableOptions by source
| Option | SQS | Kinesis / DynamoDB Streams |
|---|---|---|
--starting-position | Not used | Required |
--function-response-types ReportBatchItemFailures | Yes | Yes |
--bisect-batch-on-function-error, --maximum-retry-attempts, --maximum-record-age-in-seconds | No (use the queue's redrive policy) | Yes |
--parallelization-factor | No | Batches per shard processed concurrently |
--scaling-config MaximumConcurrency | Yes | No |
Gotchas
- Filtered-out SQS messages are deleted, not left in the queue. Do not filter a queue that other consumers also need.
- Visibility timeout. Set the queue's visibility timeout well above the function timeout (AWS suggests at least six times), or messages reappear while still being processed and run twice.
- Stream retries block the shard. By default a failing Kinesis or DynamoDB batch is retried until the records expire, holding up everything behind it. Always set retry and age limits plus an on-failure destination.
- Duplicate mappings for the same source and function fail with
ResourceConflictException; update the existing one instead. - FIFO queues preserve order per message group. A message that keeps failing blocks later messages in the same group until it succeeds or moves to the dead-letter queue, so configure a redrive policy on FIFO sources too.
- New mappings take a moment. The state goes from
CreatingtoEnabled; check it withget-event-source-mappingbefore assuming the trigger is live.
Questions
Does an SQS trigger need add-permission?
No. Event source mappings use the function's execution role, so grant the SQS read and delete actions there instead.
How do I create a Lambda function from the CLI?
Use aws lambda create-function with a role ARN, runtime, handler and code. Creating the event source mapping is a separate step afterwards.
How do I alarm on a Lambda trigger falling behind?
For SQS, alarm on ApproximateAgeOfOldestMessage for the queue. For Kinesis, alarm on the function's IteratorAge metric.
Queue backlogs and stuck streams tend to surface as alarms long after the first failure. CloudConsole, a phone app for AWS, is built to show queue depth, function errors and logs together on a phone.
About the app → · Download on the App Store