SQS Lambda Polling
How Does the Lambda Integration Work?
Lambda service, which is not a Lambda function, but rather a Lambda SQS long-poll service that runs on your behalf, will be the one that is going to poll your SQS queue continually for all incoming messages. As soon as a message arrives, it will start by receiving this message and, later on invoking the Lambda function by passing this message as a parameter.
This is what a typical message event would look like to a Lambda function which is going to be handling the messages that are passed in:
sqs lambda polling
Along with the process of passing over the message/s to your function, a Lambda Poller service is going to delete the message as well, in case your function returns back a successful response to the handling of the messages which was passed to it.
In case your function tends to fail in handling the message/s, the message/s will return back into the queue when the visibility timeout gets expired for being handled. A dead letter queue will be in association with your source queue as an optional setting that is capable of receiving SQS messages that your source queue had already received a number of times, which is defined by the user, without getting deleted.
Check out an example illustrating the way an integration flow tends to work as shown below with the labeled CloudWatch Logs from an SQS handler Lambda function.
sqs lambda polling – sqs lambda handler
Pricing Accompanied with Polling
No additional costs are going to be applied for provisioning SQS as an event source, but since the Lambda service which acts as an SQS consumer keeps on continuously long-polling an SQS queue, you might get charged for the API calls of sqs:ReceiveMessage according to the standard SQS pricing rates, regardless of whether you may have NO messages whatsoever in the queue.
What does SQS Short and Long Polling mean?
The entire process of getting messages consumed from a queue is dependent upon whether you rely on short or long polling. As a default setting, Amazon SQS will tend to use short polling, which means the act of querying merely a subset of its servers, according to a weighted random distribution, for determining if there may be any messages available for a response.
Long polling may be used for reducing costs while providing your consumers with the ability to receive messages on the spot when they arrive directly into the queue. The upcoming sections are going to explain the differences found between consuming messages with short polling or long polling.
Short Polling for Consuming Messages:
When messages get consumed from a queue by the use of short polling, Amazon SQS will sample a subset of its servers, according to a weighted random distribution, and then return messages merely from those selected servers. Hence, a specific ReceiveMessage request may not be able to return every single one of your messages. Yet, in case you possess less than 1,000 messages in your queue, then a subsequent request is going to return those messages. In case you remain consuming from those queues, Amazon SQS will sample the entirety of its servers, allowing you to be able to receive every one of your messages.
The below diagram highlights the short-polling behavior of messages that have been returned from a standard queue right when 1 of the system components delivers a receive request. Amazon SQS will sample several of its servers that are shown in gray, and then start to return the following messages: A – C – D – B from the mentioned servers. However, as seen below that message E does not get returned for this request, but later on returned for another subsequent request.
sqs lambda polling – short polling for consuming messages
Long Polling for Consuming Messages:
sqs lambda polling – long polling for consuming messages
Whenever you notice that the wait time for the ReceiveMessage API action goes greater than 0, this will mean that long polling is working in effect. The max wait time for long polling is: 20 seconds. Long polling is capable of aiding with the reduction of the cost of utilizing Amazon SQS because it can eliminate the number of empty responses whenever no messages are available for a ReceiveMessage request, and it is capable of making empty responses false as messages become available without being included in a response.
The below benefits can be accompanied by Long Polling:
- The elimination of empty responses by making it possible for Amazon SQS to wait till a message becomes available in a queue prior to sending a response. Whenever the connection does not time out, the response to the ReceiveMessagerequest is going to include a minimum of 1 of the available messages, all the way to the max number of messages which was specified in the ReceiveMessage
- The elimination of any false, empty responses through the act of querying all rather than merely a subset of those Amazon SQS servers.
Important Note
It is possible to confirm the fact that a queue has turned empty while performing a long poll and the ApproximateNumberOfMessagesDelayed, ApproximateNumberOfMessagesNotVisible, and ApproximateNumberOfMessagesVisible metrics will be equal to Zero after a minimum of one minute after producers have stopped sending any more messages, meaning that as soon as the queue metadata will reach eventual consistency.
- Return any messages whenever they have turned available.
How Do Long and Short Polling Differ?
Short polling takes place as soon as the WaitTimeSeconds parameter of a ReceiveMessage request gets set to 0 through 1 of the following 2 ways:
- ReceiveMessagecall specifies WaitTimeSeconds to be 0.
- ReceiveMessagecall doesn’t specify WaitTimeSeconds, rather, the queue attribute ReceiveMessageWaitTimeSeconds gets set to be 0.
See Also