Amazon DynamoDB is a fully managed NoSQL database service whereas DynamoDB Streams allows you to capture a time-ordered sequence of item-level modifications made to your DynamoDB tables. These streams are available upto 24 hours. While processing the stream records, you can use a Lambda function to process the stream records in real-time, applications can access this log as changes appear before and after, making it easier to keep track of the changes to your DynamoDB tables.
DynamoDB Streams can be enabled by setting the stream on a table. This can be done by selecting the table you want to enable the stream on, and then clicking on the “Manage Stream” button. Once the stream is enabled, stream records can be accessible, the best approach to consume such records is using a Lambda function. This will enable you to easy scale up to compensate near real time processing.
Each record in the stream logs a modification to an item in the table and contains both the new and old image of the item. You can fetch either original item or new image that hold modified item. You will have an option to choose to retrieve only the keys of the modified items or the full item images. If you consider the cost of DynamoDB Streams, accessing only the keys is cheaper comparing the item itself. This might be useful if you only need what has been changed but not the item itself but if you need to access additional information about the item, you need to retrieve the item itself, it will cost more but you will have either original or modified version of the item.
You should use a primary key to identify and fetch each stream record. The primary key is composed of the table’s partition key and sort key (if applicable). Cost of DynamoDB Streams is calculated as streams read request unit which is GetRecords API call to DynamoDB Streams. Each streams read request unit can return up to 1 MB of data. If you are using DynamoDB triggers, you won’t be charged for GetRecords API calls invoked by AWS Lambda or invoked by DynamoDB global tables.
DynamoDB Streams
- AWS launched the DynamoDB Streams to give users a chronological sequence of changes at the item level in any DynamoDB table.
- It restores the changes in their original form and stores them for a period of 24 hours.
- This is a great way to boost the power of DynamoDB through change notifications, cross-region replication, continuous analytics with Redshift integration and similar situations.
- This is a low-cost addition to your existing DynamoDB package but small and medium business owners can benefit greatly from the extremely affordable DynamoDB Streams pricing.
Have a look at the cost of DynamoDB & let’s get to explore a little more about this excellent feature.
What are the Uses of DynamoDB Streams?
Before reading further, try to remember if any of these questions ever occurred to you:
- How do I set up a network across multiple tables so that based on the value of an item in one table, I can also update the item on the second table?
- How to trigger events based on individual transactions?
- How do I replicate data across multiple tables?
- How do I archive or audit transactions in DynamoDB?
On the one hand, relational databases offer native support to perform transactions, auditing, triggers and replication. In general, a transaction is any CRUD (create, read, update & delete) operation among multiple tables within a block. However, a transaction can only have two results – success (or) failure. As DynamoDB is a NoSQL database, it does not support transactions.
- In such cases, the DynamoDB Streams works as the best solution. DynamoDB Streams is extremely powerful and can easily collaborate with other AWS services to perform similar complex problems.
- When activated, DynamoDB Streams is an excellent way to capture changes to items from a DynamoDB table as soon as the modification is done.
There are several ways where this feature is extremely useful, such as when:
- An application modifies data in a DynamoDB table in one AWS Region. Another application immediately reads the modifications to data and stores the information on the changed data on another table. This creates a replica that is always synchronized with the original table.
- A mobile app is able to modify data in DynamoDB tables at the rate of thousands of updates every second. A second application can capture and store the information about the updates, which helps to provide almost real-time and accurate usage metrics for the mobile app.
- Any global multi-player game has a multi-master topology it follows, whose data is stored in several AWS Regions at once. This way, every master can stay synchronized by accessing and processing the changes which develop in the more remote AWS Regions.
- A social networking app alerts every user with a notification on their mobile device when a friend in a group uploads a new post.
- A new customer can fill data in a DynamoDB table. This causes another application to send out an automatic welcome email to the new customer.
There are numerous other such scenarios when DynamoDB Streams becomes an exceptionally efficient tool. The log of data modification information stored by DynamoDB Streams can be accessed by other applications to view the sequence of every modification and get a clear view of their original form and the modified form almost instantly.
How much is the DynamoDB Streams pricing?
AWS offers the DynamoDB in two distinct packages based on their capacity modes:
- DynamoDB Provisioned Capacity
- DynamoDB On-Demand Capacity
The DynamoDB provisioned capacity mode lets developers choose the number of resources every database will need to perform its functions beforehand. The developers may also have to decide whether they need auto-scaling of resources or whether the database should start ignoring requests once the specified limit of resources is reached.
On the other hand, the DynamoDB on-demand capacity will automatically increase or decrease the number of allocated resources as per fluctuation in API requests and charges according to data usage on a monthly basis.
To understand the complex pricing plans, you need to be aware of certain technical terms, such as:
- Read request – This is an API call to read data from a specific DynamoDB table.
- Write request – This is an API call to add, modify or delete items in the DynamoDB table.
1 read request can be up to 4 KB. Every additional read request is rounded up according to 4 KB sizes.
1 write request can be up to 1 KB. Every additional write request is rounded up according to 1 KB size.
DynamoDB Streams Pricing
Amazon Web Services charges DynamoDB Streams pricing at US$ 0.02 per 100,000 reads or write requests. The charges for the feature are the same in the On-Demand and Provisioned Capacity modes.
Further Thoughts
DynamoDB Streams is an excellent way to maintain an accurate and chronologically arranged log of every change to items on your DynamoDB tables.
See Also
AWS DynamoDb Cost Best-Practices
- CloudySave is an all-round one stop-shop for your organization & teams to reduce your AWS Cloud Costs by more than 55%.
- Cloudysave’s goal is to provide clear visibility about the spending and usage patterns to your Engineers and Ops teams.
- Have a quick look at CloudySave’s Cost Calculator to estimate real-time AWS costs.