- Amazon Kinesis enables you to capture and process large streams of data records in real time. It has several components:
- Kinesis Data Streams (KDS): Capture and store streaming data.
- Kinesis Data Firehose: Load data into destinations like S3, Redshift, or OpenSearch.
- Kinesis Data Analytics: Analyze data in real time using SQL or Apache Flink.
In this blog, we will focus on Kinesis Data Streams (KDS).
Open the Kinesis Console
1. Go to the Amazon Kinesis Console
2. In the left-hand menu, choose Data Streams.
3. Click Create data stream.
Create a Data Stream
1. Enter a name for your stream.
2. Choose Provisioned capacity.
3. Set the number of shards (1 is fine for testing).
Each shard can handle up to 1 MB/s write and 2 MB/s read throughput.
4. Click Create data stream.
Wait until the stream status changes to Active.
Put Data into the Stream
1. Once your stream is active, click on it to open its details page.
2. Go to the Put record tab.
3. Enter a partition key and some sample data ( {“message”: “Test stream”}).
4. Click Put record.
Your data record has now been added to the Kinesis stream!
Read Data from the Stream
To verify your data:
1. Still in the stream details, choose Data viewer.
2. Select the shard and click Get records.
You’ll see the record you just added displayed in the console.
Clean Up
To avoid charges, delete your stream after testing:
1. Go back to Data Streams in the console.
2. Select your stream and click Delete.
3. Confirm deletion.
Conclusion
- You have just created your first Amazon Kinesis Data Stream using the AWS Management Console.
- With Kinesis, you can easily integrate data producers and consumers to build real-time analytics pipelines, whether you’re streaming IoT data, clickstreams, or application logs.
Need help setting up Amazon Kinesis Data Streams? Skynats offers expert AWS Management Services to help you deploy, monitor, and optimize your AWS infrastructure seamlessly.
