Amazon Kinesis is a platform for handling massive streaming data on AWS,offering powerful services to make it easy to load and analyze streaming data and also providing the ability for you to build custom streaming data applications for specialized needs.
Amazon Kinesis is a streaming data platform consisting of three services addressing different real- time streaming data challenges:
Amazon Kinesis Fire hose
This service enabling you to load massive volumes of streaming data into AWS.
Amazon Kinesis Firehouse receives stream data and stores it in Amazon S3,Amazon Redshift,or Amazon Elastic search.You do not need to write any code;just create a delivery stream and configure the destination for your data.Clients write data to the stream using an AWS API call and the data is automatically sent to the proper destination.
Amazon Kinesis Streams
A service enabling you to build custom applications for more complex analysis of streaming data in real time.
Amazon Kinesis Streams enable you to collect and process large streams of data records in real time. Using AWS SDKs,you can create an Amazon Kinesis Streams application that processes the data as it moves through the stream.Because response time for data in take and processing is in near real time,the processing is typically light weight.Amazon Kinesis Streams can scale to support nearly limitless data streams by distributing incoming data across a number of shards.If any shard becomes too busy,it can be further divided into more shards to distribute the load further.The processing is then executed on consumers,which read data from the shards and run the Amazon Kinesis Streams application.
Amazon Kinesis Analytics
A service enabling you to easily analyze streaming data real time with standard SQL.
Amazon Elastic Map Reduce(Amazon EMR)
Amazon Elastic Map Reduce(Amazon EMR)provides you with a fully managed,on-demand Hadoop framework.Amazon EMR reduces the complexity and up-front costs of setting up Hadoop and, combined with the scale of AWS gives you the ability to spin uplarge Hadoop clusters instantly and start processing with in minutes.
UseCases for EMR:
Amazon EMR is well suited for a large number of use cases, including, but not limited to:
Log Processing:Amazon EMR can be used to process logs generated by web and mobile applications.Amazon EMR helps customers turn petabytes of unstructured or semi-structured data into useful in sights about their applications or users.
Click stream Analysis:Amazon EMR can be used to analyze click stream data in order to segment users and understand user preferences. Advertisers can also analyze click streams and advertising impression logs to deliver more effective ads.
AWS Data Pipeline
AWS Data Pipeline is a web service that helps you reliably process and move data between different AWS compute and storage services,and all soon-premises data sources,at specified intervals.With AWS Data Pipeline,you can regularly access your data where it’s stored,transform and process it at scale,and efficiently transfer the results to AWS services such as Amazon S3, Amazon Relational Database Service(Amazon RDS),Amazon Dynamo DB,and Amazon EMR.
AWS Cloud Formation
AWS Cloud Formation is a service that helps you model and setup your AWS resources so that you can spend less time managing those resources and more time focusing on your applications that run in AWS. AWS Cloud Formation allows organizations to deploy, modify, and update resources in a controlled and predictable way, in effect applying version control to AWS infrastructure the same way one would do with software.
AWS Cloud Formation gives developers and systems administrators an easy way to create and manage a collection of related AWS resources,provisioning and updating them in an orderly and predictable fashion.When you use AWS Cloud Formation,you work with templates and stacks
By allowing you to replicate your entire infrastructure stack easily and quickly, AWS Cloud Formation enables a variety of use cases:
Quickly Launch New Test Environments: AWS Cloud Formation let’s testing teams quickly create a clean environment to run tests without disturbing ongoing efforts in other environments.
Reliably Replicate Configuration:between Environments Because AWS Cloud Formation scripts the entire environment,human error is eliminated when creating new stacks.
Launch Applications in New AWS Regions:A single script can be used across multiple regions to launch stacks reliably in different markets.
More Information :https://www.fgrade.com/aws/