You may think that because you are studying for a test that emphasizes security, the pricing does not matter. However, this is not the case because keeping costs under control is everyone’s responsibility in an organization. The ability to choose the proper storage for the logs in an organization is not only a real-world, sought-after skill but also helps you determine the most cost-effective solution for items that present themselves on the AWS certification exams. Remember that the most cost-effective solution is not always the least expensive, as some features and benefits must be considered with each service and solution.
Note
In the Certified Security Specialty exam, you will likely find a question or scenario that asks you not only about the most appropriate but also the most cost-effective solution. To be as prepared as possible, you do not need to memorize the pricing for the different storage solutions but, rather, understand which services have higher total price points compared to the features they deliver.
To help you with transitioning from one storage system to another, the next section discusses moving logs from CloudWatch Logs to other storage systems.
Although CloudWatch Logs provides a sustainable logging solution right away, storage can be a little more costly than other solutions for logs that are not being utilized and just need to be kept for compliance purposes. Additionally, the service does not have the ability to search across multiple log groups at the same time. If you need to either cut costs for your log storage or search across multiple streams of logs for trends and anomalies, you will need to move those logs to another storage system.
It all begins with the use of CloudWatch subscription filters. As you will see in the following exercise, subscription filters allow you to push all or a subset of the logs that are coming into the CloudWatch Logs group to a Kinesis Data Firehose stream or a Lambda function, or even export the data to an S3 bucket.
Figure 9.7: Moving log files from CloudWatch Logs to S3
The next section will dive deeper into subscription filters in CloudWatch logs.
Subscription filters are a feature of Amazon CloudWatch Logs that allow users to create a real-time stream of log events from a CloudWatch Logs group to a destination such as Kinesis Data Firehose, an Amazon Lambda function, or an Amazon SNS topic.
If you didn’t want to save all of your logs from a particular CloudWatch Logs group and only wanted to retain logs that were of importance to you and your team (say, for instance, logs that contained ERROR), then this would be a good use case for setting up a CloudWatch Logs subscription filter.
A subscription filter can match a particular pattern or specific keywords and send those log events to an Amazon Kinesis Data Firehose delivery stream. The Firehose delivery stream’s destination would be an S3 bucket or Amazon OpenSearch Service cluster so that those logs could be stored more permanently.
Subscription filters can also be used in concert with other AWS services. In the next section, you will see how to ship logs to an S3 bucket using subscription filters and Amazon Kinesis.
Acedexam Training Policies
Learning resources
Other resources
Our Training Providers
