Centralizing and Analyzing Logs – Determining Security Requirements and Controls – SAP-C02 Study Guide

Centralizing and Analyzing Logs

It is essential to consolidate logging from your solution in a central location for further analysis. In that central location, you want to make sure you collect all the logs, whether from AWS services, custom solution components, or third-party services. The objective of aggregating the logs is twofold: analysis and retention. Analysis can consist of forensics or troubleshooting activity. Retention is meant to ensure compliance with your organization’s requirements or to satisfy any legal or regulatory obligation to keep activity logs for a minimum period.

As was already mentioned in Chapter 3, Designing a Multi-Account AWS Environment for Complex Organizations, you are recommended, especially if you are part of a complex organization, to centralize your various logs—for instance, from CloudTrail and CloudWatch, from all accounts into a central Log Archive account. Think of that central log archive storage as a write once, read many (WORM) kind of storage where you keep logging information as immutable data. Typically, you would land all logs onto Amazon S3 and leverage features such as S3 Object Lock to guarantee immutability.

However, because you need to make the logs available to the security operations team, internal or external auditors, and possibly a few other actors within the organization, it is important to make sure that they do not contain any sensitive information (for instance, people’s names; credit card or social security numbers). All sensitive information should be redacted before landing on the central log archive location, while you can keep a local copy of the logs in the source account, for a shorter period, with the unredacted information if needed.

You can see some examples of centralized logging and log indexing solutions in Figure 5.5:

Figure 5.5: Centralized logging and log indexing for search and audit

From the central log archive, the security operations team can put in place their preferred workflow for analysis and monitoring. They will want to feed logs data into their existing Security Information and Event Management (SIEM) system or index the information to leverage their preferred search tool (for instance, Amazon OpenSearch, which was previously known as Amazon Elasticsearch). The amount of logging data collected can rapidly become overwhelming, so it is essential to have the right capabilities in place to automate filtering and prioritization of security incidents. Some AWS services such as AWS Security Hub can help you with that, but depending on your organization’s context, you’re likely to integrate these services with your own tools, for instance, to use a single pane of glass for security incident detection and analysis or to correlate incident data collected from AWS services with some other information coming from third-party tooling.

Now that you have gone through some best practices to detect and analyze incidents, you can review how best you can respond to them.