Flow Logs – Logging and Monitoring – ANS-C01 Study Guide

Flow Logs

Flow Logs capture data on IP traffic flows between interfaces in a VPC. You select the source and destination of the flow you want to analyze, and the service will show you the path between the two inside of AWS. The Flow Log captures are external to the actual data flow in your VPC, so the captures do not affect network latency or throughput of your production traffic. Flow Logs can be created from sources such as Elastic Load Balancer (ELB), Amazon Relational Database Service (RDS), Redshift, workspaces, NAT gateways, and Transit Gateways. Instances launched in a VPC after a Flow Log has been created are automatically added to the flow capture.

The Flow Log capture is configured; then you run the capture, and the data gets stored in CloudWatch logs or, optionally, in a S3 bucket or sent to Kinesis Firehose. Analyzing the logs is often used for monitoring traffic sent and received from an EC2 instance, analyzing traffic flows inside your VPC, and troubleshooting security group restrictions in addition to other uses. Logs can be created, deleted, or changed without any interference to your actual production traffic flows. Log monitor points can be at the VPC level, a subnet, or a network interface inside of a VPC. If the Flow Log is created for a VPC, each interface inside of that VPC is monitored. When creating a Flow Log, specify the resource to be monitored, whether you want to capture all accepted and rejected traffic, and where you want the data to be stored. Once the Flow Log is created, there is a delay of up to several minutes before the data is collected and stored. Flow logging is not considered a real-time service.

If you delete a Flow Log, the data collected is retained, and all log captures are suspended. If you do not want to preserve the capture logging data, then you will need to delete it from the storage locations you defined in the original configuration or through the use of life-cycle policies.

Flow Logs are created in the console in the VPC service. Select the VPC you want to enable logging, click the Flow Logs tab, and create the Flow Log, as shown in Figure 5.12. Traffic mirroring supports filters and packet truncation so that you extract only the traffic of interest by using the monitoring tools of your choice. When configuring traffic mirroring, you specify the source, destination, and a filter and then give it a session name.

Traffic mirroring is configured in the VPC section of the AWS console and allows you to filter the packets and truncate them to reduce the amount of data generated and to capture only traffic of interest. When Flow Logs are created, there are certain traffic types that are not logged including the following: AWS DNS server traffic (however, private internal DNS is logged), AWS instance Windows license activations, instance metadata to the 169.254.169.254 address, time sync traffic to 169.254.169.123, any DHCP-related traffic, traffic from mirrored interfaces, VPC default gateway reserved IP address traffic, and traffic from endpoint interfaces, and a network load balancer.

Pricing for Flow Logs includes data ingestion of the logging traffic and storage charges. Check the CloudWatch console for detailed pricing information.

FIGURE 5.12 Creating a VPC Flow Log