Viewing the Access Logs – Logging and Monitoring – SCS-C02 Study Guide

Viewing the Access Logs

S3 access logs can be downloaded from the AWS Management Console or the AWS CLI. S3 access logs are not written in real time; they are created in batches that take a few hours to appear in your S3 logging bucket.

After a few hours, you can check your logging bucket to see whether the folder you had designated in your configuration file has appeared. Figure 7.1 shows that the folder was created after a short time, and AWS batched up the initial set of requests.

Figure 7.1: The logging folder for S3 access logs inside of our logging bucket

As you enter the folder that contains the S3 access logs, you will notice that unlike other batch groupings of logs that AWS gathers, these logs are not grouped in separate subfolders by month, day, and year. Instead, each log file’s name begins with a time-date stamp and ends with a semi-random value.

The logs themselves are in text format but have no extension added to them.

Consider the following example log:

3

0cd0fc55a0a1b1164bfd69733c5b1bedd043dd0ce88e77cb1b2cb9e6418a3e5 p

ackt-test-object [

11/Apr/2023:01:59:50 +0000] 123.456.78.90 arn:aws:iam::123456789101:user/packt 5GP5E0KH8855K76Y REST.GET.ACL – “GET /?acl HTTP/1.1” 200 – 556 – 14 – “-” “S3Console/0.4, aws-internal/3 aws-sdk-java/1.11.1030 Linux/5.4.238-155.347.amzn2int.x86_64 OpenJDK_64-Bit_Server_VM/25.362-b10 java/1.8.0_362 vendor/Oracle_Corporation cfg/retry-mode/standard” – uhbLAOkdZMQTHAfP/ixHjJtznBNWzDc4JcrOa0YTnqtBDJzmvUz2fViv2eL4IOZoadKJQet8rm0= SigV4 ECDHE-RSA-AES128-GCM-SHA256 AuthHeader packt-test-object.s3.us-east-1.amazonaws.com TLSv1.2 – –

The following are the fields in the order in which they appear in the preceding example:

• Bucket Owner: Although this is a long alphanumerical value, this is the canonical user ID for the source bucket owner.

Note

If you would like to know how to find the canonical user ID for your own AWS account, then you can find out more with the following URL: https://packt.link/Z0HUO.

The value in the example is as follows: 30cd0fc55a0a1b1164bfd69733c5b1bedd043dd0ce8 8e77cb1b2cb9e6418a3e5

• Bucket: This is the name of the S3 bucket where the action or request was processed. This field will be blank if this was a malformed request and AWS cannot determine the actual bucket.

The value in the example is as follows: packt-test-object

• Time: This is the time the request was received in UTC (which stands for Universal Time Code).

The value in the example is as follows: [11/Apr/2023:01:59:50 +0000]

• Remote IP: This refers to the apparent IP address of the requester; however, this is not always the correct IP due to proxies and firewalls.

The value in the example is as follows: 123.456.78.90

• Requester: This is the IAM of the user if the user is an IAM user or the canonical ID of the request. If this value is unknown or the user is unauthenticated, then the field has – for those types of requests.

The value in the example is as follows: arn:aws:iam::123456789101:user/packt

• Request ID: This is a unique string to identify each request.

The value in the example is as follows: 5GP5E0KH8855K76Y

• Operation: This is the type of operation performed, such as SOAP, REST, WEBSITE, or BATCH.DELETE.OBJECT.

The value in the example is as follows: REST.GET.ACL

• Key: This will be the object name, URL, or – if the operation does not have an object parameter.

The value in the example is as follows: –

• Request-URI: This is the Request-URI part of the HTTP request message.

The value in the example is as follows: “GET /?acl HTTP/1.1”

• HTTP Status: This is the status code of the HTTP response.

The value in the example is as follows: 200

• Error Code: This is the error code that S3 returned, or – if no error was returned.

The value in the example is as follows: –

• Bytes Sent: This is the number of response bytes sent minus the protocol overhead. If this number is zero, it will be represented by –.

The value in the example is as follows: 556

• Object Size: This is the total size of the object.

The value in the example is as follows: –

• Total Time: This shows the time in milliseconds that the request was in flight.

The value in the example is as follows: 14

• Turn-Around Time: This is the time in milliseconds that the S3 service spends processing the request.

The value in the example is as follows: –

• Referrer: This is the value of the HTTP Referrer header.

The value in the example is as follows: “-“

• User-Agent: This is the value of the HTTP User-Agent header.

The value in the example is as follows: “S3Console/0.4, aws-internal/3 aws-sdk-java/1.11.1030 Linux/5.4.238-155.347.amzn2int.x86_64 OpenJDK_64-Bit_Server_VM/25.362-b10 java/1.8.0_362 vendor/Oracle_Corporation cfg/retry-mode/standard”

• Version Id: This is the version ID of the request.

The value in the example is as follows: –

• Host Id: This is the x-amz-id-2 or Amazon S3 extended request ID.

The value in the example is as follows: uhbLAOkdZMQTHAfP/ixHjJtznBNWzDc 4JcrOa0YTnqtBDJzmvUz2fViv2eL4IOZoadKJQet8rm0=

• Signature Version: This is the SigV2 or SigV4 used to authenticate the request.

The value in the example is as follows: SigV4

• Cipher Suite: This is the Secure Sockets Layer (SSL) cipher used in the HTTPS request.

The value in the example is as follows: ECDHE-RSA-AES128-GCM-SHA256

• Authentication Type: This will be either AuthHeader for authentication headers or QueryString for pre-signed URLs or – for a request that was not authenticated.

The value in the example is as follows: AuthHeader

• Host Header: This is the endpoint used to connect to the S3 service.

The value in the example is as follows: packt-test-object.s3.us-east-1.amazonaws.com

• TLS version: This is the version of Transport Layer Security (TLS) negotiated by the client.

The value in the example is as follows: TLSv1.2

• Access Point ARN: This is the ARN of the access point of the request.

The value in the example is as follows: –

• aclRequired: This will be a Boolean if the requests require an access control list (ACL) for authorization.

The value in the example is as follows: –

Now that you have seen what the S3 access logs look like, how to parse them manually, and where to retrieve them, take a look at another type of logging available from S3 object-level logging, discussed in the next section.