Storing Logs on S3 – Parsing Logs and Events with AWS Native Tools – SCS-C02 Study Guide

Storing Logs on S3

One of the optimal places to store log files—either initially or after placing them in another store—is the S3 storage service. This service provides both the resiliency and durability desired for long-term storage and objects such as log files. Where it can fall short is the ability to search the various files directly.

Different Storage Tiers of S3

Amazon S3 offers several storage classes or tiers with different performance, durability, and cost characteristics. These storage tiers are designed to help you and other customers optimize their storage costs based on the frequency of access to your data, the performance required, and your business needs.

The following subsections will take you through the different storage tiers in detail and see how they are both similar and different.

S3 Standard

When you set up an S3 bucket, objects and files will be stored in the S3 Standard tier if you don’t choose any particular storage tier. This highly available, general access storage policy allows millisecond access to objects when requesting their retrieval. Although this is the most expensive of all the storage tiers, S3 Standard storage is a highly cost-effective solution compared to other types of storage services such as File (think of the Elastic File System or EFS service) and Block (think of the Elastic Block Store or EBS service).

Key Points to Remember about S3 Standard

  • The Standard tier provides high throughput, low latency, and high performance for object uploads and downloads.
  • If no other storage tier is indicated, then Standard is the default storage class.
  • It is ideal for objects that need frequent access.
  • It is constructed for 99.99% availability and 99.999999999% durability via multiple Availability Zones during a given year.
  • It can support SSL for data in transit and encryption at rest.

S3 Intelligent-Tiering

When you don’t know how often you need to access your data or objects stored in a particular S3 bucket, the best option may be S3 Intelligent-Tiering. AWS uses a proprietary algorithm to monitor the access of objects in the designated bucket to move them to the correct Frequent and Infrequent Access levels, along with pricing for the respective levels. Using this storage tier saves you from having to develop complex lifecycle policies for the objects in your bucket, allowing you to concentrate on your other tasks.

Key Points to Remember about S3 Intelligent-Tiering

  • It is designed to optimize storage costs by automatically moving objects to the most cost-effective storage tier. Savings can be up to 40% of Standard tier pricing.
  • It is designed for more extended storage of at least 30 days (minimum 30-day charge), and it takes 30 days to start figuring out access patterns.
  • It stores objects in three access tiers and optimizes storage based on frequently, infrequently, and archived instant access objects.
  • There is no performance impact or additional fees when Intelligent Tiering moves objects between tiers.
  • It was constructed for 99.9% availability and 99.999999999% durability during a given year.
  • It can support SSL for data in transit and encryption at rest.