Creating buckets to hold data – AWS Services for Data Storage – MLS-C01 Study Guide

Creating buckets to hold data

Now, you will see how to create a bucket, upload an object, and read the object using the AWS CLI:

  1. In the first step, check whether you have any buckets created by using the aws s3 ls command:

$ pwd
/Users/baba/AWS-Certified-Machine-Learning-Specialty-
2020-Certification-Guide/Chapter-5/s3demo/demo-files
$ aws s3 ls

  • This command returns nothing here. So, create a bucket now by using the mb argument. Let’s say the bucket name is demo-bucket-baba in the us-east-1 Region:

$ aws s3 mb s3://demo-bucket-baba –region us-east-1
make_bucket: demo-bucket-baba
$ aws s3 ls
2020-11-04 14:39:50 demo-bucket-baba

  • As you have created a bucket now, your next step is to copy a file to your bucket using the cp argument, as shown in the following code:

$ aws s3 cp sample-file.txt s3://demo-bucket-baba/
upload: ./sample-file.txt to s3://demo-bucket-
baba/sample-file.txt

  • To validate the file upload operation via the AWS console, please log in to your AWS account and go to the AWS S3 console to see the same. The AWS S3 console lists the result as shown in Figure 2.2. The console may have changed by the time you are reading this book!

Figure 2.2 – AWS S3 listing your files

You can also list the files in your S3 bucket from the command line, as shown here:

$ aws s3 ls s3://demo-bucket-baba/
2020-11-04 14:50:02         99 sample-file.txt

  • If you want to upload your filesystem directories and files to the S3 bucket, then –recursive will do the job for you:

$ aws s3 cp .
s3://demo-bucket-baba/ –recursive
upload: folder-1/a.txt to s3://demo-bucket-baba/folder-1/a.txt
upload: folder-2/sample-image.jpg to s3://demo-bucket-baba/folder-2/sample-image.jpg
upload: ./sample-file.txt to s3://demo-bucket-baba/sample-file.txt
$ aws s3 ls s3://demo-bucket-baba/

  • The contents of one bucket can be copied/moved to another bucket via the cp command and the –recursive parameter. To achieve this, you will have to create two buckets, demo-bucket-baba-copied and demo-bucket-baba-moved. The steps are as follows:

$ aws s3 mb s3://demo-bucket-baba-copied –region us-east-2
$ aws s3 mb s3://demo-bucket-baba-moved –region us-east-2
$ aws s3 cp s3://demo-bucket-baba s3://demo-bucket-baba-copied/ –recursive
$ aws s3 mv s3://demo-bucket-baba s3://demo-bucket-baba-moved/ –recursive
$ aws s3 ls
2020-11-04 14:39:50 demo-bucket-baba
2020-11-04 15:44:28 demo-bucket-baba-copied
2020-11-04 15:44:37 demo-bucket-baba-moved
$ aws s3 ls s3://demo-bucket-baba/

If all the commands are run successfully, then the original bucket should be empty at the end (as all the files have now been moved).

Note

In the certification exam, you will not find many questions on bucket- and object-level operations. However, it is always better to know the basic operations and the required steps.

  • The buckets must be deleted to avoid costs as soon as the hands-on work is finished. The bucket has to be empty before you run the rb command:

$ aws s3 rb s3://demo-bucket-baba
$ aws s3 rb s3://demo-bucket-baba-moved
remove_bucket failed: s3://demo-bucket-baba-moved An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: The bucket you tried to delete is not empty

  • The demo-bucket-baba-moved bucket is not empty, so you couldn’t remove the bucket. In such scenarios, use the –force parameter to delete the entire bucket and all its contents, as shown here:

$ aws s3 rb s3://demo-bucket-baba-moved –force
$ aws s3 rb s3://demo-bucket-baba-copied–force

If you want to delete all the content from a specific prefix inside a bucket using the CLI, then it is easy to use the rm command with the –recursive parameter.

  • Let’s take an example of a bucket, test-bucket, that has a prefix, images. This prefix contains four image files named animal.jpg, draw-house.jpg, cat.jpg, and human.jpg.
  • Now, to delete the contents inside the images, the command will be as follows: aws s3 rm s3://test-bucket/images –recursive
  • The bucket should now be empty.

In the next section, you are going to learn about object tags and object metadata.