Part 4. Shut down your cluster

28.

Since you have modified some of the files in the /data directory, you will want to save a new snapshot of the /data directory so you can continue from where you left off the next time you create a new cluster. You can do this by running the update_snapshot.sh script from a Terminal window on your computer (replace ~/Documents/EE3801/PyHipp with the path to your PyHipp repository):

Note

Windows users: Before running the command below, you will need to convert the Windows DOS format into unix format by doing:

$ dos2unix ~/Documents/EE3801/PyHipp/update_snapshot.sh 

(aws) $ chmod a+x ~/Documents/EE3801/PyHipp/update_snapshot.sh

(aws) $ ~/Documents/EE3801/PyHipp/update_snapshot.sh data 2 MyCluster01

The first command will make the script executable while the second command will start the process of creating the snapshot.

The first augment is the name of the snapshot data, the second number 2 specifies how many similarly named snapshots to keep, while the last argument MyCluster1 specifies the name of the cluster you want to base the snapshot on.

Keep in mind that each snapshot you keep will use up some of your AWS credits. This command might take a while, so you can return once you receive the email notification.

29.

Check that the snapshot was created properly using the following command:

(aws) $ aws ec2 describe-snapshots --owner-ids self  --query 'Snapshots[]'

You should see something like the following:

[
    {
        "Description": "data",
        "Encrypted": false,
        "OwnerId": "xxxxxxxxxxxx",
        "Progress": "100%",
        "SnapshotId": "snap-xxxx",
        "StartTime": "2020-11-04T11:10:40.838Z",
        "State": "completed",
        "VolumeId": "vol-xxxx",
        "VolumeSize": 1000
    },
]

30.

You can add the update_snapshot.sh script to your shell’s path by adding the following line to your shell’s config file (substitute in the path to your PyHipp directory):

export PATH="/Users/your-user-name/Documents/EE3801/PyHipp:$PATH"

Note

In case you are not so sure what is your full path, cd to where your PyHipp is at and do:

$ pwd 

31.

Once you have received the email notifying you that the snapshot has been completed, you can delete the cluster:

(aws) $ pcluster delete-cluster -n MyCluster01

Check that you have no clusters running:

(aws) $ pcluster list-clusters

This should return an empty list. If it does not, you will have to retry your cluster deletion command.

32.

In the future, you will just need to do the following to create a cluster, wait till you receive the notification that the head node is running, and then ssh to the head node:

(aws) $ pcluster create-cluster -c ~/cluster-config.yaml -n MyCluster01 

(aws) $ pcluster ssh -i ~/MyKeyPair.pem -n MyCluster01 

When you are done, you can do the following to create a snapshot, wait till you receive the notification that the snapshot has been completed, and then delete the cluster:

(aws) $ update_snapshot.sh data 2 MyCluster01

(aws) $ pcluster delete-cluster -n MyCluster01