AWS Spot Fleet

What is the Spot Fleet in AWS EC2 Instance?

Before exploring Spot Fleet in AWS, we need first to understand Spot Instances.

What are Spot Instances?

AWS has different regions and different availability Zones comprising multiple data centers. These data centers have a great capacity to serve numerous requests. Most of the time, the entire capacity of the data centers is not utilized due to differences in demands. Instead of keeping the resources idle, they provide support for spot instances, with an almost 90% discount on price compared to on-demand instances.

When the bidding price goes above the market, Spot Instances can be terminated by AWS anytime. Hence it needs to be managed cautiously. Some of the features of EC2 Spot Instances are:

• Spot Instances uses the unused compute capacity of the cloud.
• They are available 90% less price than the on-demand instances.
• These instances can be used to test applications, as the duration of spot instances is short.
• The pricing increases if the idle compute capacity of the cloud decreases.
• If AWS terminates the spot instance, then we don't need to pay any amount for using the spot instances.
• In cases where the user terminates the spot instances, only a minimal amount is charged.
• We can compare the prices of spot instances to the stock market prices. Here also, we can bid for the prices of spot instances.
• If the prices we bid for the instances match the demand, then we can use the instance.

What is Spot Fleet?

Spot Fleet in AWS is a collection of Spot Instances. They are also used as on-demand instances optionally. The Spot Fleet API or CLI can be used by applications to request Spot Fleet. The prices of Spot Instances vary each hour. So, EC2 tries to balance the capacity and fulfill the request according to the predefined values. We must define a maximum price per instance hour and the desired target capacity to request a Spot Fleet. It automatically chooses the lowest-priced instances to maintain and fulfill the desired target which is set.

What Problems does Spot Fleet Solve?

Some of the fundamental problems which Spot Fleet solve are:

• It solves the problem where the spot instances are terminated on short notice, which causes problems in various applications.
• The Spot instance availability is very volatile. It has a high risk that the spot instances of the organization will only be available for a short time.
• Spot Fleet's goal is to utilize the instances of different instance types and availability zones for the spot instances.
• It helps the user use a spot instance by providing an on-demand instance as a backup if it is unavailable.
• Using scripts, Spot Fleet should execute when an instance is terminated and launch the number of instances specified in the target.

Spot Fleet Request Types

Let's discuss request types of Spot Fleet in AWS. There are two types of Spot Fleet Requests.

Request

When we configure the request type as request, Spot Fleet sends an asynchronous one-time request for the desired capacity. If capacity is reduced due to Spot interruptions, then the Spot Fleet doesn't try to replenish Spot Instances. Clear the Maintain target capacity checkbox to create a Spot Fleet of type request.

Maintain

When we configure the request type as maintain, Spot Fleet places an asynchronous request of desired capacity. In this request type, Spot Fleet maintains capacity automatically whenever any interrupt happens in Spot Instances. Select the Maintain target capacity checkbox to create a Spot Fleet of type maintain.

Spot Fleet Configuration Strategies

AWS Spot Fleet is a cluster of Spot Instances and optionally On-Demand Instances. It tries to launch the number of instances to meet the specified target capacity in the Spot Fleet request. If any instances are interrupted, it tries to maintain the target capacity.

Spot Fleet in AWS provides the feature to pay for the spot instance. We can set a maximum amount per hour and spot the fleet launches the instances until it reaches the maximum amount.

A set of EC2 instances that are unused with the same instance type, OS, Availability Zone, and network platform makes a Spot capacity pool. When we request Spot Fleet for the instances, it selects the Spot capacity pools to fulfill the request. So, the instances come from the selected pools. These are the Spot Fleet configuration strategies we should follow:

• Plan a Spot Fleet request
• Allocation strategies for Spot Instances
• Attribute-based instance type selection for Spot Fleet
• On-Demand in Spot Fleet
• Capacity Rebalancing
• Spot price overrides
• Control spending
• Spot Fleet instance weighting

Work With Spot Fleets

Let's get some hands-on demos with Spot Fleet in AWS. We will know about its request states, health checks, and permissions. Then, we will create a Spot Fleet request.

Spot Fleet Request States

There are six states in the Spot Fleet request. They are: submitted, active, modifying, cancelled_running, cancelled_terminating, and canceled. This figure shows how the transitions between the request states happen. If the Spot Fleet limits exceed, the request is immediately canceled.

Spot Fleet Health Checks

Spot Fleet in AWS evaluates the condition of Spot instances in the fleet every two minutes. It uses the health check service provided by AWS to check the Spot instances. The health check status is either healthy or unhealthy.

The instance is unhealthy when the instance status check or the system status check is impaired for three continuous health checks. AWS provides the functionality to replace unhealthy Spot Instances.

Spot Fleet Permissions

IAM permissions are required to create or manage a Spot Fleet. AWSServiceRoleForEC2SpotFleet and AWSServiceRoleForEC2Spot, and aws-ec2-spot-fleet-tagging-role roles are created when we use EC2 console to create Spot Fleet. These roles permit to request, launch, termination, and tagging of resources. We must add these permissions to the user when using the AWS CLI.

Following are the instructions to create the role and grant the permissions:

• Grant permission to IAM users for Spot Fleet
• Service-linked role for Spot Fleet
• Service-linked role for Spot Instances
• IAM role for tagging a Spot Fleet

In this demo, we grant permission to an IAM user for Spot Fleet. Follow these steps:

1. Open the IAM console after login into the AWS account using this link.

2. Choose Policies from the navigation menu and click Create policy.

3. Choose JSON, replace the content with the following content, and click on Next:Tags.

4. Click on Next:Review.

5. Enter a policy name. Scroll down and click on Create policy.

6. The policy is created. Select the created policy, click on Actions, and choose Attach.

7. Choose the user and click on Attach policy.

8. We successfully attached the policy to the user.

We can create a spot fleet request using the console or AWS CLI.

Create a Spot Fleet Request Using Console

Follow these steps to create a spot fleet request using the console.

1. Open the Spot console by clicking on this link. Click on Request Spot Instances.
2. Choose Manually configure launch parameters in launch parameters. Choose an AMI image and pick a key pair(optional)
3. Under Target capacity, provide the number of instances. Choose the VPC.
4. Scroll down and review Your fleet request at a glance. Click on JSON config to download a copy of the launch configuration. Click on Launch.
5. A warning appears as we have yet to use the key pair. Mark it and click on OK.
6. The Spot Fleet request is successfully created and launched as a spot instance.
7. Click on the created request id to show the description of the Spot request.
8. Scroll down. We can use the Create Tag button to tag a spot fleet.
9. Click on the Spot request or return to the Spot request page. Select the Spot Fleet and Click on Actions. It shows the option to Modify target capacity. This option can modify the target capacity in the Spot Fleet request.
10. Click on Cancel request to cancel the Spot Fleet.
11. It will ask for the termination of the launched instance. Mark the checkbox and click on Confirm.
12. The Spot fleet request is canceled.

Spot Fleet Request in AWS CLI

We can create and manage the Spot Fleet using AWS CLI by the following command:

1. We can create the Spot Fleet request using the JSON file by using this command.

2. We use the following command to tag a spot fleet request.

3. Run this command to describe the Spot Fleet request.

4. We can also modify the Spot Fleet request using the CLI using this command.

5. To cancel the specified Spot Fleet request and terminate the instances, we use the cancel-spot-fleet-requests command.

CloudWatch Metrics for Spot Fleet

AWS provides us with the Amazon CloudWatch metrics, which we can use to monitor the Spot Fleet.

We can view the CloudWatch metrics from the Amazon Cloudwatch console. Follow these steps:

• Open the Cloudwatch console using this link).
• Choose the Metrics and click on EC2 Spot.
• It will ask to filter the metrics by dimension. Let's choose the Instance type.
• Select any FleetRequestId and view its metrics.

Spot Fleet Metrics

The following table shows the AWS/EC2Spot namespace metrics for the Spot Instances.

Spot Fleet Dimensions

We can use the following Spot Fleet dimensions to filter the data for the Spot Fleet.

• AvailabilityZone - It filters the data by Availability Zone.
• FleetRequestId - It filters the data by Spot Fleet request.
• InstanceType - It filters data according to the instance type.

Automatic Scaling for Spot Fleet

Automatic scaling refers to the capacity of Spot Fleet to automatically launch and terminate the target capacity based on demand. This feature helps Spot Fleet to scale in or out within the specified range.

Spot Fleet supports this automatic scaling.

• Target tracking scaling - This scaling scales up and down according to the target capacity.
• Step scaling - In this scaling, increase or decrease the current capacity of the fleet based on step adjustments.
• Scheduled scaling - It increases or decreases the fleet's capacity on a fixed schedule based on time and date.

IAM Permissions for Spot Fleet Automatic Scaling

AWS manages the automatic scaling using services like Amazon EC2, Amazon CloudWatch, and Application Auto Scaling APIs. It needs IAM permission for the services which support dynamic scaling.

We can create a new policy in the IAM console for providing permission. Use this JSON file to create the policy.

Spot Fleet Limitations

Spot Fleet still needs to improve in a few terms. So, the Limitations of Spot Fleet are:

• Spot Fleet does not support proper sizing of containers. So, It has limited support for containers.
• Spot Fleet does not support proactive instance auto-recovery.
• When a spot instance fails, Spot Fleet does not allow automatic failover to an on-demand instance. So, It cannot fall back on on-demand automatically.
• Spot Fleet does not ensure persistent IP. But if the instance or fleet request type is maintained, then IP persistence supports.
• There is no guarantee of storage persistence and prediction of instance termination in Spot Fleet.