AWS FSx

What is AWS FSx?

Each of us has a file server running on our system. This file server allows you to manage your files. For example, it allows you to view, edit, save and create files. Suppose you are running business software that references the files on your local file server.

You keep adding new files to your server, and it may run out of space with time. Now a case arises where your business continues to grow even though you have no space for adding new files. What to do now? Here Amazon FSx emerges as a saviour.

Amazon FSx instantly sets up your file storage system in the cloud. Instead of having a dedicated IT team for installing software and hardware units, you can use the Amazon FSx service provided by AWS to give File server services to your clients/consumers.

• Amazon Fsx is known for providing windows-based storage.
• It also provides facilities for machine learning, high computing performance, and electronic design automation (EDA).
• AWS FSx not only sets up the file server but also does the patching and takes care of creating a backup.
• Everything needed can be done using just a few clicks on the AWS Management Console.

AWS FSx File Systems

File system represents the logical structure in which files are placed within a system for easy retrieval and storage. It answers where files are stored in a system and why.

Amazon FSx is not limited to providing service for a single file system. It provides us with four different options to choose from. You are free to choose based on your familiarity with a given file system.

The four different file system services provided by AWS FSx are:

1. Amazon FSx for Windows: Designed for windows based applications.
2. Amazon FSx for Lustre: Used for supporting or computing-intensive workloads
3. Amazon FSx for NetApp ONTAP: Provides access to block storage and comes with data management system features of NetApp.
4. Amazon FSX for OpenZFS: Supports on-premises ZFS (Zettabyte File System) without hindering application code.

Let's know about each of these types in detail.

Amazon FSx for Windows

Amazon FSx for Windows is used to migrate Windows-based applications to AWS cloud storage. It supports integration with Server Message Block (SMB) protocol, Windows NTFS, and Azure Directory.

• You can create DFS (Distributed File Systems) namespaces under this. DFS namespaces are role service. It is used to define the scope. These namespaces can be used for sharing one or more files located on muliple servers in a group.
• You can use AWS Direct Connect and AWS VPN to access your files in the cloud from your local system.
• It uses Azure Active Directory to integrate your file system with your windows environment.

How does it work?

Let's understand in brief the working of AWS FSx for Windows.

• An AWS FSx is created for your windows file system.
• You can configure your files, sharing, and access permissions.
• These files can be accessed by end-users capable of performing read and write operations depending on access.

Amazon FSx for Lustre

Amazon FSx for Lustre is used for organizations or businesses that deal with or need high-performance file systems.

• It is used for Linux-based applications requiring high storage capacity and fast processing speed.
• Lustre was designed to minimize the cost of processing large datasets with high-speed processors.
• It reduces the complexity involved in traditional methods of high-speed processing for Lustre file systems.
• It supports high-performance file systems.

How does it work?

Let's understand in brief the working of AWS FSx for Lustre.

• AWS FSx for Lustre integrates with Amazon S3 to process data sets stored over here using high-performance file systems.
• These files/data are accessed via AWS services like AWS EC2, Amazon ECS, Amazon EKS, etc., or some on-premises clients. They can perform read or write operations on these files.
• It can be accessed by AWS machine learning services as well to generate useful reports.

Amazon FSx for OpenZFS

Amazon FSx for OpenZFS can deliver a million IOPS with extremely low latency of hundreds of microseconds. ZFS stands for Zettabyte File System, known for its volume management capabilities.

• It makes use of AWS Graviton for powering its processors.
• It enables backups of your file system to be stored on S3 with cross-region replication.
• It allows you to migrate data from on-premises ZFS to the cloud without hindering the application code.

How does it Works?

Let's understand in brief the working of AWS FSx for OpenZFS.

• AWS FSx for OpenZFS can be created using the management console, command-line interface, or SDKs provided by AWS.
• These File systems can be configured as per the business needs.
• Other AWS services or on-premises clients can access the files stored here and whoever has access to perform read or write operations.

Amazon FSx for NetApp ONTAP

Amazon FSx for NetApp ONTAP is helpful for a business that demands high-performance SSD storage with latency as low as sub milliseconds.

• It is accessible from any device, including Mac, Windows, Linux, etc.
• It is composed of two main components Storage Virtual Machines and Volumes.
• Here, the infrequently accessed data gets transferred to the capacity pool tier, saving on costs.

How does it Works?

Let's understand in brief the working of AWS FSx for NetApp ONTAP.

• AWS FSx for NetApp ONTAP can be created using the management console, command-line interface, or SDKs provided by AWS. NetApp Cloud Manager can also create it.
• You can optimize the file system by enabling or disabling certain features which help in cost management.
• These files can be accessed via other AWS services or on-premises clients.
• File system applications can be run using multi-protocol(NFS, SMB, and iSCSI) access to data.

Features of AWS FSx

Let's explore some of the features of Amazon FSx.

1. Simplicity

• AWS FSx is easy to use and can be launched using just a few clicks.
• Amazon FSx is a fully managed service. The file servers, storage volumes, and backups are automatically maintained.
• In addition, hardware and software updates, administrative tasks, etc., are all managed by AWS FSx.

2. Performance

• AWS FSx is designed to deliver fast, consistent and high performance.
• They can be scaled without having much on the performance.
• The storage capacity can be increased at any time, with the ability to read and write the files without any latency.

3. Availability

• AWS FSx runs on the cloud and provides access to all features related to cloud services.
• Amazon FSx allows you to store your data in multiple availability zones depending on which region your customers belong to.
• This high availability feature reduces latency and improves customer experiences.

4. Durability

• Amazon FSx provides high durability
• Your data remains protected and unaltered for the long term.
• Instead of maintaining multiple servers, multiple copies of your data are maintained to prevent data loss.

5. Security

• Amazon FSx was designed in the first place to meet high-security standards.
• Amazon FSx encrypts both data in transit and data at rest, which adds to the security of the files maintained on our file server.
• You can even run your file systems server in Amazon VPC (Virtual Private Cloud), which allows only authorized access to your data.

6. Cost Effectiveness

• Amazon FSx follows pay as you go model. You need to pay only for what you use.
• You can scale your file-server system as and when needed and need to pay only for the amount used.
• AWS FSx provides a range of storage options, including SSD (Solid state disk), HDD ( Hard Disk Drive), etc.
• It also provides a facility for removing duplicate data and optimizing costs.

7. Integration with AWS Services

Amazon FSx can be easily integrated with plenty of other AWS services:

• Amazon S3
• Amazon CloudWatch
• Amazon CloudTrail
• AWS KMS
• Amazon SageMaker
• Amazon WorkSpaces
• Amazon AppStream 2.0
• Amazon Elastic Container Service (Amazon ECS)
• Amazon Elastic Kubernetes Service (Amazon EKS)
• AWS Batch
• AWS ParallelCluster

AWS FSx Use Cases

Let's understand a few Use cases of AWS FSx.

1. Migration of Workload to Cloud

• The existing file systems which are difficult to maintain on-premises can be migrated to the cloud.
• Files, databases, and application workloads can be shared and is highly scalable.
• AWS FSx itself maintains the infrastructure.

2. Cutting on Expenditure

• A company named Infor used a cloud-based storage system provided by AWS.
• It reduced its cost by 50%, eliminating the cost involved in managing storage devices and systems.
• Amazon FSx allows us to decide the cost involved by picking up the hardware and software capacities according to our needs.

3. Increase Development

• Amazon FSx aids in the development process by reducing the release cycles.
• It uses infrastructure as a code feature to speed up the development process.
• You can replicate your configuration with zero cost involved. These clones are zero-capacity clones.

4. Simplify Business Continuity

• If we use an on-premises setup, managing backups and scaling infrastructure raises some serious challenges.
• On using AWS, FSx backups are automatically created. The autoscaling feature saves time and cost involved in expanding the infrastructure.
• Data replication to ensure data availability and durability can also be done using a few clicks on the management console.
• It allows us to focus more on the business than on maintaining infrastructure.

5. Accelerate Workloads

• Some the industries, like mass media and entertainment, needs high-performance storage.
• AWS FSx allows these data to be available to end users to view and edit. On top of this, it is accessible from any device Mac, Windows, Linux, etc.
• The files are delivered to the end users with low latency and fully managed sharing policies.

6. Analysis

• The files stored in AWS FSx can be easily integrated with other AWS services
• It can be the data source for the data lake. The data then gets transferred to the data warehouse, from which data can be analyzed, and reports can be generated.
• Various machine learning operations can be carried out on this data to make useful predictions.

Setting up AWS FSx (step-wise)

We got to know the features and use cases of AWS FSx. Now let's try the hands-on experience with setting up the AWS FSx.

We are setting an Amazon FSx for the windows file server in this demo. To create the file system in Amazon FSx, we have to create an AWS-managed directory and an ec2 instance of windows and set the managed directory to the instance. Then we will create the file system and attach it to the windows system.

Create Directory

Let's start with creating the AWS-managed directory. Follows the given steps:

• Sign in to the AWS console, type Directory service in the search bar, and click on Directory service.

• Click on Set up directory to create a new directory.

• Next, choose AWS Managed Microsoft AD, and click on Next.

• Now, select the standard edition and fill in the Directory DNS name.

• Scroll down and set the admin password. Click on Next.

• Choose the VPC and subnet from the option and click on Next.

• Next, Review the information and click on Create directory.

• The creation time is approx 20 to 40 minutes. After that, it shows the directory status as Active.

Setup and Connect Instance to Directory

Once we have created the directory, we need to set up a windows ec2 instance and join the instance to the active directory. It involves the following steps:

• Go to the AWS console, type EC2 in the search bar, and click on EC2.

• Click on Launch instance.

• Fill in the instance's name and click on the windows Microsoft icon.

• Scroll down and select a key pair, VPC and subnet. Leave other fields defaults and Click on Launch instance.

• Our ec2 instance is launched successfully.

• Go to its inbound rules and add a new rule to allow all traffic. Select the instance and click on Connect.

• Click on instances, and it shows the windows instance is running.

• Now click on RDP client.

• Click on Get password.

• Upload the private key file and click on Decrypt password.

• Now, we have to open this instance. Open the Remote Desktop Connection app in your local system and login using your DNS and user names. Click on Connect.

• Enter the password and click on Ok.

• Click on Yes.

• We are now inside our windows instance.

• Open the command prompt and run the command %SystemRoot%\system32\control.exe ncpa.cpl.

• Right-click on Ethernet 2 and click on properties.

• Click on Internet Protocol Version 4(TCP/IPv4), and click on properties.

• From the created directory Service, click on it and see its details. Copy its DNS addresses.

• Choose the use of the following DNS server address, fill in the copied DNS addresses, and click on OK.

• Open the command prompt and run the command %SystemRoot%\system32\control.exe sysdm.cpl.

• Click on Change.

• Select the Domain and fill in the domain name of the directory service. Click on OK. Now log in using the Admin user and password set earlier when creating the Directory service. Click on OK.

• A welcome message is shown.

As we connect our instance with the directory service, we have to restart the instance.

Setup AWS FSx File Server

Let's begin to set up our first AWS FSx for the windows file server by following the steps.

• Login to your AWS account and go to the AWS console. Search for FSx in the search bar and click on FSx.

• Now, click on Create file system.

• Next, select the file system type. Here we choose Amazon FSx for Windows File Server and click on Next.

• Fill in the name of the File system. Select the Deployment type as Multi-AZ, choose the storage type as SSD, and choose the capacity as 32 GiB.

• Select the VPC and VPC security groups in network and security. In windows authentication, choose AWS managed Active directory and select the created directory.

• Leave all other fields as default and click on Next.

• Next, Review and click on Create file system.

• It will take time to create it. After some time, the file system is shown as available. Click on file system id.

• Click on Attach.

• Here, it will show how to connect it to the windows server. We completed the Prerequisites earlier in this demo. So now we have to open our windows instance using Remote Desktop Connection. Open the command prompt.

• Copy the command from the last image and run it on the command prompt. It will ask for the username and password. After that, it shows the command is completed successfully.

• Open the file explorer, and we will see a new storage location in it, which is from Amazon FSx.

We successfully attach our FSx file system to the windows server.

Benefits and Limitations of AWS FSx

Benefits of using AWS FSx

Some of the benefits of using AWS FSx are:

1. Pocket-friendly

• Amazon FSx is a friendly choice, even for small businesses.
• It helps you optimize your cost depending on your chosen storage, storage capacity, regions you want your file server to be available, etc.
  • Amazon FSx allows you to choose between SSD or HDD storage devices.
  • It allows you to choose the hardware capacity and performance ability
  • AWS FSx also allows you to choose single-region availability or multi-region availability.

2. Protected

• Amazon FSx protects you from:
  • component failure
  • hardware failure
  • data loss
• It continuously monitors the server system for failures and makes required updates in the infrastructure automatically when needed.

3. Fully managed

• Amazon FSx is capable of providing management services for various kinds of file systems.
• Data duplication, backups, migrations, availability, etc., are all managed by AWS FSx.
• It is capable of scaling automatically to serve the demand.

4. Hybrid Enabled

• Amazon FSx does not limit your data to just storage and management. It has a lot more to offer.
• You can integrate your data stored in AWS FSx with other AWS services to analyze, perform machine learning operations, generate reports, etc.

5. Highly Available

• To ensure the high availability of your data stored in AWS FSx, it automatically replicates (creates multiple copies) your data across multiple availability zones.
• Backups are also created across different regions to prevent data loss.

Limitations of using AWS FSx

Knowing the benefits, let's discuss some limitations associated with using AWS FSx.

• Usage capacity in Amazon FSx is limited for each service, and you need to pay extra for additional usage.
• The default limit for the number of file systems is 100, which can be increased with additional charges.
• The default storage capacity for all file systems is limited to 512 TiB.
• The default throughput capacity for all file systems has a constraint of 10 GBps which can be increased when needed.
• 500 user-initiated backups can be maintained by default for all the file systems.

AWS FSx Pricing

The pricing for AWS FSx depends on the file system used, so let's discuss each separately.

Amazon FSx for Windows File Server Pricing

Under the AWS FSx Windows File server, no minimum setup charges are involved, and you only need to pay for what you use.

Storage Capacity

• The price for the storage depends on the following:
  • Type of deployment, single or multiple availability zones
  • Type of storage used, HDD or SSD.
• For SSD storage capacity, you are charged monthly with $0.130 per GB, which decreases to a monthly charge of$0.065 per GB if the data deduplication feature is enabled.
• For HDD storage capacity, you are charged monthly $0.013 per GB, which decreases to a monthly charge of$0.0065 per GB if the data deduplication feature is enabled.

Throughput Capacity

• Throughput price depends upon whether you opt for single or multi-AZ.
• You are charged monthly $2.200 per MBps of throughput capacity.
• Throughput price remains unaffected even if you enable the data deduplication feature.

Backups

• Both automatic and user-initiated backups are charged under this.
• For backups, you are charged $0.050 per GB monthly
• It reduces to $0.025 per GB with an enabled data deduplication feature.

Amazon FSx for Lustre Pricing

The AWS FSx for Lustre follows the same. No minimum setup charges are involved, and you only need to pay for what you use.

File System Storage

Persistent File System

• This is used for long-term storage. Data backup exists to prevent data loss.
• The table below shows the monthly charge per GB with various storage options:

Scratch File System

• This is used for temporary storage. In this case, data is not replicated, and no backup is maintained if the file server fails, data gets lost.
• You are charged monthly $0.140 per GB for using scratch file systems. It gets reduced to$0.070 per GB, enabling the data deduplication feature.

Backup Storage

• For backups, you are charged $0.050 per GB monthly
• It reduces to $0.025 per GB with the data deduplication feature enabled, as only the most recent data gets saved.

Amazon FSx for OpenZFS Pricing

Under the AWS FSx for OpenZFS, no minimum setup charges are involved, and you only need to pay for what you use.

Storage Capacity

• For SSD storage capacity, you are charged monthly $0.090 per GB, which decreases to a monthly charge of$0.045 per GB if the data deduplication feature is enabled. These charges are applicable for Single AZ file systems.
• $0.0060 per IOPS is charged monthly for SSD IOPS. By default, there are three SSD IOPS for each GB of SSD storage.

Throughput Capacity

• It is measured in MB/s-months.
• You are charged monthly $0.260 per MBps of throughput capacity for single Availability Zone file systems.

Backups

• For backups, you are charged $0.050 per GB monthly for single Availability Zone file systems.
• It reduces to $0.025 per GB with an enabled data deduplication feature.

Amazon FSx for NetApp ONTAP Pricing

Under the AWS FSx for NetApp ONTAP, there are no minimum setup charges involved, and you only need to pay for what you use.

Storage Capacity

• For SSD storage capacity, you are charged monthly $0.125 per GB, which decreases to a monthly charge of$0.0438 per GB of data compression and the deduplication feature is enabled.
• Monthly charge of $0.0170 per IOPS is set for SSD IOPS. This remains unaffected by data compression and deduplication feature.

Capacity Pool Usage

• Standard capacity pool storage costs $0.0219 per GB monthly, which is reduced to$0.0077 per GB with data compression and deduplication features enabled.
• For Capacity pool read requests, $0.0004 is charged per 1000 requests.
• For Capacity pool write requests, $0.0050 is charged per 1000 requests.

Throughput Capacity

• It is measured in MB/s-months.
• You are charged monthly $0.720 per MBps of throughput capacity, which remains unaffected even if you enable storage compression and data deduplication.

Backups

• For backups, you are charged $0.050 per GB monthly.
• It reduces to $0.0175 per GB with storage compression and data deduplication feature enabled.

AWS FSx vs EFS

Let's look at the differences between AWS FSx and AWS EFS.

Purpose

• AWS EFS is a serverless, cloud-based file management system.
• On the other hand, AWS FSx is a fully managed storage accessible via some protocols.

Availability

• Every data stored in standard AWS EFS is replicated and stored across multiple AZ.
• AWS FSx allows you to choose between Multi-AZ and Single-AZ depending on the need and saving costs.

Data encryption

• Both Amazon FSx and EFS provides data encryption facility for data-in-transit and data at rest.
• For data at rest, encryption is done using AWS KMS for encryption keys.
• For data in transit AWS EFS uses TLS 1.2, whereas AWS FSx uses SMB Kerberos session keys.

Support

• AWS EFS is used for Network File Systems (NFS) and Linux-based workloads.
• AWS FSx, on the other hand, supports various file systems depending on varying requirements.

Classes

• AWS EFS offers four storage classes depending on how frequently you need the data.
• AWS FSx offers four different file system types depending on the type of system you are familiar with and want to use.