AWS Transfer Family

What is AWS Transfer Family?

The AWS Transfer family is used to share data from inside the AWS Storage solutions with third parties. Customers can download or upload data from their on-premise server to AWS storage solutions such as Amazon Simple Storage Solution (S3) or Amazon Elastic File System (EFS) without making any changes to their application or workflow. With AWS Transfer Family, customers don't have to manage any infrastructure since the file server provisioned by the customer will be managed by AWS.

Features of AWS Transfer Family

Identity Management

The AWS Transfer family supports multiple identity solutions. Users can authenticate using any one of the below identity services.

AWS Transfer Family Service Managed

• Simple Key based authentication

AWS Directory Service or AWS Managed Microsoft AD

• Microsoft AD—Customers who need password-based authentication can opt for this service.

Custom Identity Provider

• Amazon API Gateway and AWS Lambda are used for providing custom identity solutions as per the customer’s requirements.

Managed Workflow for Secure File Transfer

AWS provides several prebuilt API actions, allowing clients to choose between copying, deleting, and tagging actions for their files. In addition to that, customers will also have the custom option for lambda invocation for file processing, which cannot be done by the prebuilt API.

Monitoring File Server Metrics

Amazon Cloudwatch provides the following monitoring metrics for the provisioned file server in the AWS Transfer Family. so that customers get to visualize the data transfer in and out.

Elastic Resource

The AWS Transfer Family File server has built-in auto-scaling capabilities. So, depending on our file transmission, the file server on the backend will dynamically scale in and scale out. According to AWS, the endpoints that we provide for the file server are designed to be available 24 x 7 for 365 days.

Native AWS Service

Once the data/files are uploaded to the AWS storage solution such as Amazon S3 or Amazon EFS,

• Those files can be encrypted by the AWS Key Management Service.
• Those files uploaded to S3 can be used as a source for the data lake.
• Many AWS Services, such as AWS Translate and Amazon Athena, can be used for file processing once the file is uploaded to Amazon S3.

How AWS Transfer Family Works?

• Initially, we need to set up a File Transfer Server with our desired protocol, such as SFTP, FTPS, or FTP. Clients can download or upload files to AWS storage services such as EFS or S3 once the file server comes online.
• From the storage service, other processes can pick up that file, such as Data Lake.
• From the data lake, customers can perform analytics and machine learning, or customers can directly put their data into the database for further access and processing
• All these file transfers in and out can be achieved without changing any client’s or user’s application or workflow.
• The file server will be managed by AWS, so customers don't have to worry about managing the infrastructure.

Use Cases for AWS Transfer Family

The AWS Transfer Family is mainly used for the three below-mentioned use cases:

How to Get Started with the AWS Transfer Family?

Now we are going to create one SFTP file server on the console. There are eight steps involved in creating the file server.

• Go to the AWS management console and enter the AWS Transfer Family in the search box and click the AWS Transfer Family.

  

• In the AWS Transfer Family Console, click the Create server button.

  

Step - 1:
Choose Protocol Options

• Select the SFTP protocol as mentioned in the below image.

Step - 2:
Identity Providers

• Select the Service Managed option among the three options.

Step - 3:
Endpoint Configuration

• Endpoint Type: Publicly Accessible

• Custom Hostname-None

  

Step - 4:
Domain Choice

• Select Amazon S3

  

Step - 5:
Create a Workflow

• Go to the AWS Transfer Family console, click the workflow option in the left navigation pane, and click the Create workflow button.

• Enter the following description: demo-workflow

• Click the Add Step button to add the nominal step.

  

• In the nominal step, select and enter the below options. Select Copy File

  

  Enter the Name: workflow-copy-step Destination Bucket: bucketest12321 (Select the bucket in your account of your choice.) Key: /

  OverwriteExisting: FALSE and click "Create workflow".

  

• Review the step and click Create step

  

• Now the workflow is created.

Step - 6:
IAM Role Creation

• Create one IAM role with S3 full access.
• IAM role name: s3-full-access-role

Reference for creating an IAM role

Step - 7:
Additional Options

• Select the ‘create a new role’ option and enter the showing workflow and IAM role which we created in the last step and click the Next button.

  

  

Step - 8:
Review and Create

• Click Create Server after reviewing the details.

• We can see the newly created file server showing as Online. We can also add users and start transferring the files.

AWS Transfer Family Pricing

There are three components involved in determining the pricing of the AWS Transfer family.

They are,

1. Endpoints enabled for Protocol per hour
2. Data uploaded in Gigabytes
3. Data downloaded in Gigabytes

SFTP Pricing

FTP Pricing

FTPS Pricing

AS2 Pricing

Benefits of AWS Transfer Family

• As a fully managed file transfer (MTS) service, the AWS Transfer family enables the secure transfer of files into and out of Amazon S3 and Amazon EFS.
• AWS will operate and manage all the infrastructure necessary to maintain high availability and performance for the file server
• Customer's files are durably stored in Amazon S3 or Amazon EFS which helps us to enable to use of those files with CRM, ERP, and web-serving applications as well as home directories and developer tools
• We can also process the file available in S3 in our data lake with any analytics or machine learning service to extract business insights.
• There is no upfront cost, so customers only pay for the files processed.
• Using IAM policies, customers can provide granular file access permissions.
• AWS also supports managed file transfer for multi-region backup and disaster recovery use cases for enterprises.

Companies Using AWS Transfer Family

• AWS Transfer Family is used majorly in Financial service industries.

• Companies such as Verisk, Finra, and Whole Foods Market adopted AWS Transfer Family for analytics, data migration, and application integration purposes respectively.

• Other customers include Finra, Blutv, Discover,elula, Liberty Mutual Insurance, Pearson, Myriota, WallaNews, Ohpen, OpenGamma, Belong, SOCAR, Veeva, Zillow, ThinkCX, Kontor, Celgene, QRT and New Media.