Amazon Simple Workflow (Amazon SWF)

What is Amazon Simple Workflow Service or AWS SWF?

More and more applications are using asynchronous and decentralized computation. The main reason for utilizing this method is the scalability of such systems. Developers can deploy and scale out individual portions of the program independently if the application's load grows by creating autonomous distributed components. The availability of cloud services is another driver.

Application developers must blend their current on-premises assets with new cloud-based assets as they begin to benefit from cloud computing. The inherent scattered character of the process being modeled by the application is still additional justification for the asynchronous and distributed approach; for instance, the automation of order-fulfillment organizational processes may span numerous systems and human jobs.

Such application development might be challenging. You must manage the higher latencies and unreliability that come with distant connections and coordinate the operation of several dispersed components. You would normally need to design extensive infrastructure, such as message queues and databases, as well as the intricate logic to synchronize them.

AWS SWF, which offers a programming paradigm and infrastructure for coordinating dispersed components and reliably retaining their execution state, facilitates the development of asynchronous and distributed applications. You are free to concentrate on developing the features that make your application stand out by trusting Amazon SWF.

AWS SWF Concepts

The following sections present the key concepts necessary to comprehend Amazon SWF processes:

SWF Workers and Deciders

• Workers are the programs communicating with the Amazon SWF to get tasks, process the tasks received, and provide the results.
• The decider is software that coordinates activities like ordering, concurrency, scheduling, and so on based on the application logic.
• Workers and deciders are both hosted on cloud infrastructure such as Amazon EC2 or computers behind firewalls.
• Deciders maintain a continuous picture of task progress and begin new tasks, whereas Amazon SWF store's jobs and allocates them to workers to process.
• Amazon SWF assures that the work is only issued once and that it is never replicated.
• Workers and Deciders do not need to keep track of the execution state since Amazon SWF keeps track of it indefinitely. Workers and decision-makers operate autonomously and rapidly.

SWF Domains

• Domains are containers that separate different kinds, executions, and task lists within the same account.
• Workflow, activity categories, and workflow execution are all domain-specific.
• A domain may be registered using the AWS Management Console or the RegisterDomain operation in the Amazon SWF API.

WORKFLOW:

• A workflow is a collection of operations that work together to achieve a goal and logic that coordinates the processes.
• A process, for example, may accept a client order and do whatever operations are required to fulfill it.

SWF ACTIVITIES:

• You must specify each essential activity when creating an Amazon SWF process. Each action is subsequently registered as an activity type with Amazon SWF.
• When you register the activity, you give it a name, a version number, and some timeout settings based on how long you expect it to take.

AWS Flow Framework

• The AWS Flow Framework is an enhanced SDK for writing distributed, asynchronous programs that can run as workflows on Amazon SWF. It is available for the Java programming language, and it provides classes that simplify writing complex distributed programs.
• With the AWS Flow Framework, you use preconfigured types to map the definition of your workflow directly to methods in your program.
• The AWS Flow Framework supports standard object-oriented concepts, such as exception-based error handling, which makes it easier to implement complex workflows. Programs written with the AWS Flow Framework can be created, executed, and debugged entirely within your preferred editor or IDE.

ACTIVITY MANAGER:

• An activity worker is a program that receives activity tasks, executes them, and returns the results.
• It should be noted that a person might complete the job, in which case the person would utilize the activity worker software for task reception and disposition.
• The decider directs the workflow by receiving decision tasks from Amazon SWF and responding back to Amazon SWF with decisions. A decision represents an action or set of actions that are the next steps in the workflow. A typical decision would be to schedule an activity task.
• Decisions can also be used to set timers to delay the execution of an activity task, to request cancellation of activity tasks already in progress, and to complete or close the workflow.
• The mechanism by which both the activity workers and the decider receive their tasks (activity tasks and decision tasks respectively) is by polling the Amazon SWF service.
• Amazon SWF informs the decider of the state of the workflow by including, with each decision task, a copy of the current workflow execution history. The workflow execution history is composed of events, where an event represents a significant change in the state of the workflow execution. Examples of events would be the completion of a task, the notification that a task has timed out, or the expiration of a timer that was set earlier in the workflow execution. The history is a complete, consistent, and authoritative record of the workflow's progress.
• Amazon SWF access control uses AWS Identity and Access Management (IAM), which allows you to provide access to AWS resources in a controlled and limited way that doesn't expose your access keys. For example, you can allow a user to access your account, but only to run certain workflows in a particular domain.

EVENTS:

• The workflow execution history is made up of events, each of which signifies a major change in the workflow execution's state. 
• The completion of a job, notice that a process has timed out, or the expiry of a timer set earlier in the workflow execution are all events.

Execution of the Workflow

The following steps outline how to create and run a process in Amazon SWF:

• Create activity workers to carry out the workflow's processing phases.
• Create a decider to implement your workflow's coordination logic.
• Register your Amazon SWF operations and process.
• You may do this step either programmatically or through the Management Console.
• Begin your activities with employees and decision-makers.
• These actors can operate on any machine connecting to an AWS SWF endpoint. To host a decider or activity worker, you may employ EC2 instances, servers in your data center, or even a mobile device. When the decider and activity workers are launched, they should begin querying Amazon SWF for assignments.
• Begin one or more workflow executions.
• Executions can be started programmatically (or) using the Management Console.
• Every execution runs separately, and each can be given its input data set. Amazon SWF sets the original ruling job when an implementation is initiated. As a result, your decider begins to generate decisions that initiate activity tasks.
• Use the Management Console of AWS to view workflow executions.
• You may filter and examine full information of both running and completed executions. 

Features of AWS SWF

Scalability Amazon SWF automatically adapts the resources to match the use of your application. When you add additional cloud processes or enhance the complexities of the workflows, there is no need for manual management of the workflow service.

Reliability Amazon SWF operates on AWS's extremely available data centers, so state tracking is accessible whenever applications want it. Amazon SWF saves the tasks, transmits them to the appropriate application components, and monitors their progress.

Simplicity The new cloud workflow internet service, Amazon SWF, eliminates the complexity of existing workflow methods and process automation. It removes the need for developers to handle the automation process, allowing you to focus on the application's unique capabilities.

Logical Division The flow of the underlying task's sequential logic is logically separated from the work units that carry business logic by Amazon SWF. You may quickly manage the business logic by changes in the business needs without worrying more about task distribution, state machinery, or flow control.

Flexible You may alter the application program logic in any computer language and execute the modified application components on the web or on-premises using Amazon SWF.

AWS SWF Pricing

Pay just the amount you use. There is no set minimum payment. Utilize the Simple Monthly Calculator to project your monthly expenses.  Every charge is metered daily and billed once a month.

Free Tier Each month, Amazon SWF provides the following to both new and current users as part of the AWS Free Usage Tier:

• You can start 1,000 process executions for free.
• You may free use 10,000 tasks, timers, signals, and markers.
• 30,000 workflow days are available for free usage. Each 24-hour period during which execution is either open or maintained in AWS SWF is a workflow day.

Executions of Workflow A workflow is a collection of actions carried out in a certain sequence.  Every time a workflow is carried out, it is regarded as a separate workflow execution. You pay for a workflow to run for the first 24 hours when a workflow is initiated. Workflow execution's initial 24 hours are free.

Start a Workflow Execution Region: US East (Ohio)

• $0.00 for first 1,000 workflow executions
• $0.0001 per workflow execution above the free tier

Open and Retained Workflows Region: US East (Ohio)

• $0.00 for first 30,000 combined open workflow and retained workflow days (24-hour period)
• $0.000005 per 24-hour period that a workflow is retained above the free tier
• $0.000005 per 24-hour period that a workflow is open above the free tier

Tasks, Markers, Timers, and Signals Until a workflow execution is completed, your worker or client machines will execute additional tasks and may add “markers” (custom workflow execution log entries), start timers, or receive signals. For each of these, you pay:

Region: US East (Ohio)

• $0.00 for the first 10,000 combined tasks, signals, timers, markers
• $0.000025 per task, signal, timer, or marker above the free tier

Data Transfer Region: US East (Ohio)

Data transfer “in” and “out” refers to transfer into and out of Amazon SWF. Data transferred between Amazon SWF and other AWS services (e.g., Amazon EC2, where workers and deciders may be deployed) within a single region is free of charge (i.e., $0.00 per GB). Data transferred between Amazon SWF and other AWS services in different regions will be charged at Internet Data Transfer rates on both sides of the transfer.

• Your free usage is calculated each month and automatically applied to your bill – free usage does not accumulate.
• As part of AWS’s Free Usage Tier, AWS customers will receive free 15 GB of data transfer each month aggregated across all AWS services for one year.
• Data transfer out rate tiers aggregate outbound data transfer across Amazon EC2, AWS Lambda, Amazon S3, Amazon RDS, Amazon SimpleDB, Amazon DynamoDB, Amazon SWF, Amazon SQS, Amazon SNS, AWS Storage Gateway, and Amazon VPC.

Example Cost Calculation

To illustrate Amazon SWF pricing, consider a simple video processing application with three tasks: download a video, encode it, and store the encoded video in Amazon S3. This application is used to encode 10,000 videos, each of which comprises a unique workflow execution. The workers and deciders in this application are all running on Amazon EC2 in the same region as Amazon SWF.

The cost incurred for running 10,000 workflow executions of this workflow daily in US East (N. Virginia) is $1.75 as follows:

$1.00 for starting 10,000 workflow executions (10,000 x $0.0001) $0.75 for initiating tasks ($0.000025 per task x 3 tasks per workflow execution x 10,000 workflow executions)

We assume a video can be encoded and uploaded to S3 in less than 24 hours. As a result, the workflow execution completes within 24 hours of starting, and there is no ongoing workflow execution charge.

Suppose you also want Amazon SWF to retain information about the encoding workflow executions for seven days after completion. The cost for retaining information about the workflow executions is $0.35 ($0.000005 per day per execution x 7 workflow days of retention x 10,000 workflow executions).

In total, the Amazon SWF cost to coordinate the encoding and uploading of 10,000 videos and to retain information about the associated workflow executions and processing steps for seven days is $2.10 ($1.75 for running the workflow executions + $0.35 for retaining information on them).

AWS GovCloud Region AWS GovCloud is an AWS Region designed to allow `U.S. government agencies and contractors to move more sensitive workloads into the cloud by addressing their specific regulatory and compliance requirements.

Getting Started with AWS SWF

• First of all, login to your AWS account, and in the search tab, search for SWF. Then you will be moved to the AWS Simple Workflow Service Dashboard as shown below.
• You will be shown two options: "Launch sample walkthrough" and "Create a New Domain."
• You will be shown five steps: create a domain, registration, deployment, run an execution, and next steps. Click on "Get Started."
• Enter the Domain Name as shown below and click on the continue button.
• Then, the registration process will take some time. After that, click on the continue button.
• Now you will get two options "Complete walkthrough without deployment" and "Deploy on Amazon EC2 micro instance". Once you enter all required information as shown below, click on the acknowledgment checkbox and Click on the "Continue" button.
• Finally, your instance will get created and shown by running and moving to the next steps, as shown below.

For more information, you can just go through this official docs file of AWS.

Benefits of AWS SWF

Logical Discrimination The stepwise logic control sequence for your background task is separated from the real work units that include your distinctive business logic by Amazon SWF. This enables you to manage, maintain, and expand your application's "state machinery" independently of the essential business logic that makes it unique. You may quickly modify the application logic as the business needs shift without worrying about core state mechanism, job distribution, and flow control.

Reliable The state monitoring and task processing engine for Amazon SWF is accessible anytime applications require it since it runs inside Amazon's high-availability data centers. The tasks are stored redundantly by Amazon SWF, which also distributes them to app components with reliability, monitors their progress, and retains their most recent state.

Simple Amazon SWF, a fully managed cloud process flow web service, removes the complexities of custom-coded flow process solutions and process automation systems. This frees developers from having to handle the underlying plumbing of process automation, allowing them to concentrate their efforts on the unique feature of their application.

Scalable Amazon SWF adapts to your application's consumption. There is no need for manual management of the flow process services if you add additional cloud process operations to your app or raise the sophistication of the process flows.

Flexible Amazon SWF allows you to develop your software applications, coordinate logic in any computer program, and execute it in the cloud.

Encoding Video AWS S3 and EC2 are used for video encoding. Large recordings are posted to AWS S3 in pieces in this use scenario. The upload of pieces must be tracked. Failures may arise throughout this operation due to encoding problems in one or more chunks. Such problems must be noticed and dealt with using Amazon SWF.

Data Center Relocation Workers encapsulating components operating in the data center can be combined with those operating in the cloud in Amazon SWF-based applications. New employees of the same kind are first installed in the cloud to smoothly transfer a data center worker. The data center workers, as well as the new cloud-based workers, continue to work as normal.

AWS SWF vs. SQS