AWS Macie

What is AWS Macie?

As the number of security breaches is on a rise, it's important to have a well-rounded security platform to specifically protect sensitive data like Personal Identifiable Information (PII). Also, when data getting in the AWS Cloud increases, you might be looking for cutting down on manually classifying data and assigning rights, and automating findings.

To resolve this problem, AWS offers its widely used and accepted service called AWS Macie.

By using AWS Macie, you get the ability to detect any data breaches and intrusions. AWS Macie is defined as an AWS service that provides data security and data privacy service by implementing machine learning technology and pattern matching which helps to discover and protect the sensitive data in AWS at scale. Ongoing monitoring and analysis which are captured and stored in the cloud data are used to access and periodically track the account behavior to identify any potential unsecured activity. If any anomaly is detected, AWS Macie can send alerts to the admins as well.

With AWS you become more aware of your sensitive data and the protection level you have. With its dashboard alerts, you get the flexibility to keep a check on track when your data is accessed or moved. Mostly, administrators are unleashing the advantages of AWS Macie to prevent unauthorized access and data leaks with AWS Macie's capability of continuous and constant monitoring and analyses of data.

How Does It Work?

Now we shall be learning how AWS Macie works to discover, detect and protect sensitive data.

When you enable and integrate AWS Macie with your AWS account, AWS Macie will produce the S3 bucket list (for the region where it is enabled) within a few minutes. AWS Macie then starts to monitor by keeping an eye on the S3 buckets’ security and access control. While monitoring, AWS Macie also generates thorough details of all the findings detected ( like any potential illegal accesses or any inadvertent data disclosure in your account) to send alerts to you accordingly.

Now let us understand the three components which enable AWS Macie to discover and protect our sensitive data at scale.

Macie has three main features :

1. Macie Summary Dashboard :
   Once you set up the AWS Macie dashboard, you shall get an overview of how the data is getting accessed and processed. Many findings are displayed on the dashboard like the total number of buckets, objects, and S3 storage consumed. Within the AWS Macie dashboard, you can also segment the S3 buckets into different types based on their accessibility like public, encrypted, and private buckets. We can also classify the buckets based on how they are shared within and outside the AWS account or your organization.
2. Macie Jobs :
   To automatically discover, record, and report sensitive data, we have AWS Macie Jobs where you can create and schedule the sensitive data discovery jobs in Amazon S3 buckets to track the sensitive data discovery and protection.

This job can either be run once for on-demand analysis or can also be scheduled if the requirements (for recurring analysis and monitoring) are to generate reports regularly.

3. Macie Findings :
   AWS Macie Findings is defined as a detailed report for sensitive data residing in the S3 buckets or S3 objects, where all the potential violations in the policy are listed. AWS Macie has two types of findings :

   • Policy findings :
     A detailed report of all the potential policy violations concerning the security and privacy of the Amazon S3 bucket.
   • Sensitive data findings
     A detailed report of all the sensitive data in an Amazon S3 object.

   You can also schedule Amazon CloudWatch Events by integrating it with AWS Macie where you can send all findings and build a custom remediation and alert management system.

   A few examples of policy findings and sensitive data findings are shown below :

Amazon Macie Concepts

As we move along the article, you might come across various terms related to AWS Macie. Let us learn about each concept one by one which will help strengthen our concepts related to AWS Macie.

• Alert :
  An alert represents a notification whenever potential security issues are discovered by AWS Macie. We can find these alerts on the AWS Macie console with a comprehensive detailed record of all the activity that occurred in the last 24 hours.

  • Basic alert :
    All alerts generated after AWS Macie performs its checks to detect security breaches are called basic alerts. We divide these basic alerts into two types:
    • Managed basic alerts :
      These alerts are curated by AWS Macie which we cannot modify. We only get the ability to enable or disable the existing managed basic alerts.
    • Custom basic alerts :
      The Custom basic alerts can be created as well as modified to suit your exact specifications.
  • Predictive alerts :
    All the automatic alerts based on activity happening in your AWS infrastructure that is found to be deviating from the normal activity baseline are classified as Predictive alerts. Here, AWS Macie is continuously monitoring the IAM user and role activity by which it builds a normal behavior model for it. When the activity starts to look different, it generates automatic predictive alerts.

• User :
  A user is defined as the AWS Identity and Access Management (IAM) identity that is responsible for requesting AWS Macie.

• Research Tab :
  With the help of this research tab in the AWS Macie console, you can construct and trigger queries along with conducting in-depth investigative research of your data and activities that AWS Macie is continuously monitoring.

• Data Source :
  The data source is defined as the origin or the location where the set of data resides. We cannot modify the existing CloudTrail events or add new CloudTrail events to the existing list that AWS Macie manages. But we get the ability to enable or disable the supported CloudTrail events, by telling AWS Macie as per the data security process. You can also integrate AWS Macie with the AWS S3 buckets by mentioning the relevant S3 prefixes.

• Automatic Content Classification Methods :
  When you set AWS Macie to start monitoring the data, it makes use of several automatic content classification methods for recognizing and prioritizing the sensitive and critical data to accurately assign business value to it. Each of these classifications has its designated risk level ranging from 1 and 10, where 10 is the highest risk and 1 is the lowest.

Amazon Macie Features

We shall now be discussing the various features that AWS Macie offers for data discovery and protection of your sensitive data in AWS.

Continuous evaluation of the AWS S3 buckets :

You can continuously evaluate the Amazon S3 buckets and generate reports for the S3 resource for all the accounts by implementing AWS Macie. With these reports, we can filter out details, search, as well as arrange the S3 buckets with respect to the metadata variables (like name of the bucket, markers, as well as security controls like accessibility to public or encryption status). You can also generate alerts when any type of S3 bucket (unencrypted, publicly accessible, or shared externally to accounts outside AWS) gets identified.

Sensitive data types are Fully Managed :

With AWS Macie, we get a growing list of sensitive data types including PII and data privacy regulations like GDPR, PCI-DSS, and HIPAA. Various data detection techniques like machine learning are used which continually add to and improve over time.

On-demand and Automated sensitive data discovery jobs at Scale :

By implementing AWS Macie, you can automatically and continuously schedule and run jobs one-time, daily, weekly, or monthly for sensitive data discovery on either all the S3 buckets or a sub-group of objects in the AWS S3 bucket. This way you can automatically track changes (only new or modified) over the bucket.

Sensitive data types are customizable :

With Amazon Macie, you get the capability to include a custom data identifier ( Set of criteria that the user can define to detect the sensitive data) and determine the data types using regular expressions which enable AWS Macie to discover unique sensitive data for your organization.

Comprehensive and functional security and sensitive data discovery findings :

AWS Macie prioritizes its findings (which include details, like the sensitive data type, tags, public accessibility, and encryption status) and automatically writes them to its customer-owned S3 bucket. This helps AWS Macie reduce alert volume and speeds up triage. These findings can be retained for 30-days which are made available in the AWS Management Console or API.

Deployment with few clicks without upfront data source integration :

You can enable AWS Macie with just a single click in the Management Console or one API call, while within a few steps, you can start using AWS Macie across various accounts. Once the AWS Macie is enabled, it starts to generate a continuous AWS S3 resource summary across all of its accounts.

Supports multi-account along with integration with AWS Organizations :

If your organization wants to track any sensitive data discovery and protection, it can do so by implementing the AWS Macie multi-account configuration. Here one AWS Macie admin account is capable of managing and maintaining all accounts for various members which can include creating and administrating sensitive data discovery jobs over a range of accounts. Various accounts are supported as AWS Organizations integrated with AWS Macie in addition to individual accounts. All the combined findings from the security and sensitive data discovery are sent to Amazon CloudWatch Events by aggregating it in the AWS Macie admin account. With this single admin account, we can regulate and integrate the workflow, event management, and integration with AWS Step Functions to automate remediation actions.

Benefits of AWS Macie

Listed below are a few of the great benefits offered by AWS Macie to its customers for discovering and protecting its sensitive data :

• Easy and Quick to set up :
  With one click, you can easily and quickly set up AWS Macie via the AWS Management Console which also provides multi-account support through AWS Organizations integration. With this, an organization can maintain compliance and prevent the need for an IT team to manually tackle the classification of data and permissions attached to it.

• Customizable sensitive data types :
  AWS Macie provides the ability to let the users add custom-defined data types by implementing regular expressions which helps AWS Macie discover distinct sensitive data for the organization.

• Ongoing monitoring of your S3 buckets :
  AWS Macie continuously evaluates your Amazon S3 environment where you get a summary of your S3 buckets across your AWS accounts if situations where AWS Macie detects any unencrypted buckets, publicly accessible buckets, or buckets shared outside the AWS Organization. You can also schedule one-time, daily, weekly, or monthly data discovery jobs for all the Amazon S3 buckets in your AWS account which shall automatically track the deviations in your bucket and only validate the new or modified S3 object over time.

• Meet privacy regulations :
  Implementing AWS Macie in your accounts will help you to maintain lists of all the sensitive data types like PII, GDPR, PCI-DSS, and HIPAA which help you to check if all the privacy regulations are met or not.

  The following are some other important key benefits that AWS Macie offers :

  • Recognize and protect the different data types, like PII (Personal Identifiable Information), API keys, PHI (Protected Health Information), regulatory documents, and secret keys.
  • For instant auditing, we can validate the compliance integrated with automated logs.
  • Identification of changes made to the policies and access control lists.
  • When data and account credentials are away from the protected zones, you can use AWS Macie to receive notifications.
  • Receiving actionable alerts by observing changes in user behavior.
  • Detection of large business-critical documents frequently getting exchanged or shared internally and externally.

AWS Macie Use Cases

So far we learned how AWS Macie provides various features, benefits, and more. Let us understand some of its use cases that help us understand where AWS Macie benefits can be leveraged :

Ongoing identification of sensitive data :

With AWS Macie we can continuously generate actionable findings which would help us keep a check and meet the security requirements. These features of AWS Macie can be leveraged throughout your Amazon S3 environment to validate your security posture and access.

Increase your knowledge about customizing the data discovery :

With automated data pipelines, we can identify sensitive data and use AWS Macie to validate and determine if sensitive data is appropriately protected or not during data ingestion.

Discovering precise details about data security :

AWS Macie helps to unleash precise details about sensitive data for security and compliance. You can do data analysis on a one-time, daily, weekly, or monthly basis by scheduling jobs to track if your sensitive data is protected or not.

AWS Macie Setup Guide

By far, we have discussed so much about AWS Macie. Let us explore one of its major benefits - The easy and quick setup of AWS Macie.

Yes, you heard it right. With the following five basic steps, we will set up AWS Macie over the AWS S3 bucket to discover and protect sensitive data.

• Step - 1 :
  Login to your AWS account and open your AWS Management Console as shown below :

• Step - 2 :
  Now head to the search option and select AWS Macie from the panel. Don't forget to choose the right AWS Region from where you want to start discovering and protecting your sensitive data.

• Step - 3 :
  After signing in to the Macie Console, click on the “Get started” option as shown below :

• Step - 4 :
  Now you shall see the description in the below image. Move ahead to choose the "Enable Macie" option.

• Step - 5 :
  Finally, you can just click on “Get started” on the menu present on the left-hand side panel and start scheduling data discovery jobs by selecting the “Create Job” option as shown below.

  Voila! You now have AWS Macie enabled on your S3 account which will now discover and protect your sensitive data.

AWS Macie Pricing

Let us deep dive into the Pricing structure offered by AWS Macie. Below is the diagram that depicts the various pricing details of AWS Macie which might vary from region to region.

AWS Macie pricing is charged on two bases :

First, the number of Amazon S3 buckets that are evaluated for access controls and bucket-level security, and second depending upon the amount of data processed at the time of sensitive data discovery.

Ongoing evaluation of the Number of Amazon S3 buckets for security and access controls : You get charged based on the total number of buckets in the AWS account once your 30-day free trial is over and you start to get charged on a per-day pro-rated basis.

For sensitive data discovery, how much quantity of data is getting processed :

Once AWS Macie is in action, you can configure and start to submit buckets for sensitive data discovery. This can be done by scheduling periodic(or one-time) sensitive data discovery jobs, selecting buckets that should be scanned, and submitting the analysis fetched to AWS Macie. Here, AWS Macie charges only for the bytes processed by it while inspection. You shall also incur the standard charges for the GET and LIST requests for Amazon S3.

Sensitive data discovery on Free Tier :

While scheduling the sensitive data discovery jobs, AWS Macie doesn't charge you for the first 1 GB of data processed every month for each account. After this, you start getting charged for each GB of data processed ( $0.10 per S3 bucket and month) as shown in the pricing details table above.

Amazon Macie vs. Amazon GuardDuty

Below we compare the features offered by AWS Macie and Amazon GuardDuty.

AWS Macie only checks all the S3 buckets in your AWS account. It intelligently classifies the data to help you ensure that proper access controls are applied to those S3 buckets and that the sensitive data is identified and protected.

Whereas with Amazon GuardDuty, you can employ intelligent and continuous threat detection and monitoring of the AWS accounts, S3 data, and other workloads to minimize any hidden risk. The GuardDuty also looks into:

• Unauthorized deployment
• Abnormal API activity
• Compromised EC2 instances
• Attempts to turn off AWS CloudTrail logging

Integration with AWS Security Hub

You can easily integrate AWS Macie with the AWS Security Hub. The AWS Security Hub is a cloud security posture management AWS service where all the security warnings and findings are gathered, organized, and prioritized in a single location in the AWS environment. These findings are then used to perform checks for security best practices, alert aggregation, and enabling automated remediations.

The below diagram shows how you can integrate AWS Macie with the Security Hub :

As shown above, we can select a destination for each category of findings, and simultaneously update the frequency at which the findings should be published.