AWS Security Hub           is really a service that provides you aggregated visibility into your security and compliance posture across multiple           Amazon Web Services (AWS)           accounts. By joining Security Hub with           Amazon QuickSight          -a scalable, serverless, embeddable, machine learning-powered business intelligence (BI) service built for the cloud-your senior leaders and decision-makers may use dashboards to empower data-driven decisions and facilitate a secure configuration of AWS resources

In organizations that operate at cloud scale, having the ability to summarize and perform trend analysis is paramount to remediating and identifying problems early, which leads to the entire success of the business. Additionally, QuickSight dashboards could be embedded in dashboard and reporting platforms that leaders already are familiar with, making the dashboards more user-friendly even.

 

With the answer in this website post, it is possible to provide leaders with cross-AWS Region views of data make it possible for decision-makers to measure the health insurance and status of an organizations IT infrastructure instantly. Additionally you can enrich the dashboard with data sources unavailable to Security Hub. Finally, this solution gives you the flexibility to possess multiple administrator accounts across several AWS organizations and combine them right into a single view.

In this website post, you shall learn to build an analytics pipeline of one’s Security Hub findings, summarize the info with Amazon Athena, and visualize the info via QuickSight utilizing the following steps:

• Deploy an AWS Cloud Development Kit (AWS CDK) stack that builds the infrastructure you will need to begin with.
• Create an Athena view that summarizes the raw findings.
• Visualize the summary of findings in QuickSight.
• Secure using guidelines QuickSight.

For a high-level discussion without code examples please see Visualize AWS Security Hub Findings using Business and Analytics Intelligence Tools.

Prerequisites

This website assumes :&lt that you;/p>

• Have a simple understanding of how exactly to authenticate and access your AWS account.
• Have the ability to run commands with a command line prompt on your own local machine.
• Have a simple knowledge of Structured Query Language (SQL).

Solution overview

Figure 1 shows the flow of events and a high-level architecture diagram of the perfect solution is.

The steps shown in Figure 1 include:

• Detect
• Collect
• Aggregate
• Transform
• Analyze
• Visualize

Detect

AWS supplies a amount of tools to continuously help detect security findings. These tools belong to three types:

In this website, you shall use two built-in security standards of Security Hub-CIS AWS Foundations Benchmark controls and AWS Foundational Security GUIDELINES Standard-and a serverless Prowler scanner that acts as a third-party partner product. Where AWS Organizations can be used, member accounts send these findings to the known member account’s Security Hub

Collect

Inside a region, security findings are centralized right into a single administrator account using Security Hub.

Aggregate

Utilizing the cross-Region aggregation feature within Security Hub, findings within each administrator account could be aggregated and synchronized across multiple regions continuously.

Ingest

Security Hub not merely offers a comprehensive view of security security and alerts posture across your AWS accounts, it acts as a data sink for the security tools also. Any tool that may expose data via AWS Security Finding Format (ASFF) may use the BatchImportFindings API action to push data to Security Hub. For additional information, see Using custom product integration to send findings to AWS Security Hub and Available AWS service integrations in the Security Hub User Guide.

Transform

Data appearing out of Security Hub is exposed via Amazon EventBridge. Unfortunately, it’s nearly in an application that Athena can consume. EventBridge streams data through Amazon Kinesis Data Firehose to &lt directly;a href=”https://aws.amazon.com/s3/” target=”_blank” rel=”noopener noreferrer”>Amazon Simple Storage Service (Amazon S3). From Amazon S3, an &lt could be created by you;a href=”https://aws.amazon.com/lambda/” target=”_blank” rel=”noopener noreferrer”>AWS Lambda function that flattens and fixes a number of the column names, such as for example by detatching special characters that Athena cannot recognize. The Lambda function saves the outcomes back again to S3 then. Finally, an AWS Glue crawler dynamically discovers the schema of the creates and data or updates an Athena table.

Analyze

You’ll aggregate the raw findings data and create metrics along various grains or pivots by developing a simple yet meaningful Athena view. With Athena, additionally you can use views to become listed on the info with other data sources, such as for example your organization’s configuration management database (CMDB) or IT service management (ITSM) system.

Visualize

Using QuickSight, you’ll register the info sources and build visualizations you can use to recognize areas where security could be improved or reduce risk. This post shares steps detailing how exactly to do that in the Build QuickSight visualizations section below.

Use AWS CDK to deploy the infrastructure

To be able to analyze and visualize security related findings, you shall have to deploy the infrastructure necessary to detect, ingest, and transform those findings. You shall use an AWS CDK stack to deploy the infrastructure back. To begin with, review the prerequisites to be sure you have all you need to deploy the CDK stack. The CDK stack is deployed once, you can deploy the specific infrastructure. Following the infrastructure has been deployed, you shall build an Athena view and a QuickSight visualization.

The program to deploy the solution&lt install;/h3>

For the answer in this website post, you’ll want the next tools installed:

• The answer in this website post is written in Python, which means you must install Python along with CDK. Instructions on how best to install Python version 3.X are available on the downloads page.
• AWS CDK requires node.js. Directions on how best to install node.js can on the node.js downloads page.
• This CDK application uses Docker for local bundling. Directions for using Docker are available at Get Docker.
• AWS CDK-a software-development framework for defining cloud infrastructure in code and provisioning it through AWS CloudFormation. To set up CDK, visit AWS CDK Toolkit page.

To verify you have the all you need

1. Confirm you’re running version 1.108.0 or of CDK later.$ cdk ‐‐version
2. Download the code from github by cloning the repository. cd in to the clone directory.$ git clone git@github.com:aws-samples/aws-security-hub-analytic-pipeline.git$ cd aws-security-hub-analytic-pipeline
3. Create a virtualenv manually.$ python3 -m venv .venv
4. Following the initialization process completes and the virtualenv is established, you can use the next step to activate your virtualenv.$ source .venv/bin/activate
5. If you’re utilizing a Windows platform, utilize the following command to activate virtualenv:% .venv/Scripts/activate.bat
6. The virtualenv is activated once, you can install the mandatory dependencies.$ pip install -r requirements.txt

Use AWS CDK to deploy the infrastructure into your account

The next steps use AWS CDK to deploy the infrastructure. This infrastructure includes the many scanners, Security Hub, EventBridge, and Kinesis Firehose streams. When complete, the raw Security Hub data will undoubtedly be stored within an S3 bucket already.

To deploy the infrastructure using AWS CDK

1. If you’ve never used AWS CDK in the account you’re using or if you’ve never used CDK in the us-east-1, us-east-2, or us-west-1 Regions, you need to bootstrap the regions via the command prompt.$ cdk bootstrap
2. At this true point, it is possible to deploy the stack to your default AWS account via the command prompt.$ cdk deploy -all
3. While cdk deploy is running, you shall start to see the output in Figure 2. This can be a prompt to make sure you’re aware that you’re creating a security-relevant change and creating AWS Identity and Access Management (IAM) roles. Enter y when prompted to keep the deployment process:

   

   Figure 2. CDK approval prompt to generate IAM roles

4. Confirm cdk deploy is completed. When the deployment is completed, you need to see three stack ARNs. It shall look much like Figure 3.

   

   Figure 3. Final output of CDK deploy

As a complete consequence of the deployed CDK code, Security Hub and the Prowler scanner will scan your account automatically, process the info, and send it to S3. Although it takes significantly less than an full hour for a few data to be processed and searchable in Athena, we recommend waiting a day before proceeding to another steps, to make sure enough data is processed to create useful visualizations. It is because the rest of the steps roll-up findings by the entire hour. Also, it requires several minutes to obtain initial results from the Security Hub standards or more to an hour to obtain initial results from Prowler.

Build an Athena view

That you’re deployed the infrastructure to detect now, ingest, and transform security related findings, it’s time and energy to use an Athena view to perform the analyze part of the solution. Day the next view aggregates the amount of findings for confirmed. Athena views may be used to summarize data or enrich it with data from other sources. Utilize the following steps to create a straightforward example view. To find out more on creating Athena views, see Dealing with Views.

To create an Athena view

1. Open the AWS Management Console and make sure that the Region is defined to us-east-1 (Northern Virginia).
2. Demand Athena service. If you’ve never used this service, choose Get Started to demand Query Editor screen. Otherwise, the Query Editor screen may be the default view.
3. If you’re not used to Athena, you need to create a query result location also.
   1. Choose Settings in the very best right of the Query Editor screen to open the settings panel.
   2. Choose Select to choose a query result location.

      

      Figure 4. Athena settings

   3. Locate an S3 bucket in the list that starts with analyticsink-queryresults and pick the right-arrow icon.
   4. Choose Select to choose a query results bucket.

      

      Figure 5. Select S3 location confirmation

4. Select AwsDataCatalog because the Data source and security_hub_database because the Database. The Query Editor screen should appear to be Figure 6.

   

   Figure 6. Empty query editor

5. Copy and paste the next SQL in the query window:REPLACE or create VIEW “security-hub-rolled-up-finding” AS
   SELECT
   “date_format”(“from_iso8601_timestamp”(updatedat), ‘%Y-%m-%d %H:00’) year_month_day
   , region
   , compliance_status
   , workflowstate
   , severity_label
   , COUNT(DISTINCT title) as cnt
   FROM
   security_hub_database.“security-hub-crawled-findings”
   GROUP BY “date_format”(“from_iso8601_timestamp”(updatedat), ‘%Y-%m-%d %H:00’), compliance_status, workflowstate, severity_label, region
6. Pick the Run query button.

If everything is correct, you need to see Query successful in the Results, as shown in Figure 7.

Build QuickSight visualizations

Given that you’ve deployed the infrastructure to detect, ingest, and transform security related findings, and also have created an Athena view to investigate those findings, it’s time and energy to use QuickSight to visualize the findings. To utilize QuickSight, you need to grant QuickSight permissions to gain access to S3 and Athena first. You develop a QuickSight databases next. Third, you shall develop a QuickSight analysis. (Optional) When complete, the analysis could be published by you.

You shall create a simple visualization that presents counts of findings as time passes separated by severity, though it’s also possible to utilize QuickSight to inform rich and compelling visual stories.

To be able to use QuickSight, you will need to join up for a QuickSight subscription. Steps to take action are available in Registering for an Amazon QuickSight Subscription.

The very first thing you must do logged directly into QuickSight is create the info source once. If that is your first-time logging into the ongoing service, you will be greeted having an initial QuickSight page as shown in Figure 8.

Grant QuickSight usage of Athena&lt and S3;/h3>

While creating the Athena databases will enable to query data from Athena QuickSight, you need make it possible for QuickSight to learn from S3 also.

To grant QuickSight usage of Athena&lt and S3;/em>

1. Inside QuickSight, select your profile name (upper right). Choose Manage QuickSight, and choose &lt then;strong>Security & permissions.
2. Choose Add or remove.
3. Ensure the checkbox close to Athena is selected.
4. Ensure the checkbox close to Amazon S3 is selected.
5. Choose Details and choose Select S3 Buckets.
6. Locate an S3 bucket in the list that starts with analyticsink-bucket and ensure the checkbox is selected.

   

   Figure 9. Example permissions

7. Choose Finish to save lots of changes.

Develop a QuickSight dataset

Once you’ve given QuickSight the required permissions, you can develop a new dataset.

To make a QuickSight dataset

1. Choose Datasets from the navigation pane at left. Choose &lt then;strong>New Dataset.

   

   Figure 10. Dataset page

2. To make a new Athena connection profile, utilize the following steps:
   1. In the FROM NEW DATA SOURCES section, pick the Athena databases card.
   2. For Databases name, enter a descriptive name. For instance: security-hub-rolled-up-finding.
   3. For Athena workgroup choose [ primary ].
   4. Choose Validate connection to check the connection. This confirms encryption at rest.
   5. Choose Create data source.
3. On the Choose your table screen, select:
   Catalog: AwsDataCatalog
   Database: security_hub_database
   Table: security-hub-rolled-up-finding
4. Finally, choose the Import to SPICE for quicker analytics option and choose Visualize.

You’re finished once, the page to generate your first analysis will open automatically. Figure 11 shows a good example of the page.

Develop a QuickSight analysis

A QuickSight analysis is a lot more than just a visualization-it can help you uncover hidden trends and insights in your computer data, identify key drivers, and forecast business metrics. It is possible to create rich analytic experiences with QuickSight. To find out more, visit Dealing with Visuals in the User Guide&lt quicksight;/a>.

For simplicity, you’ll create a visualization that summarizes findings categories by severity and aggregated by hour.

To make a QuickSight analysis

1. Choose Line Chart from the Visual Types.

   

   Figure 12. Visual types

2. Select Fields. Figure 13 shows what your field wells should appear to be at the ultimate end of the step.
   1. Locate the year_month_day_hour field in the field list and drag it to the X axis field well.
   2. Locate the cnt field in the field list and drag it to the Value field well.
   3. Locate the severity_label field in the field list and drag it to Color field well.

      

      Figure 13. Field wells

3. Add Filters.
   1. Select Filter in the left navigation panel.

      

      Figure 14. Filters panel

   2. Choose Create one… and choose the compliance_status field.
   3. Expand the filter and clear NOT_AVAILABLE and PASSED (Note: based on your data, you will possibly not have many of these statuses).
   4. Choose Apply to use the filter.

      

      Figure 15. Filtering out findings that failing&lt aren’t;/p>

You need to visit a visualization that appears like Figure 16 now, which shows an overview count of events and their severity.

Publish a QuickSight analysis dashboard (optional)

Publishing a dashboard is a superb solution to share reports with leaders. This two-step process lets you share visualizations as a dashboard.

To create a QuickSight analysis

1. Choose Share on the application form bar, choose &lt then;strong>Publish dashboard.
2. Select Publish new dashboard as, and enter a dashboard name, such as for example Security Hub Findings by Severity.

You can even embed dashboards into web applications. This involves utilizing the AWS SDK or through the AWS Command Line Interface (AWS CLI). To find out more, see Embedding QuickSight Data Dashboards for Everyone.

Encouraged security posture in QuickSight

QuickSight includes a amount of security features. As the AWS Security portion of the QuickSight User Guide switches into detail, here’s a listing of the standards that connect with this type of scenario. For additional information see Security in Amazon QuickSight&lt aws;/a> within the QuickSight user guide.

Tidy up (optional)

When done, you can tidy up QuickSight by detatching the Athena view and the CDK stack. Follow the detailed steps below to completely clean up everything.

To completely clean up QuickSight

1. Open the console and choose Datasets in the left navigation pane.
2. Select security-hub-rolled-up-finding then choose Delete dataset.
3. Confirm dataset deletion by choosing Delete.
4. Choose Analyses from the left navigation pane.
5. Pick the menu in the low right corner of the security-hub-rolled-up-finding card.

   

   Figure 17. Example analysis card

6. Select Delete and confirm Delete.

To eliminate the Athena view

1. Paste the next SQL in the query window:DROP VIEW “security-hub-rolled-up-finding”
2. Pick the Run query button.

To eliminate the CDK stack

1. Run the next command in your terminal:cdk destroy
   

   Note: In the event that you experience errors, you may want to reactivate your Python virtual environment by completing steps 3-5 of Use AWS CDK to deploy the infrastructure.

Conclusion

In this website, you used Security Hub and QuickSight to deploy a scalable analytic pipeline for the security tools. Security Hub allowed one to join and collect security findings from multiple sources. With QuickSight, you summarized data for the senior leaders and decision-makers to provide them the proper data in real-time.

You ensured your sensitive data remained protected by explicitly granting QuickSight the capability to read from the specific S3 bucket. By authorizing access and then the data sources had a need to visualize your computer data, you ensure least privilege access. QuickSight supports a great many other AWS data sources, including Amazon RDS, Amazon Redshift, Lake Formation, and Amazon OpenSearch Service (successor to Amazon Elasticsearch Service). As the data doesn’t live in a Amazon Virtual Private Cloud (Amazon VPC), you didn’t have to grant usage of any specific VPCs. Limiting usage of VPCs is another smart way to boost the security of one’s environment.

When you have feedback concerning this post, submit comments in the Comments section below. When you have questions concerning this post, take up a new thread on the Security Hub forum. To start out your 30-day trial offer of Security Hub, visit AWS Security Hub.

Want more AWS Security how-to content, news, and show announcements? Follow us on Twitter.