AWS Security Hub           offers you a consolidated view of one's security posture in           Amazon Web Services (AWS)           and can help you check your environment against security standards and current           AWS security recommendations          . Although Security Hub has some similarities to           security information and event management (SIEM)           tools, it isn't designed as standalone a SIEM replacement. For instance, Security Hub only ingests AWS-related security findings and will not ingest higher volume event logs directly, such as for example AWS CloudTrail logs. When you have use cases to consolidate AWS findings with other styles of findings from on-premises or other non-AWS workloads, or if you want to ingest higher volume event logs, we advise that you utilize Security Hub together with a SIEM tool.

There’s also other advantages to using Security Hub and a SIEM tool together. Included in these are having the ability to store findings for longer intervals than Security Hub, aggregating findings across multiple administrator accounts, and additional correlating Security Hub findings with one another along with other log sources. In this website post, we shall demonstrate ways to use Amazon OpenSearch Service (successor to Amazon Elasticsearch Service) as a SIEM and integrate Security Hub with it to perform these three use cases. Amazon OpenSearch Service is really a managed service that means it is better to deploy fully, manage, and scale Kibana and Elasticsearch. OpenSearch Service is really a distributed, RESTful analytics and internet search engine that is with the capacity of addressing an increasing number of use cases. It is possible to expand OpenSearch Service with AWS services like Kinesis or kinesis Data Firehose, by integrating with other AWS services, or through the use of traditional agents like Logstash and Beats for log ingestion, and Kibana for data visualization. Even though OpenSearch Service isn’t a SIEM out-of-the-box tool also, with some customization, it could be utilized by you for SIEM tool use cases.

Security SIEM plus Hub use cases

By enabling Security Hub inside your AWS Organizations account structure, you immediately start receiving the advantages of viewing all your security findings from across various AWS and partner services about the same screen. Some organizations desire to go a step further and use Security Hub together with a SIEM tool for the next reasons:

• Correlate Security Hub findings with one another along with other log sources – This is actually the hottest reason customers elect to implement this solution. When you have various log sources beyond Security Hub findings (such as for example application logs, database logs, partner logs, and security tooling logs), then it seems sensible to consolidate these log sources right into a single SIEM solution. You’ll be able to view both your Security Hub findings and miscellaneous logs in exactly the same place and create alerts predicated on interesting correlations.
• Store findings for longer than 3 months following the last update date – Some organizations want or have to store Security Hub findings for longer than 3 months following the last update date. They could wish to accomplish this for historical investigation, or for compliance and audit needs. Either way, you’re provided by this solution the capability to store Security Hub findings in an exclusive Amazon Simple Storage Service (Amazon S3) bucket, that is consumed by Amazon OpenSearch Service then.
• Aggregate findings across multiple administrator accounts – Security Hub includes a feature customers may use to designate an administrator account should they have enabled Security Hub in multiple accounts. A Security Hub administrator account can view data from and manage configuration because of its member accounts. This enables customers to see and manage almost all their findings from multiple member accounts in a single place. Customers have multiple Security Hub administrator accounts sometimes, since they have multiple organizations in AWS Organizations. In this example, you should use this treatment for consolidate all the Security Hub administrator accounts right into a single OpenSearch Service with Kibana SIEM implementation to truly have a single view across your environments. This related post walks through this use case in greater detail, and shows how exactly to centralize Security Hub findings across multiple AWS administrators and Regions. However, this website post takes this process by introducing OpenSearch Service with Kibana to the utilization case further, for a complete SIEM experience.

Solution architecture

The perfect solution is represented in Figure 1 shows the flexibleness of integrations which are possible when you develop a SIEM through the use of Amazon OpenSearch Service. You’re allowed by the answer to aggregate findings across multiple accounts, store findings within an S3 bucket indefinitely, and correlate multiple AWS and non-AWS services in a single place for visualization. This post targets Security Hub’s integration with the perfect solution is, however the following AWS services can also integrate:

Each one of these ongoing services has its dedicated dashboard within the OpenSearch SIEM solution. This allows for customers to see findings and data which are highly relevant to each service that the SIEM tool is ingesting. OpenSearch Service allows the client to generate aggregated dashboards also, consolidating multiple services inside a single dashboard, if needed.

Prerequisites

We advise that you enable Security &lt and Hub;a href=”http://aws.amazon.com/config” target=”_blank” rel=”noopener noreferrer”>AWS Config across all your Regions and accounts. For more information about how exactly to do this, start to see the documentation for Security Hub and AWS Config. We also advise that you utilize Security AWS and Hub Config integration with AWS Organizations to simplify the setup and automatically enable these ongoing services in every current and future accounts in your company.

Launch the solution

To be able to launch this solution inside your environment, it is possible to launch the solution through the use of an AWS CloudFormation template either, or by following a steps presented later in this article to customize the deployment to aid integrations with non-AWS services, multi-Organization deployments, or launch inside your existing OpenSearch Service environment.

To launch the answer, follow the instructions for SIEM on Amazon OpenSearch Service on GitHub.

Utilize the solution

Before you begin utilizing the solution, we’ll demonstrate how this solution appears in the Security Hub dashboard, as shown in Figure 2. Navigate by &lt here;a href=”https://github.com/aws-samples/siem-on-amazon-opensearch-service/#3-configuring-opensearch-dashboards” target=”_blank” rel=”noopener noreferrer”>following Step 3 from the GitHub README.

The Security Hub dashboard highlights all major the different parts of the ongoing service in a OpenSearch Service dashboard environment. This includes supporting every one of the service integrations that exist within Security Hub (such as for example GuardDuty, AWS Identity and Access Management (IAM) Access Analyzer, Amazon Inspector, Amazon Macie, and AWS Systems Manager Patch Manager). The dashboard displays both security and findings standards, and you may filter by AWS account, finding type, security standard, or service integration. Figure 3 shows a synopsis of the visual dashboard experience once you deploy the perfect solution is.

Use case 1: Correlate Security Hub findings with one another along with other log sources and create alerts

This solution uses OpenSearch Service and Kibana to help you to read through both Security Hub findings and logs from any AWS and non-AWS systems. After that you can create alerts within Kibana predicated on interesting correlations between Security Hub and any logged events. Although Security Hub supports ingesting a massive amount of findings and integrations, it cannot create correlation rules such as a SIEM tool can. However, it is possible to create such rules using SIEM on OpenSearch Service. It’s vital that you have a closer look when multiple AWS security services generate findings for an individual resource, because this means that elevated risk or multiple risk vectors potentially. Based on your environment, the original amount of findings in Security Hub may be high, so you might have to prioritize which findings require immediate action. Security Hub offers you the &lt natively;a href=”https://docs.aws.amazon.com/securityhub/latest/userguide/findings-filtering-grouping.html” target=”_blank” rel=”noopener noreferrer”>capability to filter findings by resource, account, severity, and several other details.

Within the findings, it is possible to send notifications through alerts which are generated by SIEM on OpenSearch Service in a number of ways: Amazon Simple Notification Service (Amazon SNS) by eating messages within an appropriate &lt or tool;a href=”https://docs.aws.amazon.com/sns/latest/dg/sns-email-notifications.html” target=”_blank” rel=”noopener noreferrer”>configuring recipient email addresses, Amazon Chime, Slack (using AWS Chatbot) or custom webhook to your organization’s ticketing system. It is possible to react to these new security incident-oriented findings through ticketing then, chat, or incident management systems.

Solution overview for use case 1

Figure 4 gives a synopsis of the answer for use case 1. This solution requires that you have Security GuardDuty and Hub enabled in your AWS account. Logs from AWS services, including Security Hub, are ingested into an S3 bucket, are automatically extracted then, transformed, and loaded (ETL) and populated in to the SIEM system that’s running on OpenSearch Service using AWS Lambda. After capturing the logs, you will be in a position to visualize them on the dashboard and analyze correlations of multiple logs. Within the SIEM on OpenSearch Service solution, you shall develop a rule to detect failures, such as for example CloudTrail authentication failures in logs. Then, you’ll configure the solution to create alerts to Amazon SNS and send emails when logs match rules.

Implement the perfect solution is for use case 1

You’ll now setup this workflow to alert you by email when logs in OpenSearch match certain rules that you create.

Step one 1: Create and visualize findings in OpenSearch Dashboards

Security Hub along with other AWS services export findings to Amazon S3 in a centralized log bucket. It is possible to ingest logs from CloudTrail, VPC Flow Logs, and GuardDuty, which are generally found in AWS security analytics. In this task, you import simulated security incident data in OpenSearch Dashboards, and utilize the dashboard to visualize the info in the logs.

To navigate OpenSearch Dashboards

1. Generate pseudo-security incidents. It is possible to simulate the full total results by generating sample findings in GuardDuty.
2. In OpenSearch Dashboards, go directly to the Discover screen. The Discover screen is split into three major sections: Search bar, index/display field list, and time-series display, as shown in Figure 5.

   

   Figure 5: OpenSearch Dashboards

3. In OpenSearch Dashboards, select log-aws-securityhub-* or log-aws-vpcflowlogs-* or log-aws-cloudtrail-* or any index patterns and add event.module to the display field. event.module is really a field that indicates where in fact the log originates from. If you’re collecting other threat information, such as for example Security Hub, @log-type is Security Hub, and event.module indicates where in fact the log comes from (either Amazon Inspector or Amazon Macie for instance). Once you’ve added event.module, filter the required Security Hub integrated service (for instance, Amazon Inspector) to show. When testing the surroundings covered in this website post outside a production context, you should use Kinesis Data Generator to create sample user traffic. Other tools can be found also.
4. Choose the following on the dashboard to start to see the visualized information:
   • CloudTrail Summary
   • VpcFlowLogs Summary
   • GuardDuty Summary
   • All – Threat Hunting

Step two 2: Configure alerts to complement log criteria

Next, you shall configure alerts to complement log criteria. You need to create the destination for alerts first, and set what things to monitor then.

To configure alerts

1. In OpenSearch Dashboards, in the left menu, choose Alerting.
2. To include the facts of SNS, on the Destinations tab, choose Add destinations, and enter the next parameters:
   • Name: aes-siem-alert-destination
   • Type: Amazon SNS
   • SNS Alert: arn:aws:sns::<111111111111>:aes-siem-alert
     • Replace <111111111111> together with your AWS account ID and correct the spot name
     • Replace with the spot you are using, for instance, eu-west-1
   • IAM Role ARN: arn:aws:iam::<111111111111>:role/aes-siem-sns-role
     • Replace &<111111111111> together with your AWS account ID
3. Choose Create to perform setting the alert destination.

   

   Figure 6: Edit alert destination

4. In OpenSearch Dashboards, in the left menu, select Alerting. You’ll set what things to monitor now. You monitor a CloudTrail trail authentication failure here. You can find two normalized log times: @timestamp and event.ingested. The difference is between your log occurrence time (@timestamp) and the SIEM reception time (event.ingested). Use event.ingested for logs with a big time lag from occurrence to reception. It is possible to specify flexible conditions by selecting Define using extraction query for the filter definition.
5. On the Monitors tab, choose Create monitor.
6. Enter the next parameters. When there is no description, utilize the default value.
   • Name: Authentication failed
   • Approach to definition: Define using extraction query
   • Indices: log-aws-cloudtrail-* (manual input, not pull-down)
   • Define extraction query: Enter the next query.

     
     "query": 
         "bool": 
             "filter": [
             "term": "eventSource": "signin.amazonaws.com",
             "term": "event.outcome": "failure",
             "range": 
                 "event.ingested":

         ]
     
     
      

   • Enter the next remaining parameters of the monitor:
     • • Frequency: By interval

      

     • • Monitor schedule: Every 3 minutes

      

   •  

   • Choose Create to generate the monitor.

   •  

     Step three 3: Create trigger to send email via Amazon SNS

     You’ll set the alert firing condition now, referred to as the trigger. This is actually the setting for alerting once the monitored conditions (Monitors) are met. Automagically, the alert will be triggered if the amount of hits is higher than 0. In this step , you shall not change it out, only give it a genuine name.

     To create the trigger

     1. Select Create trigger and for Trigger name, enter Authentication failed trigger.
     2. Scroll right down to Configure actions.

        

        Figure 7: Create trigger

     3. Set what the trigger must do (action). In this full case, you intend to publish to SNS. Set the next parameters for the physical body of the e-mail
        • Action name: Authentication failed action
        • Destination: Choose aes-siem-alert-destination – (Amazon SNS)
        • Message subject: (SIEM) Auth failure alert
        • Action throttling: Select Enable action throttling, and set throttle action to only trigger every ten minutes.
        • Message: Copy and paste the next message in to the text box. After pasting, choose Send test message in the bottom right of the screen to verify that you can have the test email.Monitor ctx.monitor.name entered alert status just. Please investigate the presssing issue.Trigger: ctx.trigger.nameSeverity: ctx.trigger.severity

          @timestamp: ctx.results.0.hits.hits.0._source.@timestamp

          event.action: ctx.results.0.hits.hits.0._source.event.action

          error.message: ctx.results.0.hits.hits.0._source.error.message

          count: ctx.results.0.hits.total.value

          source.ip: ctx.results.0.hits.hits.0._source.source.ip

          source.geo.country_name: ctx.results.0.hits.hits.0._source.source.geo.country_name

        

        Figure 8: Configure actions

     4. You shall receive an alert email in a minute. You can examine the occurrence status, including the past history, by the next method:
        1. In OpenSearch Dashboards, on the left menu, choose Alerting.
        2. On the Monitors tab, choose Authentication failed.
        3. You can examine the status of the alert in the History pane.

        

        Figure 9: Email alert

     Use case 1 demonstrates how to correlate various Security Hub findings through this OpenSearch Service SIEM solution. However, it is possible to take the answer a step further and build more technical correlation checks by following procedure in your blog post Correlate security findings with AWS Security Amazon and Hub EventBridge. This information may then be ingested into this OpenSearch Service SIEM solution for viewing about the same screen.

     Use case 2: Store findings for longer than 3 months after last update date

     Security Hub includes a maximum storage time of 3 months for events, however your organization may necessitate data storage beyond that period, with flexibility to specify a custom retention period to meet up your preferences. The SIEM on Amazon OpenSearch Service solution creates a centralized S3 bucket where findings from Security Hub and different other services are stored and collected, and this bucket could be configured to store data as as you need long. The S3 bucket can indefinitely persist data, or it is possible to create an S3 object lifecycle policy to create a custom retention timeframe. Lifecycle policies enable you to either transition objects between S3 storage classes or delete objects following a specified period. Alternatively, you should use S3 Intelligent-Tiering to permit the Amazon S3 service to go data between tiers, predicated on user access patterns.

     Either lifecycle policies or S3 Intelligent-Tiering shall enable you to optimize charges for data that’s stored in S3, to help keep data for archive or backup purposes when it’s no longer obtainable in Security Hub or OpenSearch Service. Within the perfect solution is, this centralized bucket is named aes-siem-xxxxxxxx-log and is configured to store data for OpenSearch Service to take indefinitely. The Amazon S3 User Guide has instructions for configuring an S3 lifecycle policy that’s defined by an individual on the centralized bucket explicitly. Or it is possible to follow the instructions for configuring intelligent tiering to permit the S3 service to control which tier data is stored in automatically. After data is archived, you should use Amazon Athena to query the S3 bucket for historical information that is taken off OpenSearch Service, because this S3 bucket acts as a centralized security event repository.

     Use case 3: Aggregate findings across multiple administrator accounts

     You can find cases where you might have multiple Security Hub administrator accounts within one or multiple organizations. For these use cases, it is possible to consolidate findings across these multiple Security Hub administrator accounts right into a single S3 bucket for centralized storage, archive, backup, and querying. Thus giving you the capability to develop a single SIEM on OpenSearch Service to reduce the amount of monitoring tools you will need. To carry out this, you should use S3 replication to copy findings to a centralized S3 bucket automatically. It is possible to follow this detailed walkthrough on how best to set up the right bucket permissions to be able to allow replication between your accounts. It is possible to follow this related &lt also;a href=”https://aws.amazon.com/blogs/architecture/visualize-aws-security-hub-findings-using-analytics-and-business-intelligence-tools/” target=”_blank” rel=”noopener noreferrer”>blog post to configure cross-Region Security Hub findings which are centralized within a S3 bucket, if cross-Region replication is suitable for the security needs. With cross-account S3 replication create for Security Hub archived event data, it is possible to import data from the centralized S3 bucket into OpenSearch Service utilizing the Lambda function within the solution in this website post. This Lambda function automatically normalizes and enriches the log imports and data it into OpenSearch Service, so that users just need to configure data storage in the S3 bucket, and the Lambda function will import the info.

     Conclusion

     In this website post, we showed ways to use Security Hub with a SIEM to store findings for longer than 3 months, aggregate findings across multiple administrator accounts, and correlate Security Hub findings with one another along with other log sources. We used the answer to walk through building the SIEM and explained how Security Hub could possibly be used within that treatment for add greater flexibility. This post describes one treatment for create your personal SIEM using OpenSearch Service; however, we advise that you read the post &lt also;a href=”https://aws.amazon.com/blogs/architecture/visualize-aws-security-hub-findings-using-analytics-and-business-intelligence-tools/” target=”_blank” rel=”noopener noreferrer”>Visualize AWS Security Hub Findings using Business and Analytics Intelligence Tools, to be able to see a different approach to visualizing and consolidating insights from Security Hub.

     For more information, you can test out this solution through the brand new &lt also;a href=”https://security-log-analysis-platform.workshop.aws/en/” target=”_blank” rel=”noopener noreferrer”>SIEM on AWS OpenSearch Service workshop.

     When you have feedback about this post, submit comments in the Comments section below. When you have questions about this post, please take up a new thread on the Security Hub forum or contact AWS Support.

     When you have feedback concerning this post, submit comments in the Comments section below. When you have questions concerning this post, contact AWS Support.

     Want more AWS Security news? Follow us on Twitter.