Note: This blog post describes how to automatically update AWS WAF IP sets with the most recent AWS IP ranges for AWS services. This related blog post describes how to perform a similar update for Amazon CloudFront IP ranges that are used in VPC Security Groups.

You can use AWS Managed Rules for AWS WAF to create baseline protections for your web applications quickly, including setting up lists of IP addresses to be blocked. In some full cases, you might need to create an IP set in AWS WAF with the IP address ranges of Amazon Web Services (AWS) services that you use, so that traffic from these ongoing services is allowed. In this blog post, we provide a solution that updates an AWS WAF IP set with the IP address ranges of the AWS services Amazon CloudFront, Amazon Route 53 health checks, and Amazon EC2 (and also the services that share the same IP address ranges, such as AWS Lambda, Amazon CloudWatch, and so on). These ongoing services are present in the AWS Managed Rules Anonymous IP list, and blocking them may cause inadvertent service impairment for applications that expect traffic from the ongoing services.

As an application owner, you can improve your security posture by using the Anonymous IP list in your AWS WAF web access control lists (web ACLs) to block source IP addresses from specific hosting providers and anonymization services, such as VPNs, proxies, and Tor nodes. Due to the generic nature of these rules, when the Anonymous is used by you IP list, you may want to exclude certain IPs from the list of IPs to be blocked, in order to allow web traffic from those sources. For example, you can allow traffic that originates from the AWS network.

Alternatively, you may want to permit only IP addresses from certain AWS services in a web ACL. This is a common requirement when an Application is protected by you Load Balancer by restricting all incoming traffic to CloudFront IP ranges. Creating your own custom list to allow expected traffic from some effort is required by the AWS network, because you need to update the list by using the &lt periodically;a href=”https://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html” target=”_blank” rel=”noopener noreferrer”>IP ranges that we provide. With the solution we here present, you don’t have to manage the exclusion list manually. When the new AWS IP ranges are published, this solution will fetch and update the list automatically.

Note: This solution only works with AWS WAF, and will not work with AWS WAF Classic.

Solution overview

Figure 1 shows the solution architecture.

AWS sends Amazon Simple Notification Service (Amazon SNS) notifications to subscribers of the AmazonIPSpaceChanged SNS topic when updates are made to the public IP addresses for AWS services. An &lt is used by this solution;a href=”http://aws.amazon.com/cloudformation” target=”_blank” rel=”noopener noreferrer”>AWS CloudFormation template to deploy an AWS Lambda function that is triggered by these SNS notifications. The function creates AWS WAF IP sets for IPv6 and IPv4 address ranges in your web ACL.

The solution workflow is as follows:

1. 1. In the CloudFormation template, you select the ongoing services that you want the AWS WAF IP set to be updated with.

1. 1. The template deploys the required AWS resources with the configuration that specifies what services to fetch from an AWS public IP address update.

1. 1. AWS Lambda function is manually invoked one first time to populate AWS WAF IP sets with selected IPs from AWS IP range.

1. 1. AWS IP range is updated once, an Amazon SNS notification is sent to subscribers of the SNS topic.

1. 1. SNS notification triggers the AWS Lambda function.

1. 1. The Lambda function fetches the selected IP updates and ranges IP sets for IPv4 addresses and IPv6 addresses.

1. 1. The application owner adds a custom AWS WAF web ACL rule that uses the IP sets to allow traffic from the AWS services that you’ve selected. This real way, the web ACL makes reference to always updated AWS WAF IP sets with no further action required from your side.

Solution prerequisites

The solution is automatically created when you deploy the AWS CloudFormation template that is available on the solution’s GitHub page. There are three resources that you must have in place before you deploy the template:

• • The Python code that will be used as the Lambda function.
    • • Download the update_aws_waf_ipset.py Python code from the project’s AWS Lambda directory in GitHub. This function is responsible for constantly checking AWS IPs and making sure that your AWS WAF IP sets are always updated with the most recent set of IPs in use by the AWS service of choice.

• • An Amazon Simple Storage Service (Amazon S3) bucket that you shall use to store the compressed Python code.
    • • Compress the file to a .zip file and upload it to an Amazon Simple Storage Service (Amazon S3) bucket in the same AWS Region where you will deploy the template. For instructions on how to create an S3 bucket, see Creating a bucket.

• • An AWS WAF web ACL to filter requests that come in from trusted sources. The web ACL uses the IP sets that the solution updates and creates with the necessary IP addresses.

Deploy the AWS CloudFormation template

The CloudFormation template deploys the required resources for this solution in your account. The following resources are deployed:

• • Two AWS WAF IP sets, IPv4Set and IPv6Set that are used to store IPv4 and IPv6 IP addresses from the services you’re interested in allowing. Those IP sets are visible in the AWS WAF console under the same Region where the template is deployed.
    • • Note: The IP address 192.0.2.0/24 that appears in the template is a placeholder for the IP addresses that will be populated by the solution, and it is used for documentation purposes only.

• • The update_aws_waf_ipset.py Python code is used in an AWS Lambda function called UpdateWAFIPSet. This is the function that will read which services the solution should collects IPs from, and which IP sets should be populated. If you don’t change those parameters, the function shall use default IP set suffixes. By default, the solution shall select ROUTE53_HEALTHCHECKS and CLOUDFRONT as the ongoing services for which to download IPs. You can update the list of IP addresses as needed, by referring to the AWS IP JSON document for a list of service IP and names ranges.

• • A Lambda execution role with permissions restricted to least privilege required.

• • The Lambda function is subscribed to the AmazonIPSpaceChanged SNS topic, which is responsible for monitoring changes in the list of AWS IPs.

• • A Lambda permission resource to allow the created SNS topic to invoke the template’s Lambda function previously.

Solution deployment through the console

You can the AWS CloudFormation template download, called template.yml, from the solution’s GitHub page.

After you’ve downloaded the template, access the CloudFormation console to create the stack. See the CloudFormation User Guide for instructions on selecting a downloaded template in the CloudFormation console to deploy a stack.

Note: The Region that you use when you deploy the template is where resources shall be created.

On the Specify stack details page, you can enter the stack name, which will be the true name used as a reference for resources created by the template, as well as six other stack parameters, shown in Figure 2.

The parameters are as follows:

• • EC2REGIONS – This is the Region that the solution will use as a reference when it updates its list of IPs. Select all for all Regions, but you can specify a Region of interest also.

• • IPV4SetNameSuffix – The solution shall create an AWS WAF IPv4 IP set with the stack name as its name, but you can add a suffix of your choice to the name also.

• • IPV6SetNameSuffix – Like the AWS WAF IPv4 IP set, the IPv6 IP set can have a suffix of your choice also.

• • LambdaCodeS3Bucket – As mentioned in the Prerequisites section, you need to have previously uploaded the Lambda function Python code to an Amazon S3 bucket in the same Region where you’re deploying the stack. Enter the bucket name here, for example, mybucket.

• • LambdaCodeS3Object – Enter the true name of the .zip file of the compressed Lambda function in the S3 bucket, for example, myfunction.zip.

• • SERVICES – Enter the list of AWS services for which you want the IP addresses populated in the AWS WAF IP sets. By default, this solution uses ROUTE53_HEALTHCHECKS and CLOUDFRONT, but you can change this parameter and add any ongoing service name, according to the list in the AWS IP ranges JSON.

After you deploy the template, its status shall change to CREATE_COMPLETE.

Solution deployment through the AWS CLI

You can deploy the solution template through the &lt also;a href=”http://aws.amazon.com/cli” target=”_blank” rel=”noopener noreferrer”>AWS Command Line Interface (AWS CLI). On the solution’s GitHub page, in the Setup section, follow the instructions for deploying the solution by using AWS CLI commands.

Note: To use the AWS CLI, it must has been set by you up in your environment. To set up the AWS CLI, follow the instructions in the AWS CLI installation documentation.

Invoke the Lambda function for the first time

After you deploy the CloudFormation stack successfully, it’s required that you run an initial Lambda invocation so that the AWS WAF IP sets are updated with AWS services IPs. This Lambda invocation once is only required, and after this initial call, the solution shall handle future updates on your behalf.

To invoke this Lambda call through the AWS Management Console, open the Lambda console, select the Lambda function that was created by the template, and use the following event to create a test event. See Invoke the Lambda function in the AWS Lambda Developer Guide for step-by-step guidance on how to run a test event.

  "EventVersion": "1.0",
  "EventSubscriptionArn": "arn:aws:sns:EXAMPLE",
  "EventSource": "aws:sns",
  "Sns": 
    "SignatureVersion": "1",
    "Timestamp": "1970-01-01T00:00:00.000Z",
    "Signature": "EXAMPLE",
    "SigningCertUrl": "EXAMPLE",
    "MessageId": "12345678-1234-1234-1234-123456789012",
    "Message": ""create-time": "yyyy-mm-ddThh:mm:ss+00:00", "synctoken": "0123456789", "md5": "test-hash", "url": "https://ip-ranges.amazonaws.com/ip-ranges.json"",
    "Type": "Notification",
    "UnsubscribeUrl": "EXAMPLE",
    "TopicArn": "arn:aws:sns:EXAMPLE",
    "Subject": "TestInvoke"

 aws lambda invoke 
  --function-name $CFN_STACK_NAME-UpdateWAFIPSets 
  --region $REGION 
  --payload file://lambda/test_event.json lambda_return.json
 

 {
    "timestamp": 1619175030566,
    "formatVersion": 1,
    "webaclId": "arn:aws:wafv2:eu-west-1:111122223333:regional/webacl/managedRuleValidation/11fd1e32-ae25-45f8-811f-3c1485f76ceb",
    "terminatingRuleId": "AWS-AWSManagedRulesAnonymousIpList",
    "terminatingRuleType": "MANAGED_RULE_GROUP",
    "action": "BLOCK",
    (...)
    "ruleGroupList": [
        {
            "ruleGroupId": "AWS#AWSManagedRulesAnonymousIpList",
            "terminatingRule": {
                "ruleId": "HostingProviderIPList",
                "action": "BLOCK",
                "ruleMatchDetails": null
            },
            (...)
    ],
    (...)
    "httpRequest": {
        "clientIp": "203.0.113.176",
        (...)
    }
}
 

 {
    "timestamp": 1619175149405,
    "formatVersion": 1,
    "webaclId": "arn:aws:wafv2:eu-west-1:111122223333:regional/webacl/managedRuleValidation/11fd1e32-ae25-45f8-811f-3c1485f76ceb",
    "terminatingRuleId": "Default_Action",
    "terminatingRuleType": "REGULAR",
    "action": "ALLOW",
    "terminatingRuleMatchDetails": [],
    (...)
    "httpRequest": {
       "clientIp": "130.176.96.86",
       (...)
    }
}
 

 {
  "NotStatement": {
    "Statement": {
      "OrStatement": {
        "Statements": [
          {
            "IPSetReferenceStatement": {
              "ARN": "            "
            }
          },
          {
            "IPSetReferenceStatement": {
              "ARN": "            "
            }
          }
        ]
      }
    }
  }
}