fbpx

How to implement single-user secret rotation using Amazon RDS admin credentials

You might have security or compliance standards that prevent a database user from changing their own credentials and from having multiple users with identical permissions. AWS Secrets Manager offers two rotation strategies for secrets that contain Amazon Relational Database Service (Amazon RDS) credentials: single-user and alternating-user.

   <p>In the preceding scenario, neither single-user rotation nor alternating-user rotation would meet your security or compliance standards. Single-user rotation uses database user credentials in the secret to rotate itself (assuming the user has change-password permissions). Alternating-user rotation uses Amazon RDS admin credentials from another secret to create and update a <code>_clone</code> user credential, which means there are two valid user credentials with identical permissions.</p> 
   <p>In this post, you will learn how to implement a modified alternating-user solution that uses Amazon RDS admin user credentials to rotate database credentials while not creating an identical <code>_clone</code> user. This modified rotation strategy creates a short lag between when the password in the database changes and when the secret is updated. During this brief lag before the new password is updated, database calls using the old credentials might be denied. Test this in your environment to determine if the lag is within an acceptable range.</p> 
   <h2>Walkthrough</h2> 
   <p>In this walkthrough, you will learn how to implement the modified rotation strategy by modifying the existing alternating-user rotation template. To accomplish this, you need to complete the following:</p> 
   <ul> 
    <li>Configure alternating-user rotation on the database credential secret for which you want to implement the modified rotation strategy.</li> 
    <li>Modify your <a href="https://aws.amazon.com/lambda/" target="_blank" rel="noopener">AWS Lambda</a> rotation function template code to implement the modified rotation strategy.</li> 
    <li>Test the modified rotation strategy on your database credential secret and verify that the secret was rotated while also not creating a <code>_clone</code> user.</li> 
   </ul> 
   <h3>To configure alternating-user rotation on the database credential secret</h3> 
   <ol> 
    <li>Follow this AWS Security Blog post to <a href="https://aws.amazon.com/blogs/security/improve-security-of-amazon-rds-master-database-credentials-using-secrets-manager/" target="_blank" rel="noopener">set up alternating-user rotation on an Amazon RDS instance</a>.</li> 
    <li>When configuring rotation for the database user secret in the Secrets Manager console, clear the checkbox for <strong>Rotate immediately when the secret is stored. The next rotation will begin on your schedule</strong> in the <strong>Rotation schedule</strong> tab. Make sure that no <code>_clone</code> user is created by the default alternating-user rotation code through your database’s user tables.</li> 
   </ol> 

   <div id="attachment_34325" class="wp-caption aligncenter"> 
    <a href="https://infracom.com.sg/wp-content/uploads/2024/05/img1.jpg"><img aria-describedby="caption-attachment-34325" src="https://infracom.com.sg/wp-content/uploads/2024/05/img1.jpg" alt="Figure 1: Clear the checkbox for Rotate immediately when the secret is stored" width="780" class="size-full wp-image-34325"></a> 
    <p id="caption-attachment-34325" class="wp-caption-text">Figure 1: Clear the checkbox for <strong>Rotate immediately when the secret is stored</strong></p> 
   </div> 

   <h3>To modify your Lambda function rotation Lambda template to implement the modified rotation strategy</h3> 
   <ol> 
    <li>In the Secrets Manager console, select the <strong>Secrets</strong> menu from the left pane. Then, select the new database user secret’s name from the <strong>Secret name</strong> column.  
     <div id="attachment_34326" class="wp-caption aligncenter"> 
      <a href="https://infracom.com.sg/wp-content/uploads/2024/05/img2.jpg"><img aria-describedby="caption-attachment-34326" loading="lazy" src="https://infracom.com.sg/wp-content/uploads/2024/05/img2.jpg" alt="Figure 2: Select the new database user secret" width="1535" height="319" class="size-full wp-image-34326"></a> 
      <p id="caption-attachment-34326" class="wp-caption-text">Figure 2: Select the new database user secret</p> 
     </div> </li> 
    <li>Select the <strong>Rotation</strong> tab on the Secrets page, and then choose the link under <strong>Lambda rotation function</strong>.  
     <div id="attachment_34327" class="wp-caption aligncenter"> 
      <a href="https://infracom.com.sg/wp-content/uploads/2024/05/img3.jpg"><img aria-describedby="caption-attachment-34327" src="https://infracom.com.sg/wp-content/uploads/2024/05/img3.jpg" alt="Figure 3: Select the Lambda rotation function" width="740" class="size-full wp-image-34327"></a> 
      <p id="caption-attachment-34327" class="wp-caption-text">Figure 3: Select the Lambda rotation function</p> 
     </div> </li> 
    <li>From the rotation Lambda menu, <strong>Download</strong> select <strong>Download function code.zip</strong>.  
     <div id="attachment_34328" class="wp-caption aligncenter"> 
      <a href="https://infracom.com.sg/wp-content/uploads/2024/05/img4.jpg"><img aria-describedby="caption-attachment-34328" src="https://infracom.com.sg/wp-content/uploads/2024/05/img4.jpg" alt="Figure 4: Select Download function code .zip from Download" width="740" class="size-full wp-image-34328"></a> 
      <p id="caption-attachment-34328" class="wp-caption-text">Figure 4: Select <strong>Download function code .zip</strong> from <strong>Download</strong></p> 
     </div>  </li> 
    <li>Unzip the .zip file. Open the <span>lambda_function.py</span> file in a code editor and make the following code changes to implement the modified rotation strategy. <p>The following code changes show how to modify a rotation function for the <a href="https://github.com/aws-samples/aws-secrets-manager-rotation-lambdas/blob/master/SecretsManagerRDSMySQLRotationMultiUser/lambda_function.py" target="_blank" rel="noopener">MySQL alternating-user rotation code template</a>. You must make similar changes in the <code>CreateSecret</code> and <code>SetSecret</code> steps of the alternating-user rotation code template for your database’s engine type.</p> <p>To make the needed changes, remove the lines of code that are in <em>grey italic</em> and add the lines of code that are <strong>bold</strong>.</p> <p>Consider using <a href="https://docs.aws.amazon.com/lambda/latest/dg/configuration-versions.html" target="_blank" rel="noopener">AWS Lambda function versions</a> to enable reverting your Lambda function to previous iterations in case this modified rotation strategy goes wrong.</p> <h4>In create_secret()</h4> <p>The following code suggestion removes the creation of <code>_clone-suffixed</code> usernames.</p> <p>Remove:</p> 
     <div class="hide-language"> 
      <pre class="unlimited-height-code"><code class="lang-text">-- # Get the alternate username swapping between the original user and the user with _clone appended to it

— current_dict[‘username’] = get_alt_username(current_dict[‘username’])

In set_secret()

The following code suggestions remove the creation of _clone-suffixed usernames and subsequent checks for such usernames in conditional logic.

Keep:

     <div class="hide-language"> 
      <pre class="unlimited-height-code"><code class="lang-text"># Get username character limit from environment variable

username_limit = int(os.environ.get(‘USERNAME_CHARACTER_LIMIT’, ’16’))

Remove:

-- # Get the alternate username swapping between the original user and the user with _clone appended to it
-- current_dict['username'] = get_alt_username(current_dict['username'])

Keep:

# Check that the username is within correct length requirements for version

Remove:

-- if current_dict['username'].endswith('_clone') and len(current_dict['username']) > username_limit:

Add:

++ if len(current_dict[‘username’]) > username_limit:

Keep:

raise ValueError("Unable to clone user, username length with _clone appended would exceed %s character
s" % username_limit)

Make sure the user from current and pending match

     </div> <p>Remove:</p> 
     <div class="hide-language"> 
      <pre class="unlimited-height-code"><code>-- if get_alt_username(current_dict['username']) != pending_dict['username']:</code></pre> 
     </div> <p>Add:</p> 
     <div class="hide-language"> 
      <pre class="unlimited-height-code"><code>++ if current_dict['username'] != pending_dict['username']:</code></pre> 
     </div> <p>Remove:</p> 
     <div class="hide-language"> 
      <pre class="unlimited-height-code"><code class="lang-text">-- def get_alt_username(current_username):

— “””Gets the alternate username for the current_username passed in

— This helper function gets the username for the alternate user based on the passed in current username.

— Args:

— current_username (client): The current username

— Returns:

— AlternateUsername: Alternate username

— Raises:

— ValueError: If the new username length would exceed the maximum allowed

— “””
— clone_suffix = “_clone”
— if current_username.endswith(clone_suffix):
— return current_username[:(len(clone_suffix) * -1)]
— else:

— return current_username + clone_suffix

     </div> 

The following code suggestions remove the logic of creating a new _clone user within the database, and rotates the existing user’s password.

Keep:

     <div class="hide-language"> 
      <pre class="unlimited-height-code"><code>with conn.cursor() as cur:</code></pre> 
     </div> <p>Remove:</p> 
     <div class="hide-language"> 
      <pre class="unlimited-height-code"><code class="lang-text">--   cur.execute("SELECT User FROM mysql.user WHERE User = %s", pending_dict['username'])

— # Create the user if it does not exist
— if cur.rowcount == 0:

— cur.execute(“CREATE USER %s IDENTIFIED BY %s”, (pending_dict[‘username’], pending_dict[‘password’]))

— # Copy grants to the new user
— cur.execute(“SHOW GRANTS FOR %s”, current_dict[‘username’])
— for row in cur.fetchall():
— grant = row[0].split(‘ TO ‘)
— new_grant_escaped = grant[0].replace(‘%’, ‘%%’) # % is a special character in Python format strings.
— cur.execute(new_grant_escaped + ” TO %s”, (pending_dict[‘username’],))

Keep:

    # Get the version of MySQL
    cur.execute("SELECT VERSION()")
    ver = cur.fetchone()[0]

Remove:

--   # Copy TLS options to the new user
--   escaped_encryption_statement = get_escaped_encryption_statement(ver)
--   cur.execute("SELECT ssl_type, ssl_cipher, x509_issuer, x509_subject FROM mysql.user WHERE User = %s", current_dict['username'])
--   tls_options = cur.fetchone()
--   ssl_type = tls_options[0]
--   if not ssl_type:
--       cur.execute(escaped_encryption_statement + " NONE", pending_dict['username'])
--   elif ssl_type == "ANY":
--       cur.execute(escaped_encryption_statement + " SSL", pending_dict['username'])
--   elif ssl_type == "X509":
--       cur.execute(escaped_encryption_statement + " X509", pending_dict['username'])
--   else:
--       cur.execute(escaped_encryption_statement + " CIPHER %s AND ISSUER %s AND SUBJECT %s", (pending_dict['username'], tls_options[1], tls_options[2], tls_options[3]))

Keep:

    # Set the password for the user and commit
    password_option = get_password_option(ver)
    cur.execute("SET PASSWORD FOR %s = " + password_option, (pending_dict['username'], pending_dict['password']))
    conn.commit()
    logger.info("setSecret: Successfully set password for %s in MySQL DB for secret arn %s." % (pending_dict['username'], arn))

  • Re-zip the folder with the local code changes. From the rotation Lambda menu, under the Code tab, choose Upload from and select .zip file. Upload the new .zip file.
    Figure 5: Use Upload from to upload the new .zip file as the Code source

    Figure 5: Use Upload from to upload the new .zip file as the Code source

  •    <h3>To test the modified rotation strategy</h3> 
       <ol> 
        <li>During the next scheduled rotation for the new database user secret, the modified rotation code will run. To test this immediately, select the <strong>Rotation</strong> tab within the <strong>Secrets</strong> menu and choose <strong>Rotate secret immediately</strong> in the <strong>Rotation configuration</strong> section.<br />
         <div id="attachment_34338" class="wp-caption aligncenter"> 
          <a href="https://infracom.com.sg/wp-content/uploads/2024/05/img6-1.jpg"><img aria-describedby="caption-attachment-34338" src="https://infracom.com.sg/wp-content/uploads/2024/05/img6-1.jpg" alt="Figure 6: Choose Rotate secret immediately to test the new rotation strategy" width="740" class="size-full wp-image-34338" /></a> 
          <p id="caption-attachment-34338" class="wp-caption-text">Figure 6: Choose <strong>Rotate secret immediately</strong> to test the new rotation strategy</p> 
         </div>  </li> 
        <li>To verify that the modified rotation strategy worked, verify the sign-in details in both the secret and the database itself. 
         <ol> 
          <li>To verify the Secrets Manager secret, select the <strong>Secrets</strong> menu in the Secrets Manager console and choose <strong>Retrieve secret value</strong> under the <strong>Overview</strong> tab. Verify that the <strong>username</strong> doesn’t have a <code>_clone</code> suffix and that there is a new <strong>password</strong>. Alternatively, make a <a href="https://docs.aws.amazon.com/cli/latest/reference/secretsmanager/get-secret-value.html" target="_blank" rel="noopener">get-secret-value </a>call on the secret through the <a href="https://aws.amazon.com/cli/" target="_blank" rel="noopener">AWS Command Line Interface (AWS CLI)</a> and verify the <code>username</code> and <code>password</code> details.<br />
           <div id="attachment_34339" class="wp-caption aligncenter"> 
            <a href="https://infracom.com.sg/wp-content/uploads/2024/05/img7.jpg"><img aria-describedby="caption-attachment-34339" src="https://infracom.com.sg/wp-content/uploads/2024/05/img7.jpg" alt="Figure 7: Use the secret details page for the database user secret to verify that the value of the username key is unchanged" width="700" class="size-full wp-image-34339" /></a> 
            <p id="caption-attachment-34339" class="wp-caption-text">Figure 7: Use the secret details page for the database user secret to verify that the value of the <strong>username</strong> key is unchanged</p> 
           </div> </li> 
          <li>To verify the sign-in details in the database, sign in to the database with admin credentials. Run the following database command to verify that no users with a <code>_clone</code> suffix exist: <code>SELECT * FROM mysql.user;</code>. Make sure to use the commands appropriate for your database engine type.</li> 
          <li>Also, verify that the new sign-in credentials in the secret work by signing in to the database with the credentials.</li> 
         </ol> </li> 
       </ol> 
       <h2>Clean up the resources</h2> 
       <ul> 
        <li>Follow the <strong>Clean up the resources</strong> section from the <a href="https://aws.amazon.com/blogs/security/improve-security-of-amazon-rds-master-database-credentials-using-secrets-manager/" target="_blank" rel="noopener">AWS Security Blog post</a> used at the start of this walkthrough.</li> 
       </ul> 
       <h2>Conclusion</h2> 
       <p>In this post, you’ve learned how to configure rotation of Amazon RDS database users using a modified alternating-users rotation strategy to help meet more specific security and compliance standards. The modified strategy ensures that database users don’t rotate themselves and that there are no duplicate users created in the database.</p> 
       <p>You can start implementing this modified rotation strategy through the <a href="https://console.aws.amazon.com/secretsmanager/home" target="_blank" rel="noopener">AWS Secrets Manager console</a> and <a href="https://console.aws.amazon.com/rds/home" target="_blank" rel="noopener">Amazon RDS console</a>. To learn more about Secrets Manager, see the <a href="https://aws.amazon.com/documentation/secretsmanager/" target="_blank" rel="noopener">Secrets Manager documentation</a>.</p> 
       <p>If you have feedback about this post, submit comments in the Comments section below. If you have questions about this post, start a new thread on <a href="https://repost.aws/tags/TAJAqKRtZ5RY6XCoNAUTcVzQ/aws-secrets-manager" target="_blank" rel="noopener">AWS Secrets Manager re:Post</a> or <a href="https://console.aws.amazon.com/support/home" target="_blank" rel="noopener">contact AWS Support</a>.</p> 
       <p><strong>Want more AWS Security news? Follow us on <a title="Twitter" href="https://twitter.com/AWSsecurityinfo" target="_blank" rel="noopener noreferrer">X</a>.</strong>
    
       <!-- '"` -->