-
Notifications
You must be signed in to change notification settings - Fork 292
Description
Hi there,
I'm attempting to set up an Aurora v2 Cluster with the following users with Auto-Rotate password turned on for all of them:
postgres(admin)lambda_writer(connects to the Aurora Writer endpoint)lambda_reader(connects to the Aurora Reader endpoint)
lambda_writer's setup is the default, and it seems to work properly.
I've set up the Database Secret so that lambda_reader's host value is the ReadOnly endpoint for Aurora, as I don't want any users connecting with this secret to tax a potentially stressed Writer instance.
Unfortunately, Aurora v2 Reader instances aren't consider by the RDS client to be "Read Replicas", so when the Multi-User Rotation lambda goes to set the secret, and sees that the hostnames for the master and child secrets differ, the "isReadReplica" (see below) function fails:
aws-secrets-manager-rotation-lambdas/SecretsManagerRDSPostgreSQLRotationMultiUser/lambda_function.py
Lines 501 to 538 in 7f56d1b
| def is_rds_replica_database(replica_dict, master_dict): | |
| """Validates that the database of a secret is a replica of the database of the master secret | |
| This helper function validates that the database of a secret is a replica of the database of the master secret. | |
| Args: | |
| replica_dict (dictionary): The secret dictionary containing the replica database | |
| primary_dict (dictionary): The secret dictionary containing the primary database | |
| Returns: | |
| isReplica : whether or not the database is a replica | |
| Raises: | |
| ValueError: If the new username length would exceed the maximum allowed | |
| """ | |
| # Setup the client | |
| rds_client = boto3.client('rds') | |
| # Get instance identifiers from endpoints | |
| replica_instance_id = replica_dict['host'].split(".")[0] | |
| master_instance_id = master_dict['host'].split(".")[0] | |
| try: | |
| describe_response = rds_client.describe_db_instances(DBInstanceIdentifier=replica_instance_id) | |
| except Exception as err: | |
| logger.warning("Encountered error while verifying rds replica status: %s" % err) | |
| return False | |
| instances = describe_response['DBInstances'] | |
| # Host from current secret cannot be found | |
| if not instances: | |
| logger.info("Cannot verify replica status - no RDS instance found with identifier: %s" % replica_instance_id) | |
| return False | |
| # DB Instance identifiers are unique - can only be one result | |
| current_instance = instances[0] | |
| return master_instance_id == current_instance.get('ReadReplicaSourceDBInstanceIdentifier') |
I've checked as to why with the following setup:
- An Aurora v2 DbCluster with:
- 1 Writer instance
- 1 Reader instance
And then I got the following in the CLI:
aws rds describe-db-instances --region ap-southeast-2 | grep Replica
"ReadReplicaDBInstanceIdentifiers": [],
"ReadReplicaDBInstanceIdentifiers": [],I think the potential solution is to check that the host value is an Endpoint for the given cluster potentially, since Secret.attach adds the dbClusterIdentifier to the Secret Value for Aurora DB Connections, but I'm not sure tbh.
Thanks!