Project Documentation
Credential Rotation Automation
This project documents a credential rotation automation workflow for IAM access keys and SES SMTP credentials used by cloud-hosted applications and supporting services. The work focused on reducing the risk of long-lived credentials, standardizing how new credentials were created and delivered, improving audit readiness, and creating a safer handoff process for applications that depended on SMTP authentication or stored service credentials.
Overview
This project documents a credential rotation automation workflow for IAM access keys and SES SMTP credentials used by cloud-hosted applications and supporting services. The work focused on reducing the risk of long-lived credentials, standardizing how new credentials were created and delivered, improving audit readiness, and creating a safer handoff process for applications that depended on SMTP authentication or stored service credentials.
The automation used AWS Lambda, Python, EventBridge scheduled execution, DynamoDB configuration tracking, Systems Manager Parameter Store, Secrets Manager, CloudWatch Logs, SNS notifications, IAM access key APIs, and validation logic. The goal was not simply to create new access keys, but to create a controlled rotation workflow that could identify when credentials needed attention, update the right storage location, notify the right owners, preserve rollback options, and reduce manual handling of sensitive values.
Background
IAM access keys and SES SMTP credentials are common in older application integrations, database mail configurations, vendor workflows, and systems that cannot use modern role-based authentication directly. Over time, these credentials can become difficult to manage if they are manually created, copied into applications, stored in different locations, or rotated only when an audit finding appears.
In a cloud operations environment, credential rotation is both a security task and an operational task. A new key must be created securely, converted or delivered in the required format, placed into the correct storage target, validated by the consuming application, and only then should the old key be disabled or deleted. If this sequence is not controlled, a rotation effort can accidentally break application mail flow or require emergency rollback.
The project addressed this by treating credential rotation as a repeatable platform workflow. Instead of relying on manual key creation and one-off handoffs, the process used automation, configuration data, notification handling, and validation checkpoints to make rotations safer and easier to operate.
Business Problem
The business problem was credential risk and inconsistent operational handoff. Long-lived IAM access keys increase exposure if they are not rotated, tracked, and retired properly. At the same time, manually rotating keys can create service disruption if application owners do not receive updated credentials, if a storage location is missed, or if an old key is deleted before the application has successfully adopted the new one.
Different applications also stored credentials in different ways. Some applications consumed values from Systems Manager Parameter Store. Some used Secrets Manager. Some required updates to database-backed configurations or EC2-hosted services. This made a single manual process difficult to standardize and increased the chance that a rotation would be handled differently from one application to another.
The platform need was to create a safer credential lifecycle: discover or track configured users, evaluate key age, create new credentials only when needed, update the approved storage target, notify owners, validate success, retain rollback options where appropriate, and clean up old credentials after the new credential path was confirmed.
Architecture
The architecture used a scheduled Lambda function as the orchestration layer for credential rotation. EventBridge triggered the automation on a defined schedule. DynamoDB acted as the configuration source that identified which credentials were in scope, where they were stored, which account or target needed to be updated, and who needed to be notified.
Lambda used AWS SDK and AWS CLI style operations through Python to inspect IAM access keys, determine whether a key met rotation criteria, create a replacement access key, and update the required credential storage target. Depending on the application pattern, the updated credential could be delivered to Systems Manager Parameter Store, Secrets Manager, or another approved application configuration target.
Systems Manager Parameter Store and Secrets Manager provided secure storage locations for applications that could read credentials through AWS APIs. IAM instance profiles or application roles were used so EC2-hosted applications could retrieve approved parameters or secrets without hardcoding credentials in source code, configuration files, or tickets. CloudWatch Logs captured execution details, and SNS notifications provided operational visibility for completed rotations, skipped users, failures, and manual review conditions.
The design also considered rollback and service continuity. New credentials were created first, then storage targets were updated, then application validation or notification occurred, and only after the update path was considered successful would the old credential be deactivated or deleted according to the rotation workflow. This sequence reduced the chance of disrupting applications during a credential change.
What I Did
I supported the credential rotation workflow by helping define and document how IAM access keys and SES SMTP credentials should be discovered, evaluated, rotated, stored, validated, and handed off to application owners. The work included reviewing key age, credential storage patterns, Parameter Store usage, Secrets Manager usage, DynamoDB configuration records, Lambda execution behavior, CloudWatch logs, SNS notifications, and the operational sequence required to avoid breaking application connectivity.
I also supported the operational handoff process. Credential rotation is only successful when the consuming application can use the new credential. That required clear communication, a defined update path, validation of application behavior, and a safe approach for old key retirement. The process helped reduce manual key handling and gave owners a clearer path for updating applications without exposing credentials through email, tickets, or source code.
The project also involved thinking through failure cases. If a new key was created but the storage update failed, the new key needed to be removed and the old key preserved. If both keys were already in an unsafe or aged state, the automation needed to treat that as a manual intervention condition rather than blindly making the situation worse. This made the automation more operationally safe and more suitable for production support.
Implementation Details
1. Scheduled rotation orchestration
EventBridge provided scheduled execution for the rotation workflow. This allowed the credential review process to run consistently without depending on a person manually starting the rotation. The schedule triggered a Lambda function that handled discovery, evaluation, target updates, logging, and notification.
2. Configuration tracking with DynamoDB
DynamoDB was used as a configuration source for the rotation workflow. Configuration data identified which IAM users or SES SMTP users were in scope, what type of credential storage target applied, which account or application context was involved, and which owner or notification path should receive updates. This separated rotation logic from hardcoded application-specific values and made the workflow easier to extend.
3. IAM access key review
The automation reviewed IAM access keys to determine whether a credential needed rotation. This included checking the existing keys for a configured user, determining whether a new key was needed, and handling edge cases such as missing keys, inactive keys, or situations where multiple old keys required manual review. This helped prevent the automation from making unsafe assumptions about credential state.
4. SES SMTP credential generation
SES SMTP credentials are derived from IAM access key material and must be delivered in the format expected by the application or mail configuration. The rotation workflow created a new IAM access key, generated the corresponding SMTP credential, and prepared the values for secure storage. This helped standardize the conversion and delivery process rather than requiring manual credential generation.
5. Parameter Store updates
For applications that consumed credentials from Systems Manager Parameter Store, the automation updated the configured parameters with the new username and SMTP password values. Parameter Store created a straightforward handoff model for EC2-hosted or application-based workloads that already had IAM permissions to retrieve approved parameters.
6. Secrets Manager updates
For applications that used Secrets Manager, the workflow stored the new credential pair in the appropriate secret structure. Secrets Manager was useful for workloads that required a stronger secret-management pattern, application-side retrieval, and separation of secret values from application code or manual operational notes.
7. Application handoff and validation
Rotation was treated as an operational handoff, not just a key creation event. Application owners or operational contacts needed to know when a credential was updated, where the new value was stored, and what validation was expected. This helped ensure that applications could adopt the new credential before the old key was fully retired.
8. Rollback and old key handling
The workflow was designed to preserve service continuity. If a storage update or validation step failed, the new key could be deleted while the old key remained active. When rotation succeeded, the old key could be retained temporarily or retired according to the defined process. This reduced the risk of a rotation causing an immediate outage.
9. Monitoring and notifications
CloudWatch Logs captured execution details, skipped records, successful updates, and errors. SNS notifications provided a way to alert operational contacts when a rotation completed, when manual action was required, or when a failure needed investigation. This made the process easier to support and review after each scheduled run.
Validation
Validation began with confirming that the Lambda function executed successfully and processed the expected configuration records. CloudWatch Logs were reviewed to confirm which credentials were skipped, which were rotated, which storage targets were updated, and whether any errors occurred during role assumption, key creation, storage update, notification, or cleanup.
Credential validation focused on confirming that the new value was written to the correct target and that the application handoff path was clear. For Parameter Store, this meant confirming that the expected parameters were updated. For Secrets Manager, this meant confirming that the new secret value was stored in the expected structure. For application-dependent targets, validation also required confirmation that the consuming system could use the new credential successfully.
Failure validation was equally important. If a rotation failed, the automation needed to preserve the old working credential and remove the newly created key when appropriate. This helped avoid leaving unused access keys behind and prevented failed rotations from creating additional credential risk.
Operational Workflow
The operational workflow started with the scheduled EventBridge trigger. Lambda read the configured rotation targets from DynamoDB, evaluated the current IAM access key state, and determined whether each credential required rotation. If a credential did not meet the rotation criteria, it was skipped and logged. If it did require rotation, the automation created a new key and prepared the SES SMTP credential value.
After the new credential was created, the automation updated the appropriate storage target. For Parameter Store targets, the relevant parameters were updated through Systems Manager APIs. For Secrets Manager targets, the secret value was updated with the new username and password structure. For application-specific targets, the workflow followed the configured update method and waited for completion or confirmation where required.
Once the storage target was updated, the process moved into notification and validation. Owners or operational contacts were notified that credentials had been rotated or that an update was pending application action. CloudWatch Logs and notification output provided a record of what occurred during the run. After the new credential path was confirmed, the old key could be deactivated or deleted according to the defined grace and cleanup process.
When failures occurred, the workflow followed a safer path. The new key was removed where appropriate, the old key remained active, the error was logged, and a notification was sent for review. This kept the process operationally safe and reduced the risk of causing application disruption during automated rotation.
Outcome
The project reduced credential risk by moving key rotation away from ad hoc manual handling and toward a controlled automation workflow. It improved audit posture by creating a more consistent process for reviewing key age, rotating credentials, logging activity, notifying stakeholders, and tracking failures or manual intervention conditions.
The automation also improved operational handoff. Instead of passing sensitive values through informal channels, credentials could be stored in approved AWS services such as Parameter Store or Secrets Manager and retrieved by applications through controlled IAM permissions. This created a cleaner separation between the platform team, application owners, and the credential storage layer.
The biggest operational value was safer repeatability. The process created a defined sequence for creating new keys, updating storage targets, validating use, notifying owners, and retiring old keys. That sequence helped reduce manual key handling, supported clearer ownership, and made credential rotation easier to operate over time.
What This Demonstrates
This project demonstrates practical AWS security automation and cloud operations experience. It shows knowledge of IAM access keys, SES SMTP credential handling, Lambda orchestration, EventBridge schedules, DynamoDB configuration tracking, Systems Manager Parameter Store, Secrets Manager, CloudWatch Logs, SNS notifications, Python automation, validation, rollback handling, and operational handoff to applications.
It also demonstrates platform engineering judgment. Credential rotation is not only a scripting task. It requires understanding application dependencies, secure storage, service continuity, audit expectations, owner communication, failure handling, and cleanup. This project shows how those concerns can be organized into a repeatable automation model that reduces credential exposure while supporting real production operations.