Loading repository data…
nathadriele / repository
The AWS DMS Task Restart and Status Checker is a Python script designed to restart various AWS Database Migration Service (DMS) tasks and check their status. This script leverages AWS SDK (Boto3) and Mage.ai for DMS task integration and management, ensuring efficient and reliable task handling.
A transparent discovery signal based on current public GitHub metadata.
This score does not audit code, security, maintainers, documentation quality, or suitability. Verify the repository and its current documentation before adoption.
AWS DMS Task Restart and Status Checker is a Python-based automation project designed to monitor, restart, and validate AWS Database Migration Service replication tasks.
The project uses Boto3 to interact with AWS DMS and Mage.ai to orchestrate the execution as a data pipeline block. It supports scenarios where multiple DMS replication tasks need to be restarted and cases where a task should only be restarted when it is in a failed state.
This solution is useful for data engineering workflows that depend on stable, automated, and observable database migration pipelines.
AWS-DMS-task-restart-and-status-checker/
├── README.md
├── requirements.txt
└── mage_blocks/
├── restart_multiple_dms_tasks.py
└── check_and_restart_dms_task.py
Before running this project, make sure you have:
The AWS user or role used by the pipeline must have permissions similar to:
{
"Effect": "Allow",
"Action": [
"dms:DescribeReplicationTasks",
"dms:StartReplicationTask"
],
"Resource": "*"
}
For production environments, it is recommended to restrict the Resource field to the specific DMS task ARNs used by the project.
Clone the repository:
git clone https://github.com/nathadriele/AWS-DMS-task-restart-and-status-checker.git
cd AWS-DMS-task-restart-and-status-checker
Selected from shared topics, language and repository description—not editorial ratings.
nathadriele /
Contains a Python script designed to validate data replication tasks in AWS Database Migration Service (DMS). The script checks for potential issues such as missing or duplicate data in the tables being replicated.
56/100 healthnathadriele /
Automated script developed in Mage.ai to monitor and validate the status of AWS DMS CDC tasks, ensuring data integrity and synchronization. Sends notifications for any detected validation issues.
55/100 healthInstall the required dependencies:
pip install boto3 mage-ai
Alternatively, if using a requirements.txt file:
pip install -r requirements.txt
This project expects the AWS credentials to be available through Mage.ai secrets.
Configure the following secrets in Mage.ai:
aws_access_key_id
aws_secret_access_key
These values are accessed in the code using:
get_secret_value('aws_access_key_id')
get_secret_value('aws_secret_access_key')
This approach restarts a predefined list of DMS replication tasks and returns their current status after the restart command is triggered.
Example task list:
task_arns = [
'arn:aws:dms:us-east-1:account-id:task:your-task-id1',
'arn:aws:dms:us-east-1:account-id:task:your-task-id2',
'arn:aws:dms:us-east-1:account-id:task:your-task-id3',
'arn:aws:dms:us-east-1:account-id:task:your-task-id4',
'arn:aws:dms:us-east-1:account-id:task:your-task-id5'
]
Each task is restarted using:
client.start_replication_task(
ReplicationTaskArn=task_arn,
StartReplicationTaskType='reload-target'
)
After the restart command, the script checks the task status using:
client.describe_replication_tasks(
Filters=[{'Name': 'replication-task-arn', 'Values': [task_arn]}]
)
The output is a dictionary containing each task ARN and its returned status.
Example output:
{
"arn:aws:dms:us-east-1:account-id:task:your-task-id1": "starting",
"arn:aws:dms:us-east-1:account-id:task:your-task-id2": "starting"
}
This approach checks the current status of a DMS replication task before deciding whether to restart it.
The task is restarted only when its current status is one of the following:
['failed', 'error', 'failed-move']
If the task is not in a failed state, no restart is performed.
Example behavior:
Current task status: running
The task is not in a failed state. No need to restart.
If the task is failed:
Current task status: failed
Restarting the DMS task...
DMS Task Restarted
starting
The task is running or completed.
After a restart, the script monitors the DMS task status in a loop.
Successful or acceptable states:
[
'stopped',
'running',
'ready',
'starting',
'stopping',
'starting-replication',
'replicating'
]
Failed states:
[
'failed',
'failed-move',
'error'
]
If the task reaches an acceptable state, the function returns:
{'status': 1}
If the task reaches a failed state, the function returns:
{'status': 0}
Mage.ai allows testing pipeline block outputs using the @test decorator.
For the multiple-task restart approach, the test validates whether:
Example:
@test
def test_output(output, *args) -> None:
expected_status = 'starting'
assert isinstance(output, dict), 'Output is not a dictionary'
for task_arn, status in output.items():
assert status == expected_status, (
f'Task {task_arn} did not start correctly. Status: {status}'
)
For the conditional restart approach, a recommended test is:
@test
def test_output(output, *args) -> None:
assert isinstance(output, dict), 'Output must be a dictionary'
assert 'status' in output, 'Output must contain the status key'
assert output['status'] in [0, 1], 'Status must be either 0 or 1'
requirements.txtboto3
mage-ai
This project contributes to data engineering workflows by automating the operational management of AWS DMS replication tasks. It reduces manual intervention, improves reliability, and provides a reusable pattern for monitoring and recovering data migration processes.
By integrating AWS DMS, Boto3, and Mage.ai, the project demonstrates a practical approach to building cloud-native automation for ETL and data migration pipelines.