MySQL Source Connector: Setup and Configuration

Required settings

Database connection
Service account, signing in with a password or, on Amazon RDS and Aurora, an IAM role
Database with binlog_format set to ROW
Database with binlog retention hours set to at least 24 hours

Setup

Create a service account

Run the following SQL to create a dedicated user for Artie. In the Dashboard, pick the matching Authentication method.

Static credentials
Role assumption

setup.sql

CREATE USER 'artie' IDENTIFIED BY 'password';
-- SELECT is required for backfills.
-- REPLICATION CLIENT and REPLICATION SLAVE are required for Artie to read the binary log.
GRANT SELECT, REPLICATION CLIENT, REPLICATION SLAVE ON *.* TO 'artie';

Amazon RDS and Aurora only. Artie signs in with an RDS IAM authentication token instead of a password:

setup.sql

CREATE USER 'artie' IDENTIFIED WITH AWSAuthenticationPlugin AS 'RDS';
-- SELECT is required for backfills.
-- REPLICATION CLIENT and REPLICATION SLAVE are required for Artie to read the binary log.
GRANT SELECT, REPLICATION CLIENT, REPLICATION SLAVE ON *.* TO 'artie';

Then follow Set up role assumption.

Enable binlog (ROW format)

Binary logging must be enabled with binlog_format set to ROW.

AWS RDS / Aurora
Self-hosted MySQL

You will need to create or update a parameter group for your RDS instance.

For Aurora clusters, create a DB cluster parameter group instead of a DB parameter group.

Create a parameter group

Navigate to RDS > Parameter groups > Create parameter group. Select the appropriate family for your MySQL version.

Set binlog_format to ROW

Open the newly created parameter group, search for binlog_format, and set it to ROW.

Apply the parameter group

Associate the parameter group with your RDS instance (or Aurora cluster), then reboot the database for the change to take effect.

Add the following to your MySQL configuration file (my.cnf or my.ini):

[mysqld]
binlog_format = ROW

Then restart the MySQL server.

Set binlog retention period

The default binlog retention is NULL, which means logs are not retained. Artie requires at least 24 hours of retention.

AWS RDS / Aurora
Azure
GCP CloudSQL / Self-hosted

CALL mysql.rds_set_configuration('binlog retention hours', 24);

SET GLOBAL binlog_expire_logs_seconds = 604800;

Set up role assumption

Role assumption lets Artie sign in to Amazon RDS or Aurora MySQL with an IAM role instead of a stored password. It works for backfills and binlog replication. Artie assumes the role and signs a new IAM authentication token for each connection, so there is no password to rotate.

Enable IAM database authentication

In the RDS console, modify the instance (or Aurora cluster) and set Database authentication to Password and IAM database authentication. See Enabling and disabling IAM database authentication.

Create the IAM role

Create a role in the AWS account that owns the database and attach a policy that allows rds-db:connect for the Artie user. Replace the Region, account ID, resource ID, and user name with your own. The resource ID is the instance’s DbiResourceId or the Aurora cluster’s resource ID (it starts with db- or cluster-), not its name.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "rds-db:connect",
      "Resource": "arn:aws:rds-db:us-east-1:123456789012:dbuser:db-ABCDEFGHIJKLMNOPQRSTUVWXY/artie"
    }
  ]
}

Add this trust policy so Artie can assume the role. On a data plane you run yourself, trust the IAM identity your data plane runs as instead of Artie’s principal. The external-ID condition is optional: include it only when you enter an AWS external ID in the Dashboard, and replace my-external-id with that exact value.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::788826912768:user/ci"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "my-external-id"
        }
      }
    }
  ]
}

Configure the connector

In the Dashboard, set Authentication method to Role assumption, then enter the AWS role ARN and the database’s AWS region. AWS external ID is optional.Until a role ARN is set, the connector still signs in with its password. Once one is set, the password is no longer used.

IAM authentication tokens are sent over TLS, so role assumption always connects with TLS. If TLS mode is Disabled, setting a role switches it to Preferred.

Role assumption isn’t available over PrivateLink, because the token is signed for the RDS endpoint that PrivateLink replaces. Connect directly or through an SSH tunnel, or use static credentials.

Additional features

GTID support - when enabled, Artie automatically uses GTID-based replication for more resilient syncing.

Enabling GTID

Ensure the following settings are enabled on your MySQL server:

SET GLOBAL gtid_mode = ON;
SET GLOBAL enforce_gtid_consistency = ON;

You can verify the settings with:

SHOW VARIABLES LIKE 'gtid_mode';
SHOW VARIABLES LIKE 'enforce_gtid_consistency';

Once GTID is enabled, Artie will automatically detect it and switch to GTID-based replication. No additional configuration is needed in the Artie dashboard.

Automatic gh-ost migration handling - Artie detects gh-ost schema migrations and processes them seamlessly.

How does Artie handle gh-ost migrations?

gh-ost is a popular tool for online schema migrations in MySQL. During a gh-ost migration, a ghost table is created and data is copied over before the tables are swapped.Artie automatically detects gh-ost ghost tables in the binlog stream and handles the table swap transparently - no manual intervention or pipeline restarts required.

Frequently asked questions

Which MySQL versions are supported?

Artie supports MySQL 5.7 and above, including MySQL 8.x. We also support Amazon Aurora MySQL, Azure Database for MySQL, and GCP CloudSQL for MySQL. For MariaDB, see MariaDB.

Can I replicate from a read replica?

Yes, if the replica writes replicated changes to its own binary log. Artie reads the replica’s local binary log, not its relay log. Configure the replica with:

log_bin = ON
binlog_format = ROW
log_replica_updates = ON for MySQL 8.0.26 and later, or log_slave_updates = ON for MySQL 5.7, MySQL 8.0.25 and earlier, and MariaDB
A healthy replication connection and SQL applier
At least 24 hours of local binlog retention

Use these queries on the replica to verify its configuration:

SHOW VARIABLES WHERE variable_name IN (
  'log_bin',
  'binlog_format',
  'log_replica_updates',
  'log_slave_updates'
);
SHOW REPLICA STATUS; -- Use SHOW SLAVE STATUS for MySQL 5.7, MySQL 8.0.21 and earlier, and MariaDB.
SHOW BINARY LOGS;

The replica must expose local binary logs containing replicated changes. A replica that retains only relay logs cannot be used for CDC. Artie can still use a read replica for backfills.

Can I restrict the service account to Artie’s IP addresses?

Yes. Create the user with a host pattern that matches Artie’s data plane IPs. Use netmask notation rather than CIDR notation: MySQL only understands CIDR host patterns such as 3.41.128.96/27 on 8.0.23 and later. On older versions, a CIDR pattern is stored as a literal hostname that never matches, so logins fail.

-- 3.41.128.96/255.255.255.224 is the same range as 3.41.128.96/27
CREATE USER 'artie'@'3.41.128.96/255.255.255.224' IDENTIFIED BY 'password';
GRANT SELECT, REPLICATION CLIENT, REPLICATION SLAVE ON *.* TO 'artie'@'3.41.128.96/255.255.255.224';

On GCP CloudSQL, a host mismatch surfaces as this user requires clear text authentication instead of Access denied. CloudSQL asks logins that match no built-in user to authenticate as Cloud SQL IAM users, which Artie does not support. If you see this error, check the user’s host pattern first.

How do I verify that binlog is enabled?

Run the following query on your MySQL instance:

SHOW VARIABLES LIKE 'binlog_format';

The result should show ROW. If it shows STATEMENT or MIXED, update your configuration as described in the setup steps above.

What happens if binlog retention is too short?

If binlog files are purged before Artie can read them, the pipeline will need to re-snapshot the affected tables. We recommend at least 24 hours of retention to provide a comfortable buffer.

Getting Started

Pipelines

Connectors

Monitoring

Artie Dashboard

MySQL Source Connector: Setup and Configuration

Required settings

Setup

Set up role assumption

Additional features

Frequently asked questions

Which MySQL versions are supported?

Can I replicate from a read replica?

Can I restrict the service account to Artie’s IP addresses?

How do I verify that binlog is enabled?

What happens if binlog retention is too short?

​Required settings

​Setup

​Set up role assumption

​Additional features

​Frequently asked questions

​Which MySQL versions are supported?

​Can I replicate from a read replica?

​Can I restrict the service account to Artie’s IP addresses?

​How do I verify that binlog is enabled?

​What happens if binlog retention is too short?

Required settings

Setup

Set up role assumption

Additional features

Frequently asked questions

Which MySQL versions are supported?

Can I replicate from a read replica?

Can I restrict the service account to Artie’s IP addresses?

How do I verify that binlog is enabled?

What happens if binlog retention is too short?