Skip to main content

Required settings

  • Account identifier
  • Virtual warehouse
  • Service account
  • Database name
  • Schema name
You can find your account identifier at the bottom left corner of your Snowflake console.Retrieving your account identifier
Artie signs in to Snowflake with a key pair. Generate one first:
Then run this script, with the public key in rsa_public_key:
In Artie, choose Private key as the authentication method and paste the contents of rsa_key.p8, including its BEGIN and END lines, into Private key.Password (not recommended): Artie also accepts a password. Replace the rsa_public_key line with password = '<password>' and choose Password as the authentication method.
If the service account already exists, generate a key pair with the commands above and assign the public key to the user:
Then paste the private key into Private key in Artie.

Advanced features

Snowflake turbo mode

Snowflake turbo mode allows you to switch between warehouse sizes based on your data volume and lag.
Artie continuously monitors the ingestion lag and row backlog for each pipeline writing to Snowflake. When both metrics exceed configurable thresholds, Artie automatically switches the pipeline to a larger Snowflake warehouse to accelerate catch-up. Once the backlog clears and metrics stay below thresholds for several consecutive checks, Artie scales back down to the pipeline’s normal warehouse.

How it works

1

Monitoring

Every 10 minutes, Artie queries the pipeline’s current ingestion latency (in minutes) and pending row count.
2

Scale up

If both the latency and row backlog exceed their thresholds, the pipeline is switched to the configured turbo warehouse and you are notified.
3

Scale down

Once both metrics drop below thresholds for 3 consecutive checks, the pipeline is returned to its normal warehouse and you are notified again.

Configuration

You can activate this feature by going to your pipeline’s destination advanced settings.

Key behaviors

  • Both thresholds must be exceeded simultaneously to activate turbo mode. A spike in only one metric will not trigger a scale-up.
  • Scale-down requires 3 consecutive below-threshold checks to avoid flapping between warehouses.
  • You will receive a notification whenever turbo mode is activated or deactivated.
Snowflake turbo mode

Snowflake eco mode

Snowflake eco mode suits pipelines that only need fast syncs during business hours. Read more in Snowflake eco mode.
  1. Edit your pipeline.
  2. Open the Destination tab, then Advanced settings.
  3. Under Optimizations, select Enable Snowflake Eco Mode, then New schedule.
  4. Name the schedule and set its peak hours and Timezone.
  5. Set the flush rules for off-peak hours.
  6. Select Save schedule.
  7. Deploy your changes.
Setting up Snowflake eco mode

Rebuild and swap backfills

A backfill can load into a new table and swap it with the destination table when it finishes, so queries never see the table empty or partially loaded. This is in early access. See Rebuild and swap backfills.

Troubleshooting

To start, We would recommend you to go with a x-small and decide whether or not you need to scale based off of our Snowflake insights metrics from our Analytics portal.Generally speaking, if your overall query execution time is high, then you may want to scale to a larger instance. If your queued overload time is high, you may want to have more virtual warehouse nodes to increase your parallelizable slots.
Snowflake’s native RBAC makes it so that the account that created the resource is the native owner. To change this, assign the ARTIE service account’s role to your account and you will be able to operate on the table. See the GIF below on how to fix this problem.Snowflake access control