Iceberg
REST catalog
How to configure Iceberg REST Catalog as a destination in Artie
Required settings
- REST Catalog URI
- Authentication (Bearer token or OAuth2)
- Catalog name
Table format
The Table format setting in the pipeline’s destination settings picks the Iceberg version for new tables. It defaults to Iceberg v2, which nearly every engine can read, including Athena, StarRocks, and older Spark and Trino versions. Tables that already have data keep their version.
Iceberg v3 stores deletes as deletion vectors, so updates and deletes write less. Pick it only when every engine that queries these tables supports v3. A table created as v3 can’t go back to v2.
Databricks Unity catalog
To configure Unity catalog as a REST catalog, you will need to do the following:
- Enable external data access
- Modify the service principal to have the following permissions:
EXTERNAL USE SCHEMA,EXTERNAL USE LOCATIONUSE CATALOG,USE SCHEMASELECTMANAGE,CREATE SCHEMA,CREATE TABLE,MODIFY,WRITE VOLUME
Grant each privilege to the service principal by its application ID. For example:
GRANT EXTERNAL USE SCHEMA ON SCHEMA catalog_name.schema_name TO `6f0c8a4e-2b1d-4c7a-9e3f-5d8b1a2c4e6f`;Once this is done, you’ll then want to grab:
- REST Catalog URI (
https://{{DATABRICKS_INSTANCE_URL}}/api/2.1/unity-catalog/iceberg-rest) - Catalog name
Authentication
A bearer token is a personal access token, so Artie acts as the user who created it. Grant the permissions above to that user, or use OAuth2 M2M with the service principal instead.
To generate a personal access token:
Go into your workspace and do the following:
- Click on your username in the top bar and select Settings from the dropdown.
- Click Developer
- Click Access tokens then Generate new token
- (Optional) Provide a description for the token and if you set expiration, make sure to note it down.

To generate an OAuth2 M2M token, you’ll want to generate an OAuth2 token for your service principal:
- Follow this guide to create an OAuth secret for your service principal
- Put the token URL as
https://{{DATABRICKS_INSTANCE_URL}}/oidc/v1/token - Put
all-apisas the OAuth2 scope