Artie's Typing Library

​

Overview 🎯

Artie’s custom typing library is the backbone of our data transfer system. It ensures that your data maintains its original structure and types when moving between sources and destinations.

​

Key Features ✨

• Schema Evolution: Automatically detects and adds missing columns
• Type Inference: Intelligently determines data types from values
• Performance: 2x faster than Go’s Reflect library
• Data Integrity: Zero data transformation or loss

​

Data Type Handling 🔍

Artie’s typing library intelligently handles data type mapping between your source and destination systems. Here’s how it works:

1. Initial Type Detection: When data first arrives, Artie analyzes the source schema and values to determine the most appropriate data type for your destination.
2. Type Preservation: Once a data type is established in your destination, Artie maintains that type to ensure consistency.
3. Automatic Conversion: If source and destination types differ, Artie automatically converts the data while preserving its integrity.

​

Numbers

We preserve the exact numeric type from your source (Postgres, MySQL, or MongoDB). Here’s how we handle different number formats:

• 5 → Integer
• "5" → String
• 5.0 → Float

​

JSON Objects

We maintain JSON objects exactly as they appear in your source, with no flattening or transformation.

💡 Pro Tip: If you prefer to store JSON as a string, you can explicitly specify the column type as STRING in your schema.

​

Arrays

We support both simple and complex array structures:

• Simple arrays
• Nested arrays with objects

​

Timestamps, Dates, and Times ⏰

Our library supports 15+ different time formats with zero precision loss. We preserve your original time layout using our custom time.Time object.

💡 Pro Tip: Like JSON objects, you can override time type inference by specifying your preferred data type in the schema.

​

Advanced

​

Need Help? 🤝

If you have questions not covered here, reach out to us at [email protected]!