Syncing User Directories from External Databases

Many organizations store their user data in existing databases — HR systems, legacy applications, or custom-built directories. Moving to a new identity platform shouldn't mean abandoning that data or maintaining it in two places. Directory sync bridges the gap.

The Problem

Your company has 5,000 employees in a SQL Server database managed by HR. You adopt Calimatic Identity for authentication and access control. Now you need to keep both systems in sync: when HR adds a new employee, they should automatically get an account. When someone leaves, their access should be revoked.

Manual processes don't scale. CSV imports are error-prone. You need automated, scheduled synchronization.

How Directory Sync Works in Calimatic Identity

Our database sync feature connects directly to your existing database and imports user records on a configurable schedule. Here's the architecture:

1. Connection Setup — Provide connection credentials for your database. We support SQL Server, PostgreSQL, and MySQL. Connections are encrypted in transit and credentials are stored using AES-256-GCM encryption.

2. Schema Discovery — Once connected, Calimatic Identity reads your database schema and presents the available tables and columns. No need to write SQL — we handle the query generation.

3. Field Mapping — Map your database columns to Calimatic Identity user fields. Map emp_email to email, full_name to displayName, department to a custom attribute, and so on. We support one-to-one mappings, computed fields, and default values.

4. Sync Execution — On each sync cycle, we query your database for changes, compare against existing records, and apply updates. New records create new users. Modified records update existing profiles. Removed records can trigger deactivation (configurable).

5. Conflict Resolution — When the same user exists in both systems with different data, conflict resolution rules determine which source wins. Options include source-always-wins, target-always-wins, and most-recent-wins.

Scheduling Options

Sync can run on different schedules depending on your needs:

• Manual — Trigger syncs on-demand from the admin panel or API
• Hourly — Good for organizations with frequent HR changes
• Daily — The most common schedule for most organizations
• Real-time — Use webhooks or change data capture for instant sync (Enterprise plan)

Each sync run is logged with detailed statistics: records processed, created, updated, skipped, and errored. You can review the results in the admin panel or query them via the API.

Field Mapping Examples

Here are common mapping patterns:

Simple mapping: Database email → Calimatic email

Computed field: Concatenate first_name and last_name → Calimatic displayName

Conditional mapping: If department equals "Engineering", assign the "developer" role; otherwise, assign "user"

Default values: If country is null, default to "US"

Handling Edge Cases

Duplicate emails — If two database records share an email address, the sync flags a conflict and skips both records rather than corrupting data. Admins are notified to resolve manually.

Character encoding — We normalize Unicode characters during import to prevent issues with special characters in names or departments.

Large datasets — Syncs process records in batches (default 500) to avoid overwhelming either system. For databases with 100,000+ records, initial sync runs in the background with progress tracking.

Network failures — If a sync is interrupted, it resumes from the last successful batch on the next run. No records are partially imported.

Security Considerations

• Database credentials are encrypted at rest with AES-256-GCM
• Connections use TLS encryption in transit
• Sync queries are read-only — we never modify your source database
• Each data source connection is scoped to a specific organization
• All sync activity is recorded in the audit log

Getting Started

Database directory sync is available on Business and Enterprise plans. To set up your first sync:

1. Navigate to Data Sources in the admin panel
2. Click Add Data Source and select your database type
3. Enter connection details and test the connection
4. Browse your schema and configure field mappings
5. Set a sync schedule and run your first sync

Check our documentation for detailed setup guides for each supported database type, including network configuration for firewalled databases and SSH tunnel support.