DataStrat AI Production Documentation
Deploy, configure, and operate DataStrat AI inside your sovereign enterprise VPC. This guide is written for client IT, Security, and DevOps teams running production workloads.
Product Overview & Capabilities
DataStrat AI is a self-contained enterprise AI appliance built to automate and streamline DCAM (Data Management Capability Assessment Model) evaluations. Combining Retrieval-Augmented Generation (RAG) with LangGraph agentic reasoning, the platform conducts structured interviews, evaluates governance artifacts, and recommends maturity scores within your private infrastructure.
Conversational assessment agent that adapts follow-up questions to stakeholder responses and validates evidence against DCAM standards.
Real-time ingestion and semantic indexing of uploaded governance documents with visual completeness tracking.
Built-in Auditor and Compliance Officer separation so AI maturity suggestions are reviewed and finalized by human officers.
Run completely self-contained with bundled containers or connect directly to managed RDS, Redis, Qdrant, and S3 services.
System Architecture
DataStrat AI uses a decoupled microservice topology segmented across three isolated Docker bridge networks
(frontend_net, backend_net, and db_net). All external traffic enters
through a single Nginx reverse proxy using a dual-subdomain routing model so authentication
and application traffic traverse standard HTTPS/HTTP ports (443 / 80).
| Service | Container Name | Role & Responsibilities | Networks |
|---|---|---|---|
nginx |
datastrat_nginx |
Reverse proxy, TLS termination (optional), dual-subdomain routing, 60 MB upload limit, 300s proxy timeout. | frontend_net, backend_net |
ui |
datastrat_ui |
Next.js React dashboard served on internal port 3000 via npm run start. |
frontend_net, backend_net |
api |
datastrat_api |
Stateless FastAPI gateway & LangGraph orchestration on port 8000. |
backend_net, db_net |
worker |
datastrat_worker |
Python Celery worker for asynchronous document ingestion, chunking, and vectorization. | db_net |
zitadel |
datastrat_zitadel |
OIDC / SAML Identity Provider (ghcr.io/zitadel/zitadel:v2.54.3) backed by dedicated zitadel_db. |
backend_net, db_net |
db_init / zitadel_init |
datastrat_*_init |
Ephemeral initialization containers for schema migrations and OIDC client provisioning. | db_net / backend_net |
loki / promtail / grafana |
datastrat_grafana |
Centralized container log aggregation and dashboard exposed at /grafana/. |
backend_net |
Prerequisites & Hardware/VM Requirements
Host & Software Requirements
- Operating System: 64-bit Linux VM (Ubuntu 22.04/24.04 LTS, Debian 12, or RHEL 9+).
- Container Runtime: Docker Engine 24+ and Docker Compose v2 (
docker composeplugin). - Host Utilities:
openssl,unzip,curlorwget, andrsync(recommended for seamless upgrades). - Container Registry Access: GitHub Personal Access Token (PAT) with
read:packagesscope to pull images fromghcr.io.
VM Sizing & Worker Concurrency
Document ingestion is memory-intensive. Configure CELERY_CONCURRENCY and
CELERY_MAX_TASKS_PER_CHILD during setup based on available host RAM:
| Deployment Tier | vCPU | RAM | Disk (SSD) | Recommended Celery Settings |
|---|---|---|---|---|
| Standard Production | 4–8 vCPU | 8–16 GB | 50–100 GB | CELERY_CONCURRENCY=2, CELERY_MAX_TASKS_PER_CHILD=10 |
| High-Throughput Production | 8+ vCPU | 32+ GB | 100+ GB | CELERY_CONCURRENCY=4 (or higher), CELERY_MAX_TASKS_PER_CHILD=10 |
| Constrained Evaluation VM | 2–4 vCPU | ≤ 4 GB | 30 GB | CELERY_CONCURRENCY=1, CELERY_MAX_TASKS_PER_CHILD=1 (prevents OOMs) |
DNS & Firewall Matrix
Before running the deployment wizard, create two DNS A records pointing to your VM or Load Balancer IP:
- Application Domain: e.g.,
app.datastrat.example.com(serves UI, API, and/grafana/). - Authentication Domain: e.g.,
auth.datastrat.example.com(serves Zitadel OIDC endpoints).
Application services (api, ui, postgres, redis, qdrant, minio)
bind to 127.0.0.1 by default via HOST_BIND_IP. Configure your VM/cloud firewall to allow
inbound public traffic only on ports 80 and 443, and block port 8080
from external access (or bind it to 127.0.0.1 in production).
Ensure outbound HTTPS (443) is permitted to ghcr.io, github.com, and your LLM provider endpoint.
Production Deployment Guide
Step 1: Authenticate to the Container Registry
Log in to GitHub Container Registry using a PAT with read:packages permission. Enter the PAT only at the hidden prompt so it is not recorded in shell history:
read -rsp 'GitHub PAT: ' GHCR_PAT; printf '\n'; printf '%s' "$GHCR_PAT" | docker login ghcr.io -u YOUR_GITHUB_USERNAME --password-stdin; unset GHCR_PAT
Step 2: Download & Extract the Release Package
Fetch the official enterprise deployment archive from DataStrat-AI/datastrat_ai_release:
wget https://github.com/DataStrat-AI/datastrat_ai_release/releases/latest/download/datastrat-enterprise-deployment.zip
sudo apt-get update && sudo apt-get install -y unzip
unzip datastrat-enterprise-deployment.zip -d datastrat-deploy
cd datastrat-deploy
Step 3: Run the Interactive Deployment Wizard (./deploy.sh)
The deploy.sh script initializes .env from .env.example, generates
cryptographic secrets, configures Nginx SSL/headers, and locks permissions with chmod 600 .env.
chmod +x deploy.sh update.sh
./deploy.sh
During the wizard, you will select:
- Infrastructure Mode: Built-in local storage containers (
COMPOSE_PROFILES=local-storage) or External Managed VPC services (RDS, ElastiCache, Qdrant Cloud, S3). - Domains: Your
app_domainandauth_domain. -
SSL / HTTPS Mode:
- Option 1 — External Load Balancer (Recommended for Cloud): Cloudflare, AWS ALB, or Azure Application Gateway terminates TLS. Nginx maps
$pref_prototohttpsand setsAUTH_TLS_MODE=external. - Option 2 — Local Machine SSL (Nginx): Nginx terminates TLS on port
443using either a generated self-signed cert in./certs/or your enterprise.crtand.keypaths. - Option 3 — Plain HTTP: Port
80only (not recommended for production).
- Option 1 — External Load Balancer (Recommended for Cloud): Cloudflare, AWS ALB, or Azure Application Gateway terminates TLS. Nginx maps
- LLM Provider & Model: OpenAI, Anthropic, Azure OpenAI, or OpenRouter (plus Ollama via
.env). - Secrets & Sizing: Auto-generates
ZITADEL_MASTERKEY,JWT_SECRET_KEY,ZITADEL_DB_PASSWORD, MinIO credentials, Grafana admin credentials, and Celery worker tuning.
Step 4: Start the Production Stack
When using external managed PostgreSQL, Redis, Qdrant, and S3 (COMPOSE_PROFILES= empty in .env):
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d
When using the bundled Postgres, Redis, Qdrant, and MinIO containers (COMPOSE_PROFILES=local-storage in .env):
docker compose --profile local-storage -f docker-compose.yml -f docker-compose.prod.yml up -d
Step 5: Verify Deployment Health
# Confirm all services are healthy/running (init containers should show Exited (0))
docker compose ps
# Verify Nginx health endpoint
curl -f https://app.datastrat.example.com/health
Environment Configuration Reference
Key variables managed in .env (derived from .env.example):
| Variable | Purpose | Production Example |
|---|---|---|
APP_URL / NEXT_PUBLIC_API_URL |
Public browser URL for the UI and API gateway. | https://app.datastrat.example.com |
ZITADEL_PUBLIC_URL / AUTH_DOMAIN |
Public OIDC issuer URL and domain for Zitadel. | https://auth.datastrat.example.com |
OIDC_CALLBACK_URL |
Redirect URI registered in Zitadel for OIDC login. | https://app.datastrat.example.com/api/v1/auth/oidc/callback |
AUTH_TLS_MODE / AUTH_SECURE |
Controls Zitadel TLS expectation behind reverse proxy/LB. | external / true |
COOKIE_DOMAIN / ALLOWED_ORIGINS |
Root cookie domain and strict CORS origin allowlist. | .example.com / https://app.datastrat.example.com |
JWT_SECRET_KEY / JWT_EXPIRY_HOURS |
HS256 signing secret and session lifetime. | 64-char random secret / 24 |
ZITADEL_MASTERKEY / ZITADEL_DB_PASSWORD |
32-char Zitadel encryption master key and dedicated Postgres password. | Auto-generated by deploy.sh |
DATABASE_URL / REDIS_URL / QDRANT_URL |
Connection URIs when using external managed services. | postgresql://user:pass@rds:5432/datastrat |
STORAGE_ENDPOINT_URL / STORAGE_BUCKET_NAME |
S3-compatible endpoint and evidence bucket name. | https://s3.amazonaws.com / datastrat-evidence |
LLM_PROVIDER / LLM_MODEL / LLM_API_KEY |
Active AI provider, model identifier, and API credential. | openai / gpt-4 |
GRAFANA_ADMIN_USER / GRAFANA_ADMIN_PASSWORD |
Credentials for the bundled Grafana log dashboard at /grafana/. |
admin / auto-generated password |
CELERY_CONCURRENCY / CELERY_MAX_TASKS_PER_CHILD |
Parallel document ingestion workers and memory recycling threshold. | 2 (8–16 GB) or 4+ (32+ GB) / 10 |
Identity & Authentication (Zitadel OIDC / SAML / SSO)
DataStrat AI ships with Zitadel v2.54.3 as its embedded OpenID Connect (OIDC) provider,
isolated on its own PostgreSQL instance (zitadel_db). On first boot, zitadel_init
automatically provisions the DataStrat OIDC application and writes client credentials to the shared
zitadel_config volume (/zitadel-config/credentials.json).
OIDC Redirect & Reverse Proxy Headers
- Callback URI:
https://app.<domain>/api/v1/auth/oidc/callback - Post-Logout URI:
https://app.<domain>/login - Nginx Buffer Sizing: Configured with
proxy_buffer_size 128kandlarge_client_header_buffers 4 64kto accommodate large OIDC/JWT headers.
Zitadel initializes a bootstrap human administrator (admin@datastrat.local) using the
ZITADEL_ADMIN_PASSWORD configured in .env. Before starting the stack in production,
set a strong, unique ZITADEL_ADMIN_PASSWORD in .env, and sign in at
https://auth.<domain> immediately after deployment to verify the account and configure MFA.
Enterprise SSO & SAML / OIDC Federation
To federate login with your corporate Identity Provider (Microsoft Entra ID / Azure AD, Okta, Ping Identity, or Google Workspace):
- Log in to the Zitadel Management Console at
https://auth.<domain>. - Navigate to Organization → Identity Providers and add your corporate SAML 2.0 or OIDC provider.
- Enable auto-creation or JIT provisioning as required by your IAM policy, then assign users to DataStrat Auditor or Compliance Officer roles.
LLM & Storage Configuration
Supported LLM Providers
Both api and worker containers read the same LLM configuration from .env:
- OpenAI:
LLM_PROVIDER=openai,LLM_MODEL=gpt-4(orgpt-4-turbo/gpt-4o). - Anthropic:
LLM_PROVIDER=anthropic,LLM_MODEL=claude-3-5-sonnet-20240620. - Azure OpenAI:
LLM_PROVIDER=azure, plusAZURE_OPENAI_ENDPOINT=https://<resource>.openai.azure.com/. - OpenRouter:
LLM_PROVIDER=openrouter,LLM_BASE_URL=https://openrouter.ai/api/v1. - Ollama (Self-Hosted / Air-Gapped):
LLM_PROVIDER=ollama,LLM_BASE_URL=http://<ollama-host>:11434.
Storage Modes: Built-in vs. Managed Cloud
When running with local-storage, persistent state is stored in Docker named volumes
(postgres_data, qdrant_data, minio_data, zitadel_db_data,
grafana_data). The minio_init container automatically creates the
datastrat-evidence bucket with a private access policy.
For enterprise cloud deployments, configure external managed services in deploy.sh (AWS RDS,
ElastiCache, Qdrant Cloud, AWS S3 / Azure Blob S3 Gateway) and omit --profile local-storage.
Nginx allows evidence uploads up to 60M (client_max_body_size 60M).
Upgrading & Maintenance
Automated Upgrade Pipeline (./update.sh)
Run ./update.sh from your deployment directory to perform a 5-stage zero-config-loss upgrade:
./update.sh
- Fetch Release: Downloads the latest
datastrat-enterprise-deployment.zipfrom GitHub Releases. - Sync Configs: Extracts updated compose and Nginx files while preserving
.envand./certs/(re-executing itself automatically ifupdate.shwas updated). - Audit
.env: Compares.envagainst the new.env.exampleand prompts for (or auto-generates) any newly added environment keys. - Prune & Pull: Runs
docker image prune -a -fand pulls updated container images. - Restart Stack: Recreates containers cleanly with
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d.
Backups & Observability
- Critical Host Files: Back up
.envand./certs/to your enterprise secrets vault. LosingZITADEL_MASTERKEYorJWT_SECRET_KEYwill invalidate encrypted Zitadel state and active sessions. - Database & Object Backups: Snapshot managed RDS/S3 volumes, or back up Docker volumes
postgres_data,zitadel_db_data,qdrant_data, andminio_dataon a scheduled basis. - Centralized Logs: Access Grafana at
https://app.<domain>/grafana/(sign-up disabled in production). Docker json-file logs are capped at10m×3files per container viadocker-compose.prod.yml.
Security & Support Contact
- Data Handling: All third-party tracing tools (such as LangSmith) are strictly disabled in production builds. Prompts, evidence, and scores remain inside your VPC when configured with a self-hosted provider such as Ollama; external cloud LLM providers receive inference requests under their respective enterprise data-handling policies.
- Sovereign Initialization: Schema migrations and seeding run in ephemeral
db_initandzitadel_initcontainers that exit upon completion. - Credential Hygiene:
deploy.shenforceschmod 600 .envand restricts internal database/cache ports to127.0.0.1.
For production assistance, deployment reviews, or licensing inquiries, contact the DataStrat AI Engineering team at
support@datastrat.ai.
When opening a ticket, please include docker compose ps output, relevant container logs from Grafana/Loki, and a redacted copy of your .env keys (never send API keys or secrets).