Projects
Systems I've Built & Architected
Production systems powering Gainsight's enterprise platform, from data pipelines to AI-powered search.
Across Two Product Lines
My Contributions to Gainsight's CustomerOS
AI · SEARCH · LLM
Community Search & AI Platform
CC (Customer Communities) · Three AI features, federated search, and a provider-agnostic AI orchestration layer
The AI brain of Gainsight Communities. A provider-agnostic AI orchestration layer powers three flagship features through a factory pattern: primary plus fallback LLM chains with automatic cross-provider retry, bidirectional payload transformation for provider compatibility, and graceful degradation on any failure. Synchronized Search Service indexes 25M+ records from 5 federated sources to Algolia with zero-downtime index swaps.
PHP (Yii/Symfony)Algolia AskAIOpenAIGeminiLlama 3 70BAWS BedrockSSE StreamingPythonRedisS3 ParquetAthena
- Cross-provider LLM fallback: dedicated adapter per provider handles payload asymmetry, with automatic retry across providers even on client errors to prevent single-provider outages
- SSE streaming with thinking-phase deduplication: stateful stream processor filters duplicate reasoning blocks per model family while preserving final output fidelity
- Domain-driven prompt architecture: composable prompt traits (model, version, temperature, reasoning effort) enable per-feature model versioning and A/B testing without code duplication
- Dynamic rate limiting with zero-restart retuning: atomic Redis counters (no locks, no races), thresholds configurable at runtime so ops can retune without a deploy
- Zero-downtime full sync: bulk-index 25M+ records into temp Algolia index, atomic rename over production. No search outage during reindex
- Analytics data lake on S3 Parquet + Athena with tag expression consolidation reducing Algolia API calls by 90%+
MCP PROTOCOL · PYTHON · AI INFRASTRUCTURE
Customer Community MCP Server
CC · Production Model Context Protocol server making community knowledge accessible to AI assistants
Production MCP server powering AI assistant integrations for enterprise community platforms. Handles the full OAuth2.1 authorization lifecycle with PKCE and dynamic client registration. Routes tool invocations to multiple backend services covering search, analytics, administration, and education. Enforces tenant-scoped security at every layer with permission-scoped tokens and per-tenant rate limiting. Deployed across multiple regions with edge-based request routing.
PythonFastMCPStreamable HTTPOAuth2.1 PKCEJWTRedisDynamoDBAlgoliaECS FargateCloudFrontLambda@Edge
- Full OAuth2.1 authorization flow with PKCE and dynamic client registration, listed on major AI assistant connector registries
- Multi-region deployment with edge-based request routing for geographic locality and failover
- Permission-scoped search tokens with access control baked at the token layer, not post-filtered
- Per-tenant rate limiting, connection lifecycle management, and graceful degradation on backend failures
- Dozens of tools spanning search, analytics, content moderation, user management, and education backends behind a single MCP interface
DISTRIBUTED SYSTEMS · DAG ENGINE
Salesforce
HubSpot
Snowflake
BigQuery
Databricks
SAP
Zendesk
Zoom
MS Teams
Intercom
MySQL
PostgreSQL
MSSQL
GA4
Data Highway: DAG-Based Pipeline Engine
CS (Customer Success) · The compute backbone processing 61B+ records/day
A distributed data pipeline platform for enterprise SaaS. Ingests data from tenant systems via 35+ connectors, runs transformations through a SQL engine, and loads results into analytics data stores and warehouses. A directed-acyclic graph of tasks coordinated by leader election and workflow orchestration across a horizontally-scalable worker cluster.
Java 8Vert.xApache DrillZookeeperPython/CeleryPostgreSQLRedisAWS SQSS3
- Processes 61B+ records/day across 600+ enterprise tenants at 98.7% job success rate
- Multi-worker architecture: DAG Workers (Java/Vert.x) + Dynamic Workers (35 isolated connector JARs) + Python/Celery ML Workers + Apache Drill SQL engine
- 60+ custom Apache Drill UDFs for tenant-specific data transformations
- Horizontally-scalable cluster coordinated by Zookeeper leader election and distributed workflow orchestration
- Failure-injection tasks (OOM, disk exhaustion, FD leak) as production resilience probes
35+ Enterprise Connectors
SalesforceHubSpotSnowflakeBigQueryDatabricksSAPZendeskZoomMS TeamsIntercomMySQLPostgreSQLMSSQLGA4+ 21 more
SECURITY · GRAPH DB
User Permissions (RBAC)
CS · Graph-native access control powering authorization across 7+ services
Centralized multi-tenant RBAC service. Permissions stored as a graph: a single traversal answers "can user U do action A on resource R?" in milliseconds, with Redis-cached evaluations and expression-based policy conditions. The permission SDK is consumed by 7+ services across the platform.
Neo4jVert.xRedisMessage QueuePostgreSQL
- Graph-native model: resources, policies, groups, and manager chains as traversable edges, so a single graph traversal resolves access in milliseconds
- Targeted Redis invalidation: editing a sharing group invalidates only affected members × resources, not the full cache
- Row-level data enforcement: restricted data never leaves the database, access control enforced at SQL query-build time
- Permission SDK consumed by 7+ platform services: analytics, query federation, reporting, API gateway, data store, catalog, and cross-product bridge
- Heartbeat-based leader election for worker coordination with automatic cleanup of orphaned graph nodes
ANALYTICS · WAREHOUSING
Usage Data Model (UDM)
CS · Multi-tenant analytics orchestration across dual warehouses
Full-lifecycle analytics engine: onboarding, schema definition, ingestion from 10+ sources, transformation, and warehouse storage. Powers Gainsight dashboards with per-tenant dynamic DDL and dual-warehouse abstraction (PostgreSQL for small tenants, Redshift for high-volume).
Vert.xRedshiftPostgreSQLMessage QueueRedis
- ~25 pluggable task builders: adding a new data source is a single builder class
- Dual-warehouse abstraction: PostgreSQL for small tenants, Redshift for high-volume OLAP, switched transparently
- Permissions injected at query-build time: restricted data never leaves the database
- Cross-system DAG submission to the pipeline engine with retry and backoff
- Secure multi-tenant data exchange with per-tenant credential isolation
PUB/SUB · EVENT-DRIVEN
Event Processing
CS · Four-tier async pub/sub with durable delivery and cross-region failover
Async event fan-out with durable delivery. Publishers emit once; subscribers register for events with optional delayed delivery and data enrichment. Four-tier architecture with purpose-matched runtimes: Receiver → Router → Relayer → Broadcaster.
Vert.xAWS SQSApache HttpAsyncPostgreSQLRedis
- Cross-region SQS failover: automatic switch to secondary queues on publish errors
- Self-healing delivery: dedicated recovery process detects and recovers orphaned delivery rows after worker crashes
- Blocked-tenant handling isolates noisy tenants from starving other tenants' event delivery
- Enrichment loop: events decorated with business context from the analytics data store before delivery
- Tiered architecture: Receiver → Router → Relayer → Broadcaster with purpose-matched runtimes
COMPLIANCE · OBSERVABILITY
Audit Service
Centralized event capture & compliance trail
Reactive audit service for compliance trails. Fire-and-forget writes via Vert.x event bus decouple caller latency from Postgres persistence. Per-tenant tables created lazily on first write. Used by every service in the platform via a lightweight audit client SDK.
Vert.xPostgreSQLRedis
- Async write via event bus: HTTP 200 returns before DB insert completes
- Random-tenant retention cleanup with +10 day safety buffer
- Separate read/write DB pools to prevent analytics from starving ingestion
INFRASTRUCTURE · CONFIG
Config Server
Centralized configuration for all CS services
Spring Boot + Vert.x hybrid config service. Every platform service pulls runtime configuration from here (DB strings, feature flags, encrypted secrets) with full audit trail and periodic refresh. The config client SDK is embedded across all Java services.
Spring BootVert.xPostgreSQLAES Encryption
- Upsert + append-only version history table in single transaction
- Database-enforced integrity with case-insensitive keys and FK cascades
- Redis-cached auth lookups with fail-fast startup ping
