An internal AI platform supporting business operations (built a multi-service foundation and auth hub)
A self-built OIDC auth hub (BFF) binding 5 AI services with Google Workspace SSO, speech synthesis, a typo-detection pipeline cross-checking OCR × speech recognition, and generative-AI content review — built on GCP with IaC
At a glance
- Problem
Multiple tools supporting siloed operational work (reading aloud, proofreading, review) with AI had to be delivered as an "always-on internal platform" — usable across the board with a single login, meeting an organization's internal-control and security standards, and running heavy, expensive AI processing stably at production quality.
- What was built
A self-built OIDC auth hub (BFF) starting from Google Workspace SSO binds 5 AI services, issuing audience-scoped short-lived JWTs per tool.
- Role
- Team
Technical architect and full-stack developer.
- Result
5 AI services unified under a single SSO | a self-built OIDC auth hub, typo detection, and high-standard multi-layered defense built with IaC
- 5 servicesAI services integrated
- Assurance
- 71 modulesTerraform modules
- 256 bit (AES-GCM)PII encryption
Client
An internal AI platform for a corporate customer | Domain: business-operations support (AI speech synthesis, text typo detection, generative-AI review, malware scanning of uploaded assets) | Form: internal multi-tenant SaaS (offered to all staff via Google Workspace SSO) | Architecture: a microservices monorepo on GCP (a shared BFF auth hub + multiple AI-tool services)
My role
Technical architect and full-stack developer. Implemented across frontend, backend, GCP infrastructure (Terraform), CI/CD, and observability — from the shared foundation (BFF auth hub, OIDC provider, back-channel logout, PII encryption) to the speech-synthesis service (ElevenLabs / Google Chirp3), the ML/OCR typo-detection pipeline (FastAPI + Python + Cloud Workflows), generative-AI content review (Gemini 2.5 / Vertex AI), and the malware scanner (ClamAV on Cloud Run). Unified each service's caching, auth, logging, and Terraform modules under a single convention, building it as a horizontally scalable platform.
Challenge (Situation & Task)
Day-to-day operations include many highly specialized, easily siloed tasks — reading aloud, proofreading, and review. Several AI tools to support these had to be delivered while simultaneously satisfying competing requirements: (1) staff use them across the board with a single login, (2) they meet an organization's internal-control and information-security standards, (3) each tool can be deployed and improved independently, and (4) heavy, expensive AI processing (GPU/LLM/speech synthesis) runs stably at production quality. Not a one-off PoC, but an "always-on internal platform" embedded in the company-wide workflow.
An internal AI platform concentrated organization-specific difficulties.
-
Integrating multiple AI tools with SSO: building tools of different natures (speech synthesis, typo detection, generative-AI review, summarization) separately fragments auth, permissions, and operations. Starting from Google Workspace SSO, an "auth hub" was needed to single-sign-on into each tool securely while separating permissions and deployments per tool.
-
High-standard security and internal control: even for an internal system, authorization bypass, PII leakage, and impersonation are unacceptable. Token lifetimes, audit logs, PII encryption, WAF, MFA, least-privilege IAM, and keyless CI/CD — a multi-layered defense that withstands internal control — had to be built in as preconditions.
-
Production operation of heavy, expensive AI processing: text typo detection is a long-running job combining OCR (images) and speech recognition (matching text to speech); LLM OCR is accurate but costly and slow. Speech synthesis demands high audio quality (fades, pronunciation dictionaries, and so on). These had to run cost-controlled, able to complete and resume.
-
Safe asset intake (a zero-trust entry point): externally supplied asset files had to be malware-scanned before reaching the platform and routed clean/quarantined. Scanning had to avoid exhausting memory even for multi-GiB assets and be idempotent under retries.
Why these technologies (Rationale)
Unify the BFF and each tool's UI on Next.js 16 / React 19 (App Router · RSC): the auth hub (BFF) keeps sensitive logic server-side and runs IP-allowlist + session validation in Edge Runtime middleware. Each tool follows the same Next.js convention to minimize learning cost and review burden.
Auth is a "self-built OIDC provider + per-tool short-lived JWT": starting from Google Workspace SSO (NextAuth.js v5 / Identity Platform), the BFF issues short-lived tokens scoped to each tool's audience (10-minute access token, 1-hour ID token). PKCE S256 is required, structurally eliminating cross-tool impersonation and auth-code interception.
Split heavy AI work into jobs + workflows: typo detection runs on FastAPI (async) + Cloud Run Jobs + Cloud Workflows. OCR and speech recognition run in parallel, and long jobs are decoupled from HTTP to guarantee progress, resumption, and idempotency. Speech synthesis places two providers (ElevenLabs / Google Chirp3) behind a provider abstraction, with a dummy mode for testing without API cost.
Generative-AI compliance review on Gemini 2.5 + grounding: per-criterion prompts (internal standards, guidelines, etc.) assess risks in documents, attaching grounded citations so the rationale is traceable. The API-key approach and the Vertex AI (GCP service-account) approach are abstracted and switched by control and cost.
Consolidate data on GCP managed services: Cloud SQL (PostgreSQL / MySQL) runs with IAM auth, required TLS (ENCRYPTED_ONLY), and private IP; cache is Memorystore (Redis); real-time progress is Firestore; assets are Cloud Storage. No dedicated VMs — leaning on managed services to reduce operational load and cost.
Infrastructure is 100% Terraform (IaC): VPC through Cloud Run, Cloud SQL, Cloud Armor, Secret Manager, Identity Platform, and Workflows codified in ~71 modules. Separates stg/prod state, and separates responsibilities — Cloud Build owns "container image and latest env," Terraform owns "infrastructure config" — to prevent drift.
CI/CD is keyless via Workload Identity Federation (OIDC): GitHub Actions authenticates to GCP without issuing service-account keys. Cloud Build separates stg/prod, DB migrations are split into a dedicated job, and CodeQL and dependency updates are automated.
What I did (Action)
[Self-built OIDC auth hub (BFF)] Starting from Google Workspace SSO, the BFF issues audience-scoped short-lived JWTs to each tool (10-minute access token, 1-hour ID token). The authorization flow requires PKCE S256, and token hand-off uses an auto-POSTed HTML form (not the URL) + an HMAC signature over the originating origin to prevent MITM. Per-device session management enables per-device revocation and detection of permission changes.
[Back-channel logout and permission sync] Built a mechanism to asynchronously deliver logout and permission changes to each tool. Events are HMAC-signed, retried with exponential backoff, and deduplicated/replay-protected by a unique constraint. Tools stay loosely coupled while company-wide logout propagates reliably.
[PII encryption and audit logs] External users' personal data is stored encrypted with AES-256-GCM, and search uses HMAC tokens for partial matching without decryption. Secret comparison is constant-time to prevent timing attacks. Session creation, revocation, and renewal are recorded in audit logs to provide internal-control evidence.
[Typo-detection pipeline] Text is extracted from assets, local OCR detects text "transitions," and LLM OCR is applied only to the set of unique text (not every frame). Speech recognition also transcribes the audio, which is cross-checked against the OCR results to detect typos, NG words, and proper-noun mismatches.
[Idempotent, resumable long jobs] Assets are split into segments and each piece of text gets a deterministic ID, so IDs stay stable through downstream merge under parallelism and are idempotent on re-run. Progress is delivered to the UI in near-real-time without ever going backward. Uploads use chunked parallel signed URLs for large assets.
[High-quality speech synthesis] Two TTS providers are unified behind a provider abstraction, with output level and fades normalized. It supports pronunciation dictionaries and text preprocessing, and scripts use optimistic locking to detect concurrent-edit conflicts. A dummy mode tests the synthesis path without hitting real APIs, and usage logs are fully recorded to make cost observable.
[Asset malware scanning (zero-trust entry)] Uploaded assets are passed to a malware scanner (Cloud Run) via events, streaming-scanned even at large sizes (without buffering, to avoid memory exhaustion) and routed to clean/quarantine buckets. Intermediate and deleted files are safely ignored, the atomicity of the move step makes it idempotent under retries, and scan results are emitted to Cloud Monitoring via OpenTelemetry.
[IaC, security, CI/CD] The whole of GCP is codified in ~71 Terraform modules. Multi-layered defense with Cloud Armor (OWASP CRS 3.3 + adaptive DDoS protection + rate limiting), Cloud SQL (IAM auth, required TLS, private IP), Identity Platform (SMS MFA, reCAPTCHA Enterprise), Secret Manager (latest-version only), and least-privilege service accounts. CI/CD authenticates keylessly via Workload Identity Federation (OIDC), with the WAF fully enabled in stg to eliminate false positives before production.
The platform's design philosophy was to "bind multiple AI tools under a single auth/operations convention, at a quality that withstands an organization's controls."
Loosely coupled integration via an auth hub: Giving each tool its own login makes auth quality vary per tool and breaks propagation of permission changes and logout. So the BFF is the sole identity provider, and tools only receive audience-scoped short-lived JWTs. Token hand-off uses an auto-POST form + origin HMAC signature instead of the URL, eliminating token leakage through redirects. Logout and permission changes are delivered over the back channel — signed, retried, and deduplicated — so tools stay loosely coupled yet follow company-wide session control.
Making heavy AI work "complete, resumable, idempotent": Typo detection combines heavy OCR (images) and speech recognition (speech). Applying LLM OCR to every frame would blow up cost, so local OCR detects text transitions and LLM is applied only to unique text. Long jobs are segmented, and deterministic IDs make the final result converge uniquely under parallelism, partial re-runs, or retries.
Building control as a first-class feature: PII is encrypted at rest and searched via HMAC tokens without decryption. Secret comparison is constant-time, the auth flow requires PKCE, tokens are short-lived, and operations leave audit logs — all built into the initial design, not bolted on. On the infrastructure side, Cloud SQL uses IAM auth, required TLS, and private IP; the entry point is Cloud Armor; CI/CD is keyless (OIDC) — multi-layered — with the WAF fully enabled in stg to surface false positives and misconfigurations before production.
Optimizing cost and operations by consolidating on managed services: With no dedicated VMs or Kubernetes, it leans on managed services — Cloud Run (services + jobs), Cloud SQL, Memorystore, Firestore, Cloud Workflows. The primary region keeps one instance warm while the secondary scales to zero for DR — an asymmetric setup — and the Artifact Registry image-retention cap keeps steady-state cost low while preserving recovery during incidents.
Responsibilities
- Platform design and BFF auth hub (OIDC / back-channel logout / PII encryption) implementation
- Speech-synthesis service (ElevenLabs / Google Chirp3) frontend & backend
- Typo-detection pipeline (FastAPI / Python / OCR / speech recognition / Cloud Workflows)
- Generative-AI compliance review (Gemini 2.5 / Vertex AI / grounding)
- Malware scanner (ClamAV on Cloud Run / Eventarc)
- GCP infrastructure build & operations (Terraform / Cloud Run / Cloud SQL / Cloud Armor / Identity Platform)
- CI/CD (Cloud Build / GitHub Actions / Workload Identity Federation) and observability
Technologies
Results in numbers
- AI services integrated
- 5 servicesA shared BFF auth hub + speech synthesis, typo detection, generative-AI review, and malware scanning unified under a single SSO.
- Access-token lifetime
- 10 minShort-lived JWT scoped to each tool's audience. ID token is 1 hour; PKCE S256 required.
- Terraform modules
- 71 modulesVPC, Cloud Run, Cloud SQL, Cloud Armor, Identity Platform — GCP 100% codified (IaC).
- PII encryption
- 256 bit (AES-GCM)Email, phone, and name stored encrypted; search via HMAC tokens without decryption.
- Streaming-scan limit
- 10 GiBScanned by ClamAV without buffering to avoid memory exhaustion. Idempotent under retries.
Results
- Integrated 5 AI services of different natures into an internal platform usable across the board with a single Google Workspace SSO login — a loosely coupled setup where each tool deploys and improves independently
- A self-built OIDC auth hub (BFF) issues audience-scoped short-lived JWTs per tool (10-minute access token). PKCE S256 is required, and an auto-POST + origin HMAC signature (never putting tokens in URLs) structurally eliminates impersonation and interception
- Logout and permission changes propagate reliably to all tools via HMAC-signed, exponential-backoff, deduplicated back-channel events (company-wide session control while staying loosely coupled)
- PII is stored encrypted with AES-256-GCM and searched via HMAC tokens without decryption. With constant-time comparison and audit logs, it meets an organization's internal-control requirements
- Automatically detects typos in text by cross-checking OCR (images) and speech recognition (speech)
- Hybrid OCR (local text-transition detection → LLM OCR only on the diff) minimizes LLM calls and cost while preserving accuracy
- Deterministic IDs make long jobs idempotent and resumable — the result converges uniquely under parallel processing, partial re-runs, or retries
- High-quality AI speech (two TTS systems, pronunciation dictionaries) built maintainably with a provider abstraction and a dummy mode
- A zero-trust entry point that streaming-scans uploaded assets (up to 10GiB, avoiding memory exhaustion) and routes them clean/quarantine atomically and idempotently
- GCP-wide IaC in ~71 Terraform modules. Multi-layered defense with Cloud SQL (IAM auth, required TLS, private IP), Cloud Armor (OWASP CRS 3.3), Identity Platform (MFA), and least-privilege IAM, with CI/CD made keyless via Workload Identity Federation (OIDC)
Facing a similar challenge?
Your business challenge can likely be solved with today's technology too. Start with a free 30-minute technical consultation and tell me where things stand.
Fixed-scope projects and technical advisory are both available.