// experience · Concentrix + Webhelp
Generative AI Engineer
Owned production LLM inference routing and evaluation for customer-support automation.
- Organization
- Concentrix + Webhelp
- Period
- Feb 2024 – Jul 2024
- Location
- Newark, CA
What I worked on
- Owned the production LLM inference routing layer for 50K daily requests across 3 foundation models on AWS Bedrock and SageMaker; used LiteLLM, provider fallback, and latency-aware routing to cut p95 latency from 4s to 1.5s.
- Built production LLM evaluation and safety gates with CloudWatch logging, hallucination checks, drift monitoring, and a 500-case test set; reduced incidents by 42% and MTTD by 35%.
- Reduced monthly inference spend from $45K to $37K through cost-aware routing, prompt caching, and provider fallback while maintaining 95%+ task success.
Technical context
Python · AWS Bedrock · SageMaker · LiteLLM · CloudWatch · LLM evaluation