Athena — Enterprise Agentic AI Platform
Provider-agnostic LLM gateway, MCP tool servers, layered agent memory, guardrails and an LLM-as-judge evaluation harness.
- Context
- Avira Digital Technologies — internal product
- Period
- Sep 2025 — Present
- My role
- ML Associate — contributed across R&D, design and implementation of the orchestration, tooling and evaluation layers as part of the engineering team.
The problem
Enterprise teams needed agents that could reach real tools and real data without hard-wiring the whole platform to one model vendor, one tool protocol, or one set of safety assumptions.
Why it needed solving
Committing an enterprise platform to a single model provider is both a cost and a continuity risk, and every new tool integration written ad hoc becomes its own maintenance burden. A shared gateway, a standard tool protocol and a measurable evaluation loop were needed before the platform could be trusted with regulated data.
The approach
A layered platform: a universal LLM gateway underneath, MCP servers standardizing tool access above it, a multi-layer memory system for continuity, guardrails on both input and output, and an LLM-as-judge harness measuring whether any of it actually works.
Architecture
- LLM gateway — a single routing layer over OpenAI, IBM watsonx and Meta Llama, reached through Azure and direct provider APIs, so model choice becomes configuration rather than code.
- MCP tool layer — MCP servers exposed over both SSE and stdio transports, giving every agent one consistent way to discover and call tools.
- Tool & data nodes — integrations spanning social platforms, medical sources (PubMed, DrugBank, ClinicalTrials.gov, FDA) and relational databases (PostgreSQL, MySQL).
- Memory layer — short-term, long-term and episodic memory, keeping context coherent across turns and across sessions.
- Guardrail layer — input and output screening for PII/PHI, jailbreaks, harm, social bias, profanity and prompt injection, run alongside IBM's Guardrails service.
- Evaluation layer — a Judge Agent that scores responses against a labeled query/response dataset.
Implementation
- Built the gateway on LangChain and CrewAI so orchestration primitives stayed consistent regardless of which provider served a request.
- Implemented MCP servers with SSE transport for networked tools and stdio transport for local ones, so the same tool contract works in both deployment shapes.
- Wired medical and social data sources in as tool nodes rather than bespoke pipelines, keeping ingestion uniform.
- Implemented the three memory tiers with distinct retention and retrieval behavior instead of a single undifferentiated history buffer.
- Layered the platform's own guardrails with IBM Guardrails so that a miss in one layer is still caught by the other.
- Delivered a social-sentiment analytics workflow that orchestrates agents to pull and analyze posts and comments across multiple platforms.
AI/ML components
Multi-provider LLM routing and model selection
Multi-agent orchestration (LangChain + CrewAI)
Short-term / long-term / episodic agent memory
LLM-as-judge response scoring against labeled data
Safety guardrails — PII/PHI, jailbreak, harm, bias, profanity, prompt injection
Agent-driven sentiment analysis across social sources
Challenges & how they were handled
Each model provider exposes a different API surface, so provider choice tended to leak into application code.
A universal gateway normalized the differences, leaving model selection as a configuration decision instead of a rewrite.
Every new tool integration risked becoming its own bespoke protocol for agents to learn.
MCP servers with SSE and stdio transports standardized tool access, so adding a tool did not mean teaching agents a new pattern.
Agents handling medical and enterprise data cannot be allowed to leak PII/PHI or be talked out of their instructions.
Input and output guardrails covering PII/PHI, jailbreaks, harm, bias, profanity and prompt injection, run in combination with IBM Guardrails.
Without measurement, “the agent got better” is an opinion.
A Judge Agent harness scoring responses against a labeled query/response dataset turned quality into something observable between iterations.
What comes next
- Broaden the labeled evaluation set so judge scores generalize across more query types.
- Extend routing decisions to account for cost and latency, not just capability.
Related work
Clinical Knowledge Graph RAG
Avira Digital Technologies — BMS client
A clinical Q&A retrieval system that moved from hybrid-search RAG to Graph RAG once relationship modeling became the real bottleneck.
Agentic Data Structuring & Query Platform
Avira Digital Technologies — BMS client
An orchestrator/sub-agent pipeline that turns raw multi-source data into tagged, queryable records — and answers questions about it in plain English.