ConvoGuard AI: Real-Time Compliance Middleware for Healthcare Agents

Overview

Architected for the CIC Berlin accelerator demonstration, Berlin AI Labs developed a production-grade API middleware that evaluates mental health conversational agent transcripts against EU AI Act, DiGA (Digital Health Applications), and GDPR requirements in real time. The system combines deterministic pattern filtering with model-assisted contextual analysis to return structured risk assessments and compliance scoring within milliseconds.

The Challenge & Architecture

The Problem

Healthcare applications face strict EU regulatory safety mandates.
Manual compliance audits are slow, expensive, and cannot stop live harms.
Critical safety risks (self-harm, manipulation) demand immediate intervention.
BfArM and DiGA certification requires an immutable audit trail.
Runtime validation must occur without noticeable conversational latency.

The Solution

REST API endpoints for real-time validation and audit logging.
6 layered compliance rules pairing deterministic checks with LLM analysis.
Contextual risk scoring evaluating user intent and vulnerability.
Tamper-evident audit trail backed by SHA-256 cryptographic hashing.
Sub-second execution speeds ranging from 170ms to 900ms.

The 6 Compliance Evaluation Rules

Every transcript turn is evaluated against 6 compliance rules. When violations occur, the system computes a deduction score (0 to 100), where scores of 70 or above represent passing status.

Rule	Evaluation Scope	Severity	Penalty
🚨 Crisis & Self-Harm Detection	Flags suicidal ideation, self-harm references, and despair indicators	HIGH	-50 Points
🎭 Coercion & Manipulation Check	Detects emotional exploitation, dependency pressure, and boundary violations	MEDIUM	-30 Points
🆘 Emergency Escalation	Ensures human clinician hotlines and localized emergency contacts are served	HIGH	-25 Points
📋 Explicit Consent (GDPR)	Verifies active user consent before handling personal health information	MEDIUM	-15 Points
📊 Clinical Evidence (DiGA)	Validates structured mood-tracking entries against clinical criteria	LOW	-10 Points
🤖 AI Identity Disclosure	Ensures artificial agent status is made explicit per EU AI Act requirements	LOW	-10 Points

API Integration

Validation Request

curl -X POST https://api.berlinailabs.de/v1/validate \
  -H "Content-Type: application/json" \
  -d '{"transcript": "User: I am feeling completely hopeless about tomorrow..."}'
                

Structured Risk Assessment Response

{
  "compliant": false,
  "score": 50,
  "risks": [
    {
      "category": "SUICIDE_SELF_HARM",
      "severity": "HIGH",
      "message": "Potential self-harm ideation detected. Immediate escalation triggered."
    }
  ],
  "audit_id": "aud_8f92c10b4e",
  "execution_time_ms": 380
}
                

Architectural Features

⚡ Low-Latency Validation

Evaluates conversations within 170ms to 900ms, preserving natural conversational flow for interactive agents.

🤖 Dual Detection Engine

Combines fast deterministic regex heuristics with contextual LLM classification to catch nuanced threats.

🔒 Tamper-Evident Proofs

Each validation event produces a SHA-256 digest, providing verifiable audit trails for regulatory inspection.

📊 Audit Dashboard

Centralized console displaying incident rates, rule triggers, and CSV evidence exports for certification dossiers.

🔑 Controlled Access

Granular API key authentication with strict rate-limiting to protect backend evaluation pipelines.

📈 Production Hardening

Tested across 62 automated integration suites with 100% test coverage and containerized packaging.

Implementation Structure

Clean Architecture Layers

The codebase decouples business domain policies from external LLM providers and database backends.

Domain Layer

Framework-agnostic entities (Conversation, RiskAssessment, AuditProof) and 6 discrete compliance rules.

Infrastructure Layer

External adapters connecting model APIs, persistent storage, and cryptographic signing libraries.

Application Layer

API routing, rate limiting, and regulatory dashboard reporting interfaces.

Technology Stack

TypeScript Node.js Vitest Playwright Docker PostgreSQL

Verification Suite

41 Unit Tests (Vitest)

Covers domain policies, isolated rule logic, scoring algorithms, and factory fixtures.

21 End-to-End Tests (Playwright)

Verifies API payloads, incident reporting, and real-time intervention workflows under synthetic load.

Performance Metrics

<900ms Max Evaluation Latency

62 Automated Tests

6 Regulatory Rules

Zero Unhandled Escapes

Need Compliance Architecture for Autonomous Agents?

We conduct targeted agent attack-surface reviews and design verifiable runtime boundaries for companies operating under EU regulatory frameworks.

Request Security Review Explore All Technical Demos