AI Engine Architecture

The Python AI Engine is the intelligence core of QBITEL Bridge, handling protocol discovery, threat analysis, and autonomous security operations.

Overview

The AI Engine is a modular Python application built with FastAPI and PyTorch. It provides REST and gRPC APIs, an autonomous multi-agent system, and a complete ML pipeline for protocol discovery and security analysis.

Module Structure

ai_engine/
  core/              # Engine core, config, orchestrator
  agents/            # Multi-agent orchestration system
  discovery/         # Protocol discovery (PCFG, transformers)
  detection/         # Field detection (BiLSTM-CRF)
  anomaly/           # Anomaly detection (Isolation Forest, LSTM, VAE)
  security/          # Zero-touch decision engine
  compliance/        # Compliance automation (9 frameworks)
  copilot/           # Protocol intelligence copilot
  crypto/            # Post-quantum cryptography
  llm/               # LLM service (Ollama, RAG, providers)
  marketplace/       # Protocol marketplace (Stripe, S3)
  legacy/            # Legacy System Whisperer (COBOL, JCL)
  cloud_native/      # Service mesh, container security
  api/               # REST (FastAPI) and gRPC APIs
  models/            # ML model management and registry
  monitoring/        # Metrics, logging, tracing
  tests/             # Test suite (unit, integration, perf)

Protocol Discovery Pipeline

The discovery pipeline transforms raw network traffic into structured protocol definitions through six stages:

  1. Statistical Analysis -- byte distributions, entropy calculations, message length profiling
  2. Pattern Extraction -- delimiter detection, header identification, field boundary analysis
  3. PCFG Inference -- learn a Probabilistic Context-Free Grammar from traffic patterns
  4. Parser Generation -- auto-generate parsers from the learned grammar
  5. ML Classification -- classify using ensemble models (CNN, LSTM, Random Forest)
  6. Validation -- verify discovered protocols with structural and semantic checks

Key Discovery Modules

Module Responsibility
statistical_analyzer Entropy, byte frequency, n-gram analysis
pattern_extractor Recurring patterns, delimiters, headers
enhanced_pcfg_inference Grammar learning with production probabilities
enhanced_parser_generator Dynamic parser code generation
protocol_classifier Ensemble ML classification

Multi-Agent System

The AI Engine implements a multi-agent orchestration framework where specialized agents collaborate autonomously:

  • Agent Pool -- manages agent lifecycle, resource allocation, and health monitoring
  • Agent Communication -- message-passing infrastructure between agents
  • Agent Memory -- persistent and working memory for context retention
  • Agent Collaboration -- task decomposition and result aggregation
  • Planning Agent -- breaks complex tasks into executable sub-goals

Zero-Touch Decision Engine

The Zero-Touch Decision Agent provides autonomous security response with configurable confidence thresholds:

Confidence Risk Level Action
≥ 0.95 Low Autonomous execution
≥ 0.85 Medium Auto-approved with logging
≥ 0.50 Any Escalated to operator
< 0.50 Any Escalated to senior analyst

LLM Integration

The AI Engine integrates with on-premise LLM providers for air-gapped deployments:

  • Ollama -- primary provider for local LLM inference
  • vLLM -- high-throughput serving for production workloads
  • RAG Engine -- retrieval-augmented generation with Qdrant vector store
  • Semantic Cache -- caches LLM responses for repeated queries
  • Guardrails -- input/output validation for LLM safety

Anomaly Detection

An ensemble of anomaly detectors provides multi-layered threat detection:

  • Isolation Forest -- unsupervised anomaly scoring
  • LSTM Detector -- temporal sequence anomaly detection
  • VAE Detector -- variational autoencoder for reconstruction-based detection
  • Ensemble Detector -- weighted combination of all detector outputs

Next Steps