Single-process operational boundary
Keep critical runtime behavior inside one JVM process.
Architecture
A single-process operational boundary for governed pipelines.
StreamKernel keeps the runtime path compact and inspectable: source plugin, core orchestrator, policy plugin, transformer chain, sink plugin, DLQ, metrics, provenance, and MLflow model governance.
Runtime narrative
Source Plugin -> Core Orchestrator -> Policy Plugin -> Transformer Chain -> Sink Plugin.
Deny and failure paths flow to DLQ, while Prometheus and OpenTelemetry metrics are emitted across kernel, transform, and sink. MLflow model registry support can promote or roll back artifacts into the transform path, and sink plugins cover Kafka, MongoDB Vector, Postgres, Postgres Vector, Delta Lake, Snowflake, and custom targets.
Config-driven pipelines
Use .properties files to define pipeline behavior.
Plugin-owned extensibility
Let customers or authors bring plugins without giving up kernel control.
Policy before delivery
Enforce policy before transform and sink delivery.
Evidence-first runtime
Emit logs, metrics, settings, and replay metadata.
Commercial AI boundary
Keep protected AI implementation details outside public source.
Plugin model
Customers can bring plugins without giving up kernel control.
Source, policy, transform, sink, metrics, and AI paths are extensible at the edge while the runtime preserves the auditable execution envelope. Postgres and Postgres Vector are first-class sink targets alongside Kafka, MongoDB Vector, Delta Lake, and Snowflake.
Runtime signals
- Effective settings and replay metadata.
- Per-batch policy outcomes and audit headers.
- Transform, sink, DLQ, and kernel metrics exported through Prometheus and OpenTelemetry.
- Model/version labels for AI-enriched records.
Commercial path
Need to map StreamKernel into your platform architecture?
Review the runtime boundary, plugin model, policy path, and commercial packaging options with the team.