Live video infrastructure was built around an assumption that held for fifty years: every destination needs the picture. Route the stream, deliver the frame, done.
AI breaks that assumption. A machine destination is a requester rather than a viewer. It doesn’t want the picture, it wants enough visual information to answer one specific question. A model watching for a scene change needs a fraction of the resolution. A model reading a ticker needs full fidelity in one corner of the frame. A monitoring system scanning forty feeds needs almost nothing from each of them, almost all of the time, and then suddenly needs a great deal from one.
The network still hands all of them the same thing. Everything.
The cost of sending pictures to requesters
The waste is measurable. In a joint Intel and V-Nova study using Intel Deep Learning Streamer and MobileNetV2-SSD on UHD video sequences, roughly 1.4% of decoded pixels were ultimately used for inference. The rest still consumed decode cycles, scaling effort and memory bandwidth. Addressing that mismatch in sparse inference scenarios cut processing time by up to 96% with the model architecture and hardware unchanged.
In most live workflows the practical choice is still between sending the model the full-quality source or generating a lower-resolution derivative ahead of time. The first pays full price for pixels discarded before inference. The second bakes today’s model requirements into the pipeline, forcing another rendition when tomorrow’s model needs something different.
The question the router learned to ask
Traditional routing asks which destination gets the stream. AI routing asks which model gets the stream. Compute-aware routing asks how much of the stream that model actually needs.
abbe, the AI-Based Broadcast Engineer from swXtch.ai, is where that third question gets asked. VC-6 (SMPTE ST 2117-1) is what answers it, through selective data recall built into the encoded representation. Resolution layers and custom spatial regions are directly addressable, so a pipeline fetches and decodes only the bytes needed for a requested Level of Quality or region, rather than the full-resolution frame.
One representation serves multiple models without generating or synchronizing separate proxies, which is what distinguishes this approach from a proxy ladder or an ABR rendition set. Instead of moving complete frames, the fabric moves only the visual information each task requires.
One source. Many consumers. No unnecessary pixels.
Where this actually stands
At IBC, swXtch.ai premieres abbe, with initial capabilities for live media transport and SRT management alongside a live AI audio-processing workflow.
VC-6 integration into abbe is under way now, targeted for the end of October. At that point multiple AI models will draw the resolution or region their task requires from a single VC-6 stream, without a separate decode and downscale stage for each one. We’re recruiting pilot partners at IBC to put it to work.
The first workflow we’re building end to end is many concurrent feeds observed continuously at low resolution, cheap enough to run at scale. When a model detects something that matters, the pipeline pulls the higher-resolution representation or the specific region for a more expensive second look, then triggers the production action, whether that’s routing, clipping, logging or alerting. None of that rests on theory.
The component science behind it is already published, and the codec benchmarks are reproducible. The VC-6 AI Blueprint demonstrates adaptive multi-model inference from a single decoder state, running low-resolution inference and refining selected regions at higher fidelity. NVIDIA’s developer blog documents independent access to selected quality levels and regions, measuring roughly 63% of encoded bytes for the first level down and 27% for the second. Benchmarks are public in V-Nova’s vc6-samples repository. In live multicast testing on swXtch infrastructure at 1080p30, receive-side processing fell around 70% and end-to-end AI latency around 80% against uncompressed ST 2110.
What we measure before we claim
The workflow above gets compared against a conventional full-decode pipeline on equivalent hardware.
The number we care most about is cost per analyzed stream-hour, because that’s the one an operator puts in a budget. Underneath it sit concurrent streams per GPU, end-to-end inference latency, data actually decoded per analyzed stream, and compute utilization. Model accuracy parity and broadcast-grade operational reliability are guardrails rather than trade-offs. A pipeline that analyzes more streams by analyzing them worse has proven nothing.
We’ll publish those results when we have them.
Come prove it with us
At IBC we’re selecting a small group of pilot partners, organizations running live video at scale who want this validated against their own use case and their own numbers.
What a pilot partner gets is concrete. Access to the joint evaluation program when the integration lands at the end of October. Influence on the architecture while it can still bend. A defined test window against an agreed baseline, measured on your content and your infrastructure, with the results shared back whichever way they come out.
If you’re moving many live feeds and expect machines to be watching more of them every year, come find us. V-Nova is in Hall 5 at stands 5.D52 and 5.D54. swXtch.io is in Hall 5 at MR13.
You can also request a meeting in advance.
V-Nova is also contributing to the IBC Accelerator project Delta Protocol: The Future of Live Streaming. The project’s final showcase takes place on Saturday from 11:30 a.m. to 12:30 p.m. on the Future Tech Stage in Hall 14, with its pod in the Accelerator Zone at stand 14.D25.
