Time budgets

Low latency says nothing. Real-time sets a deadline.

“Low latency” never says how low. Real-time makes you say it: to call something real-time, you have to define what real means, how late data can arrive and still be used. That limit is your time budget, the number to design, measure and test against.

See the decoding window See the examples

01Low compared to what?“Low latency” has no number in it. There is nothing to design for and nothing to test against.

02Real-time has a deadlineData is real-time when it arrives while it is still useful. How long that is, is your time budget.

03Late is the same as lostData that misses its deadline is thrown away, however close it came.

01Define real-time

“Real” is a number you choose

Every system has a moment after which data is no use: a frame that missed the screen, a command that arrived after the machine moved. That moment is what real-time means for you. Name it, and the wish becomes a limit you can check.

What we hear

“We need low latency.”

Low compared to what? No start, no end, no limit. Every network looks fine on a good day.

Real-time, defined

“Camera to screen in 150 ms. The network gets 90 ms of it.”

A start, an end and a limit. Now it can be designed, measured and tested, and everyone knows their share.

Where it starts

The moment something happens: a camera takes a frame, an operator moves a stick, a sensor takes a reading.

Where it ends

The moment the data is used: shown on a screen, played in an ear, acted on by a machine.

The limit

The longest the trip may take. Data that arrives after it is no longer any use.

02The decoding window

Late is the same as lost

A live video player shows a new frame every 33 ms. To keep that pace, it gives each frame a fixed time to arrive: the decoding window. A frame that misses its window is skipped, even if it shows up a moment later. Move the window and switch between two networks with the same average.

Network

Decoding window100 ms

Swipe to see all frames

Shown on time Arrived too lateFrozen or broken, waiting for the next full frame

6 of 20frames shown clean

100 msbehind live, the price of the window

62 msaverage travel time, same for both networks

3 frames arrive after the window closes. Each late frame spoils the picture until the next full frame. With the same average, the steady network is clean from a 65 ms window. The jumpy one needs 205 ms.

The average hides the problem

Both networks take the same time on average. Only the steady one gives a clean picture with a short window. The slowest moments decide.

One late frame spoils more than itself

Most frames only carry what changed since the one before. When one is missing, the picture stays frozen or broken until the next full frame, often a second or more away.

A wider window costs delay

Widening the window saves frames, but puts the whole picture further behind live. Your budget sets how wide it can be.

03Examples

Every stream has its own deadline

The right budget depends on who or what uses the data, and what happens when it is late. Here are five we see often.

Tele-operation

150-300 msfrom the camera on the machine to the operator’s screen

An operator drives a wheel loader from a control room far away. They steer by what they see, so the budget starts at the camera on the machine and ends on the screen in front of them. The steering command then needs its own trip back.

When data is late: The operator reacts to a moment that is already gone. At 20 km/h, a machine moves more than a meter in 200 ms.
What matters most: A delay that stays the same. People get used to a steady delay and drive around it. They can’t get used to one that jumps.

Tele-operation with NanoPing

Live video and remote production

200-500 msfrom the camera to the production team’s screens

Cameras at a stadium feed a production team in another city. Each camera makes a new picture every 33 ms, and the player at the other end shows them at that same pace, one slot per picture.

When data is late: A picture that misses its slot is skipped. The screen freezes or breaks up, and since each picture builds on the one before, it can stay broken until the next full picture.
What matters most: Every picture on time. In a 300 ms budget, a picture that arrives at 301 ms is as useless as one that never came.

See it measured: NanoPing vs. SRT

Voice and intercom

Under 150 msfrom one person’s mouth to the other’s ear

A director talks to a camera operator over a headset. The sound is cut into small pieces, often 20 ms each, and every piece has its own moment to play.

When data is late: A late piece is a click or a gap in a word. Once the delay passes about 300 ms, people start talking over each other.
What matters most: Short and steady at once. A conversation needs both.

Industrial robots

10-50 msfor each position or safety message

Self-driving robots slide under shelves and carry them around a warehouse. They share their positions with a fleet system and listen for safety signals over Wi-Fi or private 5G.

When data is late: A self-driving machine trusts each message for a short, fixed time. If the next one is late, it stops to stay safe. A network that jumps around shows up as machines halting for no visible reason.
What matters most: Steady timing. The budget is tight, and a miss costs a full stop, not a small glitch.

Telemetry and sensor data

Secondsfrom the sensor to the dashboard

Sensors on an offshore wind farm report temperature, vibration and power output to a control room on shore, once a second.

When data is late: Usually nothing. A reading that is a second old is still a good reading.
What matters most: Complete data. With seconds to spare, there is time to send anything that goes missing again. Real-time doesn’t always mean fast.

04Everywhere

Any live stream has a deadline

It isn’t only video and control. Wherever data describes a moment that is passing, it has a use-by time.

Machine visionDistance ÷ belt speedA camera spots a bad part on a belt. The reject signal has to reach the pusher before the part rolls past it.

Cloud gaming and VR11-17 ms per frameA new picture 60 to 90 times a second. A late one feels like lag, or like the world sliding under your feet.

DronesThe failsafe timerWhen control messages stop for too long, the drone gives up on the pilot and hovers or flies home.

Power grid protectionA few msTrip signals between substations have to land within milliseconds to shut off a fault before it spreads.

Vehicle to vehicleAbout 100 msCars share their position ten times a second. An older message describes a car that has already moved on.

Playing music togetherAbout 25 ms one wayMusicians in different places start to fall out of time with each other beyond this.

Prices and live oddsBefore the next updateA price is worth nothing once a newer one exists. Acting on the old one costs money.

Live captionsIn step with the pictureText that shows up after the speaker has moved on is no help to the viewer.

05Find your number

Three questions to find your budget

1
What happens when data is late?
A frozen picture, a robot that stops, a lost word, or nothing at all. This tells you what a miss costs.
2
Who or what uses it?
Eyes, ears, hands or a machine. People get used to a steady delay. Machines follow fixed timers.
3
Where does the clock start and stop?
From the camera to the screen, the stick to the machine, the sensor to the dashboard. Count every step, not just the network.

Then put it in one sentence

Our video must get from the camera to the screen within 150 ms, and the network gets 90 ms of it.

06With NanoPing

Built to hit a deadline, not an average

Give NanoPing your budget and it spends it on getting data there in time, over the links you already have.

One deadline to set

You tell NanoPing how long data may take. Everything below works inside that limit.

Repair data up front

Extra repair data travels with the stream, so a lost packet is rebuilt on arrival instead of waiting for a resend.

Resends as a backup

What the repair data can’t cover is sent again, while there is still time.

Every link at once

Traffic uses all your connections at the same time, so one slow link doesn’t make data late.

Steady delivery

Data is handed on at an even pace, so the receiving end sees the same delay every time.

See it measured against SRT How it fits together

Tell us your budget

Send us your deadline and the network you have. We’ll show you what fits inside it, and test it with you.

Talk to an engineer Read the documentation