How to Fix Missed Rallies in an AI-Analysed Tennis Video

Automatic rally detection will find most of the points in your footage. It will not find all of them. That is true of every tool that does this, and the honest question is not whether a detector misses points — it is what happens to your afternoon when it does.

The answer depends entirely on what you are making.

A missed point costs nothing, or it costs everything

If you are building a highlight reel, a missed rally is a non-event. You were going to keep eight points out of a hundred and forty. Losing one you never planned to use changes nothing.

If you are condensing a whole match — the kind of video where a viewer follows the score from the first point to the last — one missed rally breaks the entire thing. The score no longer adds up. Someone watching cannot tell whether they missed a point or the video did.

That is the difference between “slightly annoying” and “unusable”, and it is why correction matters as much as detection.

Finding the ones that were missed

A missed rally leaves a fingerprint: a gap that is too long.

Between two real points there is a predictable amount of dead time — players walk back, pick up a ball, towel off, serve again. Fifteen or twenty seconds. Occasionally much longer, at a changeover or a break.

So when you see a ninety-second gap in the middle of a set, one of two things is true. Either something legitimate happened — a changeover, a long deuce argument, someone re-tying a shoe — or there is a rally sitting in there that the detector walked past.

You cannot tell which from the number alone, and no threshold can tell you either. A rule that flags “every gap over 45 seconds” will hide the real miss that happened to be 40 seconds long, and will bury you in changeovers. The only thing that actually works is looking: play the gap, and in three seconds you know.

What you want, then, is not a clever filter. It is every gap, listed, sorted longest first, each one playable in one click. Work down from the top and the real misses surface fast, because they cluster at the long end.

The four things that go wrong

Missed points are the most damaging failure, but they are not the only one. Across a full match you will typically meet four:

A point was missed entirely. Nothing was detected where a rally happened.
Two points were run together. The detector saw one long rally where there were two, because the players restarted quickly.
One point was cut in half. The opposite — a single rally became two entries, usually split at a long pause mid-point.
The boundary is off. The rally is there, but it starts a beat too late and the serve toss is gone, or it ends before the ball has finished bouncing.

Each one needs a different fix, and all four are boundary work rather than editing: you are correcting a list of in and out points, not cutting video.

How this works in Rallytics

Rallytics has a Rally Editor built for exactly this pass. You open it from your rally list, and it puts the whole session on one timeline.

The list on the left holds every rally and every gap between them. Filter it to gaps only, sort by longest, and you have the search described above. Click any row to play it.

When you find one, four operations handle the four failures:

Add rally — draw a new one in the empty stretch, with two handles you place against the video.
Split — cut one entry into two, discarding the dead time in the middle. Two boundaries, not one, because the reason to split is that the players were walking back between the points and you do not want that time in either half.
Merge — join two entries that are halves of the same point.
Adjust — drag a start or an end to where it should have been. This is the one you reach for when the serve got clipped, and it pulls the start earlier, out into the pause before it.

Everything is undoable, several steps back, so working quickly costs nothing.

Two details that matter once you are actually following a score:

Detected rally numbers never move. A rally you add between #11 and #12 becomes #11a. If you have notes against #47, they still point at the same point after an hour of corrections.

Fix the list before you export, not after. Once a highlight video exists, patching it means matching timestamps between two files that no longer agree — which is the slow, error-prone route nobody wants to be on. Get the rally list right first, then export once.

What to expect

A correction pass on a full match is a handful of minutes, not an afternoon — you are playing a few dozen gaps at a few seconds each, and acting on the small number that turn out to hold a point.

That is the number worth comparing between tools. Detection accuracy gets quoted a lot and is hard to verify on your own footage. How long it takes to fix what was missed is something you can measure on your first video.