A practical troubleshooting rule: fix reliability before “accuracy”
In real deployments, “bad accuracy” is rarely the root problem.
Most failures originate from a small set of breakpoints: unstable power, broken time sync,
poor anchor geometry in worst zones, network latency/loss, or event logic that amplifies noise.
1) The triage order (do this every time)
- Anchor health: power, reboot loops, online/offline stability.
- Time synchronization: sync status, drift, master/slave timing behavior.
- Anchor participation in the target zone: how many anchors actually “see” the tag simultaneously?
- End-to-end latency: movement → position → event → alert.
- Event logic: hysteresis, dwell gating, boundary rules, false trigger suppression.
Only after these are verified should you tune filters, smoothing, or thresholds.
2) Symptom-based troubleshooting map
2.1 Symptom: tracks “jump” across walls or snap to wrong areas
Most likely causes
- NLOS/multipath dominance near metal racks, tanks, cranes, or vehicles.
- Weak geometry (anchors mostly on one side of the zone).
- Anchor coordinate errors (wrong height, mirrored axes, incorrect rotation).
What to check (evidence)
- In the affected zone, how many anchors contribute to each fix (not site-wide average)?
- Does the error only occur at corners/ends/portals (geometry issue) or everywhere (systemic)?
- Verify anchor coordinates and map alignment against physical reference points.
Fixes that work in production
- Add or reposition anchors to improve geometry in the specific worst zone.
- Move anchors away from large metal surfaces; raise height to reduce shadowing where possible.
- Correct coordinate frames (height, orientation). Re-run baseline paths after correction.
2.2 Symptom: alarms trigger too late (especially collision warnings)
Most likely causes
- Excessive filtering/smoothing that improves “track beauty” but adds delay.
- Update rate too low for the speed and safety distance.
- Network/backhaul latency spikes under load.
What to check (evidence)
- Measure end-to-end latency: physical motion timestamp → event timestamp → alert timestamp.
- Check whether latency grows during peak traffic (shift changes, vehicle clusters).
- Confirm the configured update rate matches the safety envelope.
Fixes
- Reduce smoothing window; move to event-level hysteresis rather than heavy track smoothing.
- Increase update rate only in critical zones; keep lower elsewhere to control noise and power.
- Harden the network path (wired where possible; prioritize RTLS traffic; reduce hops).
2.3 Symptom: frequent false alarms near zone boundaries
Most likely causes
- Boundary “ping-pong” due to jitter.
- No hysteresis or dwell gating in event rules.
- Zone edges placed too close to reflective surfaces or moving obstructions.
What to check
- Compare boundary events with track variance in the same area.
- Identify whether false alarms correlate with vehicles passing between tag and anchors.
Fixes
- Implement hysteresis: require boundary crossing persistence (time or distance) before triggering.
- Use dwell-based confirmation for non-emergency events.
- Redraw zones to avoid “knife-edge” boundaries in high-multipath locations.
2.4 Symptom: tags “freeze” or disappear intermittently
Most likely causes
- Anchor dropouts (power instability, PoE negotiation issues, cable faults).
- Wireless beacon battery sag in low temperature or end-of-life.
- Network packet loss or gateway saturation.
What to check
- Anchor uptime logs: does the same anchor flap offline repeatedly?
- Packet reception / participation count trends over time (not just instantaneous snapshots).
- Battery age and ambient temperature history for wireless beacons (e.g., WX/XB).
Fixes
- Stabilize power: validate PoE switches, cable runs, grounding, and connector quality.
- Replace or rotate wireless beacons reaching battery end-of-life; tighten battery maintenance SOP.
- Reduce network hops; segment RTLS traffic; validate gateway placement and capacity.
3) Failure modes by layer (where to look first)
3.1 Power & physical layer
- PoE instability: causes anchor reboots and “random” jumps that look like algorithm issues.
- Mounting height/orientation drift: anchors shifted during maintenance or construction.
- Water/dust ingress: shows as intermittent faults over weeks, not immediate failure.
3.2 Time synchronization layer
When time sync degrades, the system often produces “plausible but wrong” trajectories.
In tunnel-style deployments, wireless sync (e.g., STD) must be verified under real operating load.
- Validate sync status continuously, not only at commissioning.
- Watch for drift after firmware updates, power cycles, or topology changes.
3.3 Geometry & environment layer
- Worst-zone geometry: the center of the site may look perfect while corners fail.
- NLOS hot spots: near racks, tanks, cranes, and moving vehicles.
- Portal effects: indoor/outdoor boundaries where reflections and sky-view change abruptly.
3.4 Network & compute layer
- Latency spikes often appear only at peak traffic times.
- Packet loss manifests as intermittent freezes and missed events.
- Gateway constraints are common in restricted-network sites (private networking / LPWAN gateways).
3.5 Event logic layer
- False alarms are usually rule design problems: missing hysteresis, missing dwell gating, or unrealistic thresholds.
- Missed alarms are often latency + update-rate mismatches, not accuracy limits.
4) GPS RTK failure modes (for outdoor or hybrid deployments)
When RTK behaves poorly, the error is usually not the rover terminal—it is the reference chain:
reference station placement, correction delivery, or antenna environment.
4.1 Common symptoms
- Slow or failed RTK fix convergence.
- “Fixed” solution drops to float intermittently.
- Centimeter claims collapse to meter-level in specific yard areas.
4.2 High-leverage checks
- Reference station antenna sky view and multipath sources (near buildings/metal).
- Correction delivery stability (NTRIP / network path).
- Rover antenna placement on vehicles (shadowing, cable integrity for split designs like URTC).
5) A production-grade “evidence pack” for acceptance and maintenance
To avoid endless debates, every deployment should maintain a small evidence pack that can be re-run:
- Worst-zone walking/driving paths (repeatable routes).
- Anchor participation report per zone (how many anchors contribute).
- End-to-end latency measurement for the key event types.
- Event correctness log (false triggers / missed triggers with timestamps).
6) When to stop tuning and redesign
If a zone cannot meet stability requirements even after clean power, verified time sync, and reasonable rule hysteresis,
the fix is usually structural: geometry, mounting, or environment constraints.
At that point, tuning becomes a time sink—redesign the anchor/beacon layout in that zone.
TL;DR
Most RTLS problems are not “accuracy issues.” They are system reliability issues caused by one of five breakpoints: power, time sync, geometry, network latency/loss, or event logic.
Troubleshooting should follow a strict triage order: confirm anchor health → confirm time sync → confirm anchor participation in worst zones → confirm network end-to-end latency → only then tune filters or thresholds.
Key takeaways
- If anchors are unstable, no solver tuning will save the system—fix power/time sync first.
- “Jitter” complaints usually come from geometry or NLOS, not from tags.
- False alarms are often event logic + latency + hysteresis problems, not precision problems.
- Worst zones (corners, rack ends, portals) are where failures surface first—test there.
- Treat troubleshooting as evidence collection: participation count, latency, and event correctness.
Quick facts
FAQ
Why does RTLS look stable in a demo area but fail in production zones?
Demo areas usually have clean geometry and low obstruction. Production zones include corners, portals, metal density, and traffic—those conditions change anchor participation and timing stability.
What is the fastest way to identify if the issue is geometry or network latency?
Compare (a) anchor participation count in the failing zone with (b) end-to-end latency spikes. Geometry issues are location-specific; network latency issues correlate with time/load.
How do you reduce false zone-entry/exit events without hiding real events?
Use hysteresis and dwell gating at the event layer, not heavy smoothing at the track layer. Track smoothing delays responses and often causes late alarms.
Why do collision alarms sometimes trigger late even with good positioning?
End-to-end latency (solver + network + event pipeline) is too high, or update rate is insufficient for the speed and warning distance.
What should be re-tested after site changes?
Worst zones: rack changes, machine relocation, new metal structures, doorway/portal modifications, or anchor relocation—all can shift multipath and geometry.