Safety

Fault modes covers the machine lying to itself. This page covers the other direction: something is in front of the machine, and the machine has to not hit it.

Two separate functions do that, and they are separate on purpose.

  safety_monitor collision_monitor
Watches the filter’s own health the lidar cloud
Trips on position stddev, slip points inside a zone
On trip latches — stays stopped releases when the zone clears
Scenario safety_stop the four hazard_* scenarios

The latch is the whole difference. A pose that has stopped being trustworthy does not become trustworthy again because the next message looked fine, so that stop is permanent until someone intervenes. An obstacle that walks away really is gone, and a latching obstacle stop could never satisfy the resume half of the test. Neither monitor can be bypassed, because both sit in the command path rather than beside it:

flowchart LR
    ctrl["ArcTracker"] -->|cmd_vel_nav| cm["collision_monitor<br/>obstacle zones"]
    cm -->|cmd_vel_raw| sm["safety_monitor<br/>filter health, latching"]
    sm -->|cmd_vel| world["simulation"]
    pc["lidar cloud"] --> cm
    ekf["EKF"] --> sm

If either node dies, nothing republishes its output topic, the simulator’s cmd_vel_timeout expires and the machine stops. Failing open is not an option either of them has.

Saying that it stopped

Stopping is half of it. ISO 18497-4 treats a protective stop as a state the machine is in and reports, not merely one it enters, and until recently this stack did not report it at all. By the time a gated stop reaches safety_monitor it is already a zero twist on cmd_vel_raw, which is indistinguishable from the controller legitimately commanding a halt. The collision monitor did publish collision_monitor_state — but a search of the repository found that string in exactly one place, the config line that names it. Nothing subscribed. The observable behaviour of a protective stop was that the machine went quiet and, four minutes later, Nav2 gave up with error 105.

safety_monitor now subscribes to that topic and reports it as safety/protective_stop in the DiagnosticArray it already publishes on ~/status, so one place answers “is the machine safe, and if it is stopped, why”:

State Level Message
no protective device on this run WARN not monitored
nothing being gated STALE idle
commands gated, device silent ERROR protective device silent while commands are still being gated
DO_NOTHING OK clear
SLOWDOWN, LIMIT, APPROACH WARN speed reduced by <polygon>
STOP WARN protective stop: <polygon>
STOP, held past protective_stop_hold ERROR held N s by <polygon>: stopped, not recovering

Four of those rows exist because the interesting failure is not the stop. It is not monitoring, which a diagnostic that only ever said “no hazard” would report identically to a clear road; that is the failure mode the diagnostic exists to catch, so a run with no protective device says so every cycle rather than staying silent. Silence while commands are still flowing through the gate is separated from silence while nothing is driving, because only the first is a fault — and it is given the same settling tolerance as the other checks, since without it the startup race between the two publishers reported ERROR every time.

It is reported and deliberately not latched. safety_monitor’s own stops latch, and routing a hazard stop into that path would turn every nuisance stop into a mission kill — which, given the turn described below, would end most runs. The elapsed hold is carried as a held_s key so the difference between “a person is walking past” and “this machine has been parked in a field for two minutes” is visible without inferring it from timestamps.

Two limits worth stating. protective_stop_hold defaults to 10 s and is not measured — no measurement of a legitimate stop’s duration exists yet. It is an operator-attention threshold, not a safety one; the stop is already in force either way and nothing actuates on it. And the report is only as live as its source: Nav2’s collision monitor publishes state when it has a command to gate, so once the controller gives up the topic goes quiet and the diagnostic drops to idle. That is honest — nothing is being gated any more — but it means the hold escalation only fires while something is still trying to drive.

Verified by driving the shipped binary through all seven rows above (results/_protective_stop_probe.py), and end to end against O3DE, where a hazard stop 48 s into a gated run was reported as WARN protective stop: hazard [held_s=0.9] within one check period.

The same gap, found by looking for it

safety_monitor’s other gate had it too. It stops the machine when the slip ratio exceeds max_slip_ratio, default 0.8 — but the ratio is rated against ground_speed, a reference that only exists where something is watching the ground, and rateWheels leaves the ratio at zero when nothing is. Zero is perfect grip. The gate therefore read OK for the clearest possible reason to be suspicious of it.

That is not hypothetical: the reference is modelled as a radar in the headless simulator and stood in for by the cameras’ visual_ground_speed under the crosscheck profile, and O3DE has neither on the default robust profile. So every O3DE run so far has been driving with safety/traction reporting OK and nothing behind it.

SlipEstimate already carried reference_valid; the check simply did not read it. It does now, and an unreferenced estimate reports WARN not monitored rather than being gated on — a check with nothing to measure must not latch a stop, and must not claim to pass either. slip_monitor was never at fault: it widens the covariance it publishes exactly as designed, because absence of evidence is not grip. Note the covariance and the ratio go to different places — wheel/odom/checked is fused only by the crosscheck profile — so this is a reporting defect, not a filter one.

Deliberately not fixed by putting visual_ground_speed into the orchard bringup. That would arm a latching 0.8 gate with a reference whose noise on this machine has never been measured, and the first false trip would end a mission. Measure what the ratio actually reads there first.

The collision monitor link in that chain is present in the orchard bringup and in any scenario that declares hazards. Scenarios that declare none turn it off with collision_monitor:=false, because their thresholds were measured on the shorter chain and the monitor republishes on its own cycle rather than per message. With Nav2 driving, controller_server and behavior_server are the nodes remapped onto cmd_vel_nav; with the harness driving, it is scenario_runner. Same gate, either way — nav_hazard_stop measures 0.782 m through Nav2 against 0.785 ± 0.006 through the harness.

Why this layer is not learned

EU Regulation 2023/1230 applies from 14 January 2027 and puts an ML-based safety function into the high-risk category, needing third-party assessment. So the protective layer here is deliberately geometric and boring: points in a polygon, a threshold, a stop. Anything smarter may add caution on top of it and may never release its stop.

This also matches how France’s ARPA protocol treats the problem — obstacle detection and human detection are assessed separately. We follow that split. The protective layer does not know what a person is, and does not need to.

The two zones

ISO 18497 clause 4.12 asks for a warning zone and a hazard zone, which is what the collision monitor is configured with.

Zone Extent from rear axle Half-width Action
warning 5.0 m 0.8 m slow to 50%
hazard 3.5 m 0.8 m stop

The lengths are derived, not chosen. At 0.8 m/s:

braking at max_decel 0.4 m/s^2   v^2/2a  = 0.800 m
one cloud frame at 11.92 Hz              = 0.067 m
one monitor cycle at 20 Hz               = 0.040 m
                                  total  ~ 0.907 m, taken as 1.0 m

The footprint reaches 2.5 m ahead of the rear axle, which is the frame origin, so the hazard zone ends at 3.5 m. Re-derive this if the speed, the deceleration or the cloud rate changes; it is not a constant.

The half-width was measured in O3DE, and it is the number that moved most. It was originally 0.5 m, which reaches only 50 mm past a machine whose own half-width is 0.45 m — less than the 0.11–0.21 m worst position error the suite itself records. An object the machine was about to graze could sit outside the zone. It is now 0.8 m; see what the orchard allows for why not wider.

Worth knowing when reading the results: the warning zone does most of the work. Slowing to 0.4 m/s before the hazard zone is reached is why the centred stop clears the object by 0.78 m rather than by roughly 0.16 m.

The test object

ISO 18497 clause 5.3 specifies a seated human as two stacked coaxial cylinders:

Part Diameter Height Centre height
body 380 mm 530 mm 265 mm
head 140 mm 270 mm 665 mm

800 mm overall, water-filled, matte polyethylene, olive green Munsell 2.5 GY 3.5/3.

Both branches use those dimensions. The colour and material are not modelled — they need a purpose-built material asset and change nothing for a lidar. That is fine for every claim on this page and is not fine for any camera-based claim, so it has to be revisited before one is made.

Two branches, and what each can honestly answer

The same standard is tested twice, because neither environment can answer both halves.

  Headless O3DE
The object is a synthetic cloud injected by kraken_sim real geometry the lidar has to find
Tests the protective layer perception, honestly
Needs a GPU no yes
Runs in CI yes no
Repetitions cheap — the ≥7 that 18497-4 wants one sim restart per case
Cannot answer can anything be seen at all anything needing many repeats

Headless assumes perception works and asks whether the machine reacts correctly. O3DE removes that assumption and asks whether there is anything to react to. A stopping distance measured in O3DE would be measuring both at once, which is why the gates live in the headless suite.

The headless scenarios

Five, and none of the first four means much alone — that is the point of the set.

Scenario Proves Measured
hazard_stop_centre it stops for an object in its path clearance 0.7854 ± 0.0063 (7 seeds), path 16.52 m
hazard_blank_run it does not stop when the aisle is empty path 27.94 m, no nuisance stop
hazard_pass_offset it does not stop for a clearly non-colliding offset clearance 0.5600, path 27.95 m
hazard_start_blocked it will not move off at all with the zone occupied path 0.0000, clearance 0.3100
nav_hazard_stop the gate holds when Nav2 is driving, not just the harness clearance 0.7817, path 8.53 m

Read them together. hazard_stop_centre alone is passed by a machine that stops for everything; hazard_blank_run alone is passed by one that stops for nothing; hazard_pass_offset is the one case where not stopping is the required answer, which ISO 18497-4 explicitly allows for an object close to the trajectory but not on it. A machine that halts for everything within a metre never finishes a row.

hazard_start_blocked is NARO’s optional item g). It needs care for two reasons. Its object sits at x = 3.0 m because the footprint already extends 2.5 m ahead of the origin and the stop zone ends at 3.5 m, leaving a valid window of roughly 2.69–3.5 m; the first attempt at 2.0 m put the object inside the machine’s own body and scored −0.19 m clearance, which is how the mistake surfaced. And a path length of zero is also the signature of a dead command pipeline, so on its own it would be passed by a machine that cannot move at all. hazard_blank_run is what tells those two apart.

The first four drive from the harness, which exercises the monitor but not the command path the orchard actually uses. nav_hazard_stop is the fifth because the production remapping is a separate thing that can be wrong: there, Nav2’s controller_server and behavior_server are what have to be pointed at cmd_vel_nav, and a monitor subscribed to a topic nobody publishes gates nothing while looking perfectly healthy. It agrees with the harness-driven figure to 4 mm, and it is the scenario that showed the stop is a freeze.

Negative controls

Every gate here has one, because a stop test with no control is close to worthless.

Control Result
all three enabled flags flipped to False clearance −0.19 m (contact), path 27.98 m
hazard_start_blocked with polygons disabled path 15.97 m instead of 0.0
the orchard mission with collision_monitor:=false drives through the pose the gated arm freezes at

Both headless controls confirm the monitor, not something else, is what produces the passing result. The third is the O3DE equivalent: orchard_bringup.launch.py takes collision_monitor:=false for exactly that purpose, because a stop observed in the orchard is only attributable to the gate if the same run without it does something different.

Thresholds are set from sweeps rather than guessed. hazard_pass_offset returns a standard deviation of 0.0000 across seven seeds — with no steering input the lateral gap is pure geometry and has nothing to disperse — so its floor is not absorbing run-to-run noise, it is set well below the measurement so that a machine which starts wandering laterally trips it before it trips the object.

Run them:

python3 -m pytest src/kraken_scenarios/test/ -q          # 20 scenarios, ~10 min
ros2 run kraken_scenarios sweep hazard_stop_centre -n 7

What O3DE showed

Place the ISO object in the orchard, then drive at it. KRAKEN_HAZARD="x,y" puts it in the level; unset removes it. See simulation for the placement mechanics.

Detection against range

Unobscured, on the aisle centreline:

Gap Points Beams Height resolved
11.97 m 11 3 0.584 m
9.94 m 17 4 0.666 m
6.91 m 53 8 0.785 m
5.39 m 105 10 0.766 m
3.87 m 419 20 0.794 m

Detection is never marginal inside either zone — 105 points at the warning edge, 419 at the hazard edge — so the polygons are backed by real returns rather than assumed ones.

But the object only reaches its true 0.8 m height close in. At 12 m the returns stop at 0.584 m, because the 140 mm head is too narrow to catch a beam at that range. A perception layer keyed on human stature would miss it. The protective layer does not, because it thresholds on any point above 0.15 m.

Geometry check, at 9.7 m: near face 9.544 m against a centre at 9.72 gives a 0.176 m radius against the specified 0.19, and the width reads 0.338 m across a 0.380 m cylinder. Both fall short in the direction chord sampling predicts.

What the orchard allows

Driving an aisle and asking how close the canopy comes to the centreline, inside the hazard zone’s reach, over 85 samples and 22 m:

   
nearest canopy return 0.88 m
5th percentile 0.89 m
median 1.00 m

Which sets a hard ceiling on zone half-width:

Half-width Samples intruded
0.8 m 0 / 85
0.9 m 7 / 85
1.0 m 41 / 85

0.8 m is the widest this orchard tolerates. That is where both zones now sit — 350 mm past the machine’s flank instead of 50 mm, which covers the worst tracking error, at no cost in nuisance stops while the machine is driving a row. That qualifier was added after the fact: those 85 samples were all taken along an aisle, and the turn at the end of it is a different problem. See the turn is where it breaks.

The foliage matters at the object’s own height, not just overhead: returns occupy 1.2–2.5 m laterally across every height band including 0.2–0.8 m. The ground sits below +0.05 m, so the 0.15 m height floor removes it cleanly.

The turn is where it breaks

Every result above holds the machine parallel to the rows. Run a real coverage mission instead — three aisles, cover_rows, and no object anywhere in the level — and the gate stops the machine dead on the headland turn, on trees.

The same mission was run twice against the same level, differing only in collision_monitor:=false:

  gate on gate off
Frames with ≥4 points in the hazard box 1259 32
Duration of the intrusion 138 s and still going 2.9 s
Yaw while intruding 74.7 → 74.8° 74.7 → 81.4°
Position while intruding (−56.32, −2.72) → (−56.32, −2.69) (−56.32, −2.72) → (−56.23, −2.32)
Aisles covered 0 of 3 1 of 3

Both arms enter the turn at the same pose to within 2 cm and both see the canopy enter the box there. Ungated, the machine keeps swinging, the trunk leaves the box 2.9 s later, and it drives 26 m up the aisle. Gated, it stops at yaw 74.8° and never moves again — and because the stop zeroes angular velocity too, it cannot turn away from the thing that is stopping it. That is the deadlock: the only motion that would clear the box is the motion the gate is refusing. Nav2 replans, tries a backup, gets Collision Ahead, and gives up 243 s later with every aisle missed.

This reproduced on both gated attempts, stalling at (−56.31, −2.70) and (−56.32, −2.72).

The arithmetic says this is not tunable by a few centimetres. Aligned, the 0.8 m corridor clears the nearest canopy return by 0.88 − 0.80 = 0.08 m. The box reaches 3.5 m ahead, so each degree of heading error swings its far corner by 3.5·sin(1°) = 61 mm — the whole margin is gone at about 1.3°. A headland turn is 90°. A fixed, axis-aligned rectangle is simply the wrong shape for a machine on a curve, and no half-width that still protects anyone will survive it.

Do not read this as “the gate caused the mission to fail”. The ungated arm managed 1 aisle of 3 as well. Its cause is not established — the nav log for that run was not captured — so all that is supported is that a second failure exists at the headland which the gate does not explain. Every run on this page does log inflation radius (0.550000) is smaller than the circumscribed radius (2.551803), against a footprint whose circumscribed radius is wider than the 1.75 m half-aisle it has to fit in; that is a candidate, not a finding. It has to be run down before any change to the polygons can be credited with anything.

Note also that covered understates what the machine did. A leg is a row plus the headland turn appended to it as one FollowPath, so a row driven end to end is still written off if the turn after it fails. The gated arm swept the whole of aisle 0 down the centreline and scored it missed.

The blank run

ARPA 2 requires each harsh-condition test to be preceded by a blank test. Over 25 m of empty aisle, points intruding either polygon: 0 of 97 samples, both zones.

A counter that always returns zero looks exactly the same, so that result was controlled before being believed — widening the half-width made the same counter return 57 points at 1.0 m and 548 at 1.2 m, and dropping the height floor made the ground appear below +0.05 m. The zero is real.

Occlusion

With the object at the tree line, 1.3 m off the centreline and inside the foliage, it was detected at every range from 12 m in, with no blind frame — though close in it returns roughly 30% fewer points than in the clear aisle (335 against 475 at 3.4 m), so foliage is intercepting rays without hiding it.

Treat this as the weakest result on the page. It is one longitudinal position, and the object happened to stand in a gap between two trunks. Occlusion in an orchard is periodic; directly behind a trunk is a different test and it has not been run. What is supported is “not occluded at this position”, not “not occluded”.

What the standards ask, and where it is answered

Requirement Where
ISO 18497 4.12 — warning and hazard zones the two polygons
ISO 18497 5.3 — test object geometry both branches
ISO 18497 5.4.1 — object centred, then at lateral offsets hazard_stop_centre, hazard_pass_offset
ISO 18497 5.4.2 — obscuration by the crop partly: one canopy position, no dust
ISO 18497 5.2(d) — simulation must be validated against practical tests not done
ISO 18497-4 Annex D — ≥7 repetitions, dispersion of stopping position sweeps
ISO 18497-4 — never contact; passing a non-colliding object is acceptable hazard_pass_offset
ISO 18497-4 — a protective stop is a state the machine reports, not a silent halt safety/protective_stop
NARO item g) — blocked at start must prevent motion hazard_start_blocked
ARPA 2 — blank test before each harsh-condition test the O3DE blank run
EU 2023/1230 (14 Jan 2027) — ML safety functions are high-risk why this layer is geometric

Background on the national protocols: Piovaccari et al. 2025, doi:10.1007/978-3-031-84212-2_52.

ISO 3691-4:2023 is the neighbouring standard for industrial trucks and uses different targets — a horizontal Ø200 × 600 mm cylinder, a vertical Ø70 × 400 mm one, at 2–6% reflectance. Nothing here is tested against those.

Known gaps

Listed because they are the honest limits of the above, not as a roadmap.