Skip to content

Robot deployment fixes: bag recording + adding warning for robot-identity failure - #377

Merged
andrewjong merged 8 commits into
developfrom
johnliu/robot-deployment-fixes
Aug 5, 2026
Merged

Robot deployment fixes: bag recording + adding warning for robot-identity failure#377
andrewjong merged 8 commits into
developfrom
johnliu/robot-deployment-fixes

Conversation

@JohnYanxinLiu

@JohnYanxinLiu JohnYanxinLiu commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

What features did you add and/or bugs did you address?

  • Which GitHub issue does this address?

None — these were found while porting fixes back from a real mocap-room Jetson
deployment. Independent of the OptiTrack stack (#374/#375/#376); merges in either order.

  • Additional description if not fully described in the GitHub issue

Bag recording was broken four independent ways. Each one alone is enough to produce
no usable bag, and none of them reports an error where you would look for it:

  1. The recorder never launched. robot-base-docker-compose.yaml has forwarded
    RECORD_BAGS, but logging.launch.xml hardcoded record_bag=false and
    onboard_autonomy_all.launch.xml includes it with no arguments, so nothing ever
    overrode the default.

  2. Recorder status was bridged backwards. bag_record/bag_recording_status sat under
    the "Outgoing from Robot" header in domain_bridge.yaml but was configured
    gcs -> robot, the same direction as the command it answers.

  3. The main section errored out on an incompatible flag. bag_record_node passes
    --exclude to ros2 bag record, which Jazzy renamed to --exclude-regex.

  4. .gitignore was readded to gcs and robot/bags/ and gcs/bags

Robot identity resolution failed somewhat silently silently. On the hostname-sourced profiles (l4t,
voxl) the OS hostname is the robot's identity.

  • Please add videos and images

N/A — console-observable. The new warning at container startup:

WARNING: could not resolve a robot identity from 'airlab-desktop' using default_robot_name_map.yaml.
         ROBOT_NAME='unknown_robot' ROS_DOMAIN_ID='0'
         Topics will not be namespaced under /robot_<n>, so nothing will reach
         the rest of the stack. Fix by either:
           - on the HOST (not in this container): hostnamectl set-hostname robot-1,
             which the default map resolves to robot_<n> on domain <n>; or
           - adding a mapping YAML under robot/docker/robot_name_map/ that
             matches your hostnames and pointing ROBOT_NAME_MAP_CONFIG_FILE at it.
         If ROBOT_NAME is empty rather than unknown_robot, check stderr above
         for a resolve_robot_name.py error (missing or malformed map file).

How did you implement it?

logging.launch.xml reads $(env RECORD_BAGS false) with an explicit default so it stays
launchable when unset, and gains LOG_CONFIG (default log.yaml) to select the topic set.

Recording still starts and stops via the GCS set_recording_status toggle — that is the
intended control path and is unchanged. This is NOT currently present in the modern foxglove GCS. This must still be added but was out of scope for this PR.

For the identity warning, the unknown_robot fallback itself is unchanged. It keeps an unidentified robot out of every real robot's namespace. Merely a more apparent warning is provided to the user upon startup.

The check tests the resolved identity rather than the sentinel value, because only one
of several failure paths yields unknown_robot:

Also drops ROBOT_NAME / ROS_DOMAIN_ID from overrides/l4t-px4-realrobot.env. No compose
service declares either, and Compose only injects what a service names in environment:,
so an env file could never set them. Verified with
grep -rn '${ROBOT_NAME\|${ROS_DOMAIN_ID' --include=*.yaml ., which finds only
ROBOT_NAME_MAP_CONFIG_FILE. robot_identity.md and the configure-multi-robot skill both
prescribed that route and now point at the hostname or a map file instead.

How do you run and use it?

All commands below are against robot-desktop.

1. Negative control — this is what demonstrates the fix

The old failure was an absent node, not a broken one, so check that first:

airstack up robot-desktop          # .env ships RECORD_BAGS="false"
airstack connect robot
ros2 node list | grep bag_record

cntrl + b + d # to exit
# expect: no output

2. Recorder comes up, and comes up idle

airstack down
AUTOLAUNCH=true RECORD_BAGS=true airstack up

ros2 node list | grep bag_record
# expect: /robot_1/bag_record        <-- absent before this PR

ros2 topic echo --once /robot_1/bag_record/bag_recording_status
# expect: data: false

3. Status reaches the GCS (bridge direction)

#In another terminal
airstack connect gcs
ros2 topic echo --once /robot_1/bag_record/bag_recording_status
# expect: data: false
# before this PR: hangs forever — the topic was bridged the wrong way

Equivalently, the rqt control panel's Recording: label should now read NO rather than
staying blank.

4. Recording actually produces both sections

#In the gcs panel
ros2 topic pub --once /robot_1/bag_record/set_recording_status std_msgs/msg/Bool '{data: true}'

# wait a few seconds
ls robot/bags/
# expect BOTH: airstack_<ts>/  and  zed_<ts>/
# before this PR: only zed_<ts>/ — the airstack section died on the --exclude flag

The airstack_<ts> directory is the check that matters. Confirm it has content:

# On the host machine
docker exec airstack-robot-desktop-1 bash -lc \
  "python3 -c \"
import yaml,glob
m=yaml.safe_load(open(sorted(glob.glob('/bags/airstack_*/metadata.yaml'))[-1]))
t=[x['topic_metadata']['name'] for x in m['rosbag2_bagfile_information']['topics_with_message_count']]
print('topics:', len(t), '| sensors/* (should be 0):', sum('sensors' in x for x in t))
\""
# expect roughly: topics: 162 | sensors/* (should be 0): 0

Then stop, and confirm the mcap finalizes:

# in the gcs terminal
ros2 topic pub --once /robot_1/bag_record/set_recording_status std_msgs/msg/Bool '{data: false}'

zed_<ts>/ will be near-empty in sim — that section lists ZED topics nothing publishes
there. Expected, and unchanged by this PR.

5. Bags are gitignored

git status --short        # expect clean, despite robot/bags/ now holding bags

robot/bags/ is created by the container and is root-owned, so clean up with
docker exec airstack-robot-desktop-1 bash -c 'rm -rf /bags/*' rather than plain rm.

6. Identity warning — both branches, no files to edit

docker exec -e ROBOT_NAME_SOURCE=bogus              -it airstack-robot-desktop-1 bash   # unknown_robot branch
docker exec -e ROBOT_NAME_MAP_CONFIG_FILE=nope.yaml -it airstack-robot-desktop-1 bash   # empty-ROBOT_NAME branch

Both print the warning at shell start. A normally-resolving container prints nothing.

Testing with PyTest

  • What pytests did you add?

None.

Documentation

  • Was mkdocs.yml updated? (y/n)

ndocs/robot/docker/robot_identity.md is already in the nav (mkdocs.yml:106); this
edits that existing page rather than adding one.

  • Sufficient scope?

Yes. robot_identity.md was corrected.

  • Sufficient visual media?

The warning text above is the user-facing surface; there is no GUI component.

Versioning

  • Version bumped?

Yes

Behavior change to flag for reviewers

Anyone currently running RECORD_BAGS=true for GCS bags will now also get a bag_record
node on every robot. It comes up idle and records nothing until the GCS toggle says so, but
it is not a no-op. RECORD_BAGS=false opts out.

Follow-up branch

A Foxglove Bag Recorder panel.

🤖 Generated with Claude Code

@JohnYanxinLiu
JohnYanxinLiu marked this pull request as draft August 3, 2026 02:53
JohnYanxinLiu and others added 3 commits August 4, 2026 11:42
LOG_CONFIG selects which topic set in logging_bringup/config to record, default
log.yaml.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@JohnYanxinLiu
JohnYanxinLiu force-pushed the johnliu/robot-deployment-fixes branch from 5710c4b to fd4b131 Compare August 4, 2026 15:43
@JohnYanxinLiu JohnYanxinLiu changed the title Real-robot deployment fixes: make RECORD_BAGS and ROBOT_NAME overrides take effect Robot deployment fixes: the bag recorder never launched, and identity resolution failed silently Aug 4, 2026
@JohnYanxinLiu
JohnYanxinLiu force-pushed the johnliu/robot-deployment-fixes branch from bd428bb to 92ca2e6 Compare August 5, 2026 07:50
JohnYanxinLiu and others added 3 commits August 5, 2026 03:55
It was bridged gcs -> robot, the same direction as the command it answers, so
status never reached the GCS and the rqt Recording: label stayed blank.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ros2 bag record renamed --exclude to --exclude-regex, and the old name is now an
ambiguous prefix of four options, so argparse rejected the command and any section
using exclude: recorded nothing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
#318 dropped robot/bags/.gitignore and gcs/bags/.gitignore while moving a dozen
others; nothing has covered recorded bags since.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@JohnYanxinLiu
JohnYanxinLiu force-pushed the johnliu/robot-deployment-fixes branch from 92ca2e6 to d89765b Compare August 5, 2026 07:56
@JohnYanxinLiu JohnYanxinLiu changed the title Robot deployment fixes: the bag recorder never launched, and identity resolution failed silently Robot deployment fixes: bag recording (4 bugs) + silent robot-identity failure Aug 5, 2026
@JohnYanxinLiu JohnYanxinLiu changed the title Robot deployment fixes: bag recording (4 bugs) + silent robot-identity failure Robot deployment fixes: bag recording + adding warning for robot-identity failure Aug 5, 2026
@JohnYanxinLiu
JohnYanxinLiu marked this pull request as ready for review August 5, 2026 16:07
@andrewjong
andrewjong merged commit 234587a into develop Aug 5, 2026
2 checks passed
@andrewjong
andrewjong deleted the johnliu/robot-deployment-fixes branch August 5, 2026 18:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants