Robot Retarget Online · 在线人形机器人动作重定向

在线地址：https://imchong.github.io/Robot_Retarget_Online/

Browser-based humanoid motion retargeting — a pure-frontend port of GMR (General Motion Retargeting) that runs entirely in your browser via the official MuJoCo WebAssembly bindings. No backend, no installation. Two retargeting engines are selectable in the UI: GMR and OmniRetarget (an in-browser interaction-mesh adaptation layered on GMR's solver — preserves the relative structure between keypoints).

纯前端实现的人形机器人动作重定向工具：BVH 动捕预览 → 重定向参数配置 → 机器人动作预览与导出， 全部计算（含 MuJoCo 正运动学/雅可比与微分 IK）在浏览器内完成。

Demo · 在线演示

↑ 在线站点三步走完整流程（无需安装、无后端）： ① BVH 预览 打开自己的 .bvh 或直接加载 LAFAN1 示例动作，播放/拖动时间轴看动捕； ② 重定向设置 选机器人（示例为 Unitree G1 29 DoF）与引擎，查看人体关节 ↔ 机器人 body 的关键点对应（3D 连线）与两阶段权重映射表； ③ 重定向预览 一键求解（1800 帧 ~1.1 s），机器人动作回放并叠加人体关键点、逐帧误差曲线，导出 NPZ / CSV / JSON。

The GIF is recorded from the built site by node scripts/record_site_demo.mjs (Playwright + ffmpeg; needs npm run build first).

Pages · 功能页面

Page	功能
BVH Viewer	BVHView 风格的动捕预览：拖拽打开 .bvh、胶囊体骨架渲染、播放/拖动时间轴/变速、关节层级树与关节信息；示例动作含 LAFAN1 人形与 Labrador 狗四足动捕；视频生成 BVH（浏览器内 MediaPipe 姿态估计，实验性）
Retarget Config	引擎选择（GMR / OmniRetarget）、机器人选择（人形 Unitree G1 / Booster T1 等，四足 Unitree Go2 / A1；或从 Robot Description Gallery 挑选人形 / 四足 URDF，并可一键跳转 Gallery 查看对应模型）、IK 关键点映射表（两阶段权重/偏移）、人体缩放表、求解器参数；配置 JSON 与 GMR ik_config 双向兼容
Retarget Preview	引擎切换、一键重定向（带进度）、机器人动作回放、人体关键点叠加对比、逐帧误差曲线、导出 NPZ / CSV / JSON

How it works · 技术要点

• Algorithm (GMR): faithful TypeScript port of GMR's two-stage differential IK (motion_retarget.py) — per-body scaling about the root, local-frame pos/rot offsets, damped weighted Gauss-Newton steps with exact MuJoCo body Jacobians (mj_jacBody), mj_integratePos for the floating base, joint range clamping, per-stage convergence loop (1 + up to 10 iterations, stop when error improvement < 1e-3 — same defaults as GMR).
• Algorithm (OmniRetarget): the same solver plus an interaction-mesh objective (src/lib/retarget/omniEngine.ts). Each frame builds a k-nearest- neighbour graph over the (scaled) human keypoints and adds a soft Gauss-Newton term driving the robot's Laplacian coordinates δ_i = p_i − Σ_j w_ij p_j (translation-invariant) onto the human's, preserving relative limb structure. Tunable meshWeight / meshNeighbors; meshWeight = 0 reduces exactly to GMR.
• BVH pipeline: LAFAN1-convention loader (matches GMR's lafan_vendor): euler→quat per channel order, FK, Y-up→Z-up, cm→m, virtual LeftFootMod/RightFootMod joints.
• Video → BVH (experimental): in-browser MediaPipe Pose (@mediapipe/tasks-vision, Apache-2.0, lazy-loaded) estimates 33 3D landmarks/frame; a position→rotation solver (src/lib/mocap/) maps them onto the fixed LAFAN1 skeleton — only bone directions are used (lengths stay at the rest offsets), with One-Euro smoothing — emitting a BVH that flows through the same loader and retargeting. Draft quality; the video never leaves the browser. 浏览器内单目动捕：MediaPipe 关键点 → 旋转求解到 LAFAN1 骨架，复用同一管线。
• Rendering: three.js (Z-up), robot meshes built from the compiled MuJoCo model, checkerboard floor + soft shadows for contact/foot-skate inspection (BVHView-inspired).
• Export: NPZ with GMR-compatible keys (fps, root_pos, root_rot (xyzw), dof_pos); scripts/npz_to_pkl.py converts to the exact GMR pickle format. CSV/JSON also available.

Validation · 数值验证

The web engine is validated against the original Python GMR (tests/parity.test.ts + scripts/gmr_reference.py): on the bundled walk sample retargeted to Unitree G1, frame-0 preprocessed IK targets match GMR to ~1e-9, and the full-motion output agrees within 0.019 rad DoF RMSE / 0.27 cm root RMSE (differences stem from the damped Gauss–Newton step vs. mink's daqp QP). Throughput in-browser is ~700–800 frames/s vs ~34 frames/s for the Python pipeline on the same machine.

# regenerate the reference and run the parity test locally
python3 scripts/gmr_reference.py public/sample_motions/walk.bvh unitree_g1 /tmp/gmr_ref_walk_g1.json
npx vitest run tests/parity.test.ts

Develop

npm install
npm run dev        # http://localhost:3000
npm test           # unit + engine integration tests (runs MuJoCo WASM in Node)
npm run typecheck
npm run build      # production build to dist/
node scripts/e2e_smoke.mjs   # headless full-pipeline smoke test (needs chromium)
node scripts/record_site_demo.mjs   # re-record media/site-demo.gif (needs ffmpeg, gifsicle optional)

Sample motions

public/sample_motions/*.bvh are **trimmed clips from the Ubisoft LAFAN1 dataset (non-commercial license): walk, run, dance, fall & get up, jumps (60 s each). Regenerate from a local LAFAN1 zip via python3 scripts/prepare_lafan_samples.py.

The legacy procedural generator (scripts/gen_sample_bvh.py) is kept for reference only. For the full dataset, download LAFAN1 and drop any .bvh into the app.

public/sample_motions/dog_*.bvh (walk, run, idle, ~6–7 s each at 30 fps) are trimmed Labrador retriever mocap clips from Tencent's Lifelike Agility and Play dataset (MIT) for the quadruped workflow below; regenerate via python3 scripts/prepare_dog_samples.py.

Robots from the Gallery · 从 Gallery 挑选机器人

Retarget Config → 从 Gallery 选择 URDF / Pick URDF from Gallery lists the humanoid and quadruped URDFs of Robot Description Gallery Online (its data/robots.json + data/visibility.md, read live from the gallery site; versions such as G1's 21 URDFs are selectable). Only the category matching the loaded motion can be picked (humanoid ↔ LAFAN1 BVH, quadruped ↔ dog mocap). 在 Gallery 中查看 / View in Gallery opens the selected robot — gallery or built-in — on the gallery's own viewer page.

在重定向设置页点「从 Gallery 选择 URDF」，即可从 Robot Description Gallery 的人形 / 四足 URDF 中挑选机器人（含多版本选择）；「在 Gallery 中查看」跳转到该机器人在 Gallery 网站上的页面。

• The URDF and its meshes stream from the pinned upstream commit (jsDelivr / GitHub LFS), exactly as the gallery loads them, and never leave the browser. Collada (.dae) and ASCII-STL meshes are re-encoded to binary STL in the browser (MuJoCo reads only STL / OBJ / MSH). 模型与网格从上游仓库下载，仅在浏览器内处理。
• src/lib/gallery/urdfBundle.ts normalises what MuJoCo's URDF importer rejects (a world root link, several <material>s per visual, singular inertias), drops collision geometry (not drawn, irrelevant to IK) and keeps fixed-joint links such as a quadruped's FL_foot as bodies.
• The starting ik_config is a bundled GMR preset when every body it names exists on the robot (the gallery's G1 / H1-2 / Go2 / A1 / PM01 / Fourier robots match), otherwise a name-based template — root + four feet for quadrupeds (FL_foot, LF_FOOT, toe_fl, foot_front_left …). Review the mapping tables before trusting the result.
• Out of scope: MJCF-only gallery cards and xacro-only upstreams; SMPL-X motion.
• node scripts/e2e_gallery.mjs --url http://localhost:3000 drives the flow (G1 → walk, Go2 → dog walk) in headless Chromium; it needs network access.

Quadruped retargeting · 四足机器人重定向

Beyond humanoids, the same keypoint differential-IK engine retargets animal mocap onto quadruped robots. This follows the most mature open-source quadruped retargeting work — Peng et al.'s motion_imitation ("Learning Agile Robotic Locomotion Skills by Imitating Animals", RSS 2020): pick a small set of source keypoints on the animal (root + four feet) and the corresponding target keypoints on the robot, then solve IK. GMR's two-stage solver already does exactly this, so quadruped support is a config + asset drop-in rather than a new algorithm. Related references surveyed: Tencent Robotics X "Lifelike Agility and Play" (Nature MI 2024, real Labrador BVH) and pan-motion-retargeting (TVCG 2023).

复用同一套关键点微分 IK 引擎，将动物动捕重定向到四足机器人。方法对齐当前最成熟的开源四足 重定向工作（motion_imitation，"模仿动物"）：在动物身上选取少量关键点（躯干 + 4 只脚）与机器人 对应点，求解 IK。GMR 两阶段求解器本就如此，因此四足支持只需新增配置与资产。

Source motion	Real Labrador retriever mocap (.bvh) from Tencent's Lifelike Agility dataset (MIT), trimmed/downsampled to 30 fps: dog_walk, dog_run, dog_idle. 61-joint quadruped skeleton — flows through the same BVH loader (generic FK; human foot augmentation auto-skips).
Robots	Unitree Go2 and Unitree A1 (12 DoF), real MJCF + meshes from MuJoCo Menagerie (Unitree BSD-3). A *_mocap.xml variant adds four massless *_foot keypoint bodies at the foot tips (no added DoF). Canonical URDFs are also included under public/robots/<id>/urdf/ as reference.
Mapping	bvh_quadruped_to_unitree_{go2,a1}.json: base/trunk ← b_Hips (orientation + position) and FL/FR/RL/RR_foot ← b_{Left,Right}Hand / b_{Left,Right}Toe (position). A constant trunk-frame offset (clip-independent) and a 0.5 body scale fit the Labrador onto the smaller robots.

Natural quadruped motion · 四足自然化（默认开启）

Plain per-frame keypoint IK copies everything wrong with raw animal mocap onto the robot: the dog's pelvis bone jitters at ~150°/s even when it stands still, its paws dip up to 5 cm below the floor, and planted paws slide. With Retarget Config → Natural quadruped (on by default for dog clips) the whole clip is planned before solving (src/lib/retarget/quadruped.ts), following the quadruped retargeting notes in Robotics_Notebooks:

1. Trunk frame from keypoints — Peng et al.'s motion_imitation (retarget_root_pose): base at the centre of the four leg roots, heading from hips to shoulders, roll removed and the clip's mean pitch levelled (their per-robot FORWARD_DIR_OFFSET, made automatic); keypoints low-passed (zero-phase).
2. Hip-relative feet — same source (retarget_pose): each foot target is the robot's own hip plus the dog's hip → paw offset, so the dog's longer trunk does not drag the legs out of their natural stance; each stride is centred under its hip (their hand-tuned SIM_TOE_OFFSET_LOCAL, made automatic), and front paws use the paw joint, not the wrist.
3. Contact phases + foot locking — the spatial stage of STMR (T-RO 2025): a paw that is low and slow is planted; a planted foot is held still on the floor (per-paw floor level, no penetration), the correction is eased over the swing, and a swinging foot keeps a minimum arc above the floor instead of scuffing.
4. Regularized IK — a gentle pull toward the robot's rest keyframe (keeps knees on their natural side) and toward the previous frame (no jumps).

Measured on the robot's own kinematics (src/lib/retarget/motionQuality.ts, metrics after X-Morph's reference-quality table; tests/quadruped.test.ts), plain IK → natural, Go2 (A1 is within a few %):

clip	foot slip near floor (cm/s)	penetration p95 (cm)	joint jerk RMS	frames with a >0.25 rad joint jump	trunk roll/pitch wobble (°/s)
walk	70.8 → 16.0	3.26 → 0.20	4874 → 805	37 → 1	117 → 42
run	87.9 → 32.8	4.94 → 0.15	4095 → 846	49 → 2	106 → 45
idle	30.1 → 8.8	2.62 → 0.06	3169 → 484	15 → 0	102 → 58

关闭该开关即回到按映射表逐帧 IK 的旧行为；ik_config 与 NPZ 格式不变。

Usage · 使用方式

The quadruped workflow is on by default: in BVH Viewer → Load sample the Dog walk / run / idle (Quadruped) clips sit in their own group, and Unitree Go2 / A1 appear in the Retarget Config robot dropdown. Loading a dog clip auto-selects Go2 and greys out the humanoid robots (and loading a LAFAN1 clip switches back to a humanoid and greys out Go2 / A1) — the motion family must match the robot's ik_config convention. Then run retargeting and export as usual (NPZ dof_pos has 12 columns in the robot's MJCF joint order — see dof_names in the CSV / JSON export).

四足流程默认开启：BVH 预览 → 加载示例动作 中有独立的「四足动物」分组（Dog walk / run / idle）， 重定向设置 的机器人下拉里有 Unitree Go2 / A1。加载狗动捕后自动切到 Go2 并灰掉人形机器人 （加载 LAFAN1 人形动作则切回人形并灰掉 Go2 / A1）——动作类型必须与机器人 ik_config 的约定一致。 之后照常重定向、导出（NPZ 的 dof_pos 为 12 列，顺序同机器人 MJCF 的关节顺序，见 CSV / JSON 导出中的 dof_names）。

To hide the quadruped entry points for one visit (e.g. a humanoid-only demo), open the site with ?quadruped=0 — the hash-router form …/#/bvh?quadruped=0 also works. The flag is read from the URL on each page load only and never persisted; the site-wide default is DEFAULT_QUADRUPED in src/lib/features.ts.

如需在某次访问中隐藏四足入口（例如只演示人形），在 URL 中带 ?quadruped=0（hash 后的 …/#/bvh?quadruped=0 亦可）。该开关每次加载只读 URL、不写入 localStorage； 站点默认值见 src/lib/features.ts 的 DEFAULT_QUADRUPED。

Validated headless (tests/quadruped.test.ts, walk / run / idle × Go2 / A1): with plain IK the robot stands at ≈0.28–0.29 m, the four feet track their targets to ~0.7–2.2 cm on average (worst single frame ≈7–13 cm) and the trunk stays within ~8–12° of the dog's (faithful pitch); the natural-motion planner is checked against it on every clip (table above). scripts/e2e_quadruped.mjs drives the same flow in a headless browser (sample menu → auto robot switch → retarget → NPZ export, both robots, all three clips). Reproduce:

# trimmed dog clips from a local checkout of the Lifelike-Agility raw_mocap_data
python3 scripts/prepare_dog_samples.py --raw /path/to/raw_mocap_data
# regenerate the foot-keypoint robot variants from a Menagerie checkout
python3 scripts/add_quadruped_foot_sites.py go2.xml go2_mocap.xml 0.213
npx vitest run tests/quadruped.test.ts
npm run build && node scripts/e2e_quadruped.mjs   # headless browser flow (needs chromium)
node scripts/record_quadruped.mjs   # narrated demo video of the flow (needs system ffmpeg)

Deploy

Pushes to main build and deploy via GitHub Actions (.github/workflows/deploy.yml). One-time setup: repository Settings → Pages → Source: GitHub Actions.

Roadmap

Full plan of record in ROADMAP.md. Headlines:

• Phase 1 — GMR online (BVH/LAFAN1 → Unitree G1, Booster T1), parity-validated
• Phase 2 — pluggable engines: OmniRetarget (in-browser interaction-mesh) selectable alongside GMR
• Video → BVH input (experimental, in-browser MediaPipe pose capture)
• Quadruped retargeting — dog mocap (BVH) → Unitree Go2 / A1 via keypoint IK (motion_imitation-style)
• Robots from Robot Description Gallery Online — pick any humanoid / quadruped URDF in the browser, with a link back to the gallery
• More GMR robots (assets are drop-in: public/robots/ + manifest entry)
• holosoma as a third, trajectory-level engine (heavier; backend TBD)
• SMPL-X / AMASS input (user-supplied body model .npz, in-browser) — unlocks 10 more GMR robots (H1, H1-2, Kuavo, …); GMR parity pending

Credits & licenses

• GMR (MIT) — retargeting algorithm, ik_configs and robot model assets (public/robots/, original sources: Unitree, Booster Robotics).
• motion_imitation (Apache-2.0) — "Learning Agile Robotic Locomotion Skills by Imitating Animals"; the root + four-feet keypoint-IK scheme adopted for quadruped retargeting.
• Lifelike Agility and Play (Tencent Robotics X, MIT) — source of the bundled Labrador retriever dog mocap (public/sample_motions/dog_*.bvh, trimmed/downsampled).
• MuJoCo Menagerie — Unitree Go2 / A1 MJCF and meshes (Unitree BSD-3, see each robot's LICENSE); canonical URDFs from unitree_ros.
• OmniRetarget — interaction-mesh / interaction-preserving retargeting idea adapted for the in-browser OmniRetarget engine. The interaction-mesh / Laplacian formulation follows Ho, Komura & Tai, "Spatial Relationship Preserving Character Motion Adaptation" (SIGGRAPH 2010).
• MuJoCo (Apache-2.0) — physics & WASM bindings (mujoco-js).
• MediaPipe Tasks Vision (Apache-2.0) — in-browser pose estimation for the experimental video → BVH feature.
• Robot Description Gallery Online (MIT) — robot registry for the gallery picker; each model keeps its upstream licence (shown in the picker), some of them non-commercial.
• BVHView — UI/rendering inspiration.
• Site layout inspired by RL_Sim2Sim_Demo_Website.

Code in this repository is released under the MIT License (see LICENSE).