Skip to content

[python] Persist LeRobot dataset metadata in Paimon - #9529

Draft
XiaoHongbo-Hope wants to merge 1 commit into
apache:masterfrom
XiaoHongbo-Hope:codex/lerobot-self-contained-import
Draft

[python] Persist LeRobot dataset metadata in Paimon#9529
XiaoHongbo-Hope wants to merge 1 commit into
apache:masterfrom
XiaoHongbo-Hope:codex/lerobot-self-contained-import

Conversation

@XiaoHongbo-Hope

@XiaoHongbo-Hope XiaoHongbo-Hope commented Sep 1, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

  • make load_from_lerobot targets self-contained with frame, dataset-manifest, Episode, and Task tables
  • bind all components with dataset_id and a unique UUID metadata_version
  • append one READY manifest only after component snapshots are committed and tagged
  • preserve LeRobot v3 info, features, statistics, Episode boundaries, splits, Tasks, and extra metadata
  • validate frame control columns against the metadata being published
  • cascade managed companion cleanup across rename and drop operations

This establishes the persisted import contract used by PaimonLeRobotDataset in #9498.

Tests

  • LeRobot 0.4.4 integration: 38 passed
  • LeRobot + HDF5 + multimodal regression: 116 passed, 22 skipped
  • flake8, py_compile, and git diff --check pass
@XiaoHongbo-Hope
XiaoHongbo-Hope force-pushed the codex/lerobot-self-contained-import branch 3 times, most recently from 18b33a0 to dd6ac5b Compare September 1, 2026 12:29
@XiaoHongbo-Hope
XiaoHongbo-Hope force-pushed the codex/lerobot-self-contained-import branch from dd6ac5b to 445d4a9 Compare September 1, 2026 12:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant

@XiaoHongbo-Hope