Skip to content

Integrate planning methods#142

Open
VisionChengzhuo wants to merge 6 commits into
OpenDCAI:mainfrom
VisionChengzhuo:main
Open

Integrate planning methods#142
VisionChengzhuo wants to merge 6 commits into
OpenDCAI:mainfrom
VisionChengzhuo:main

Conversation

@VisionChengzhuo

Copy link
Copy Markdown
Contributor

Summary

  • Integrate all current docs/planning.md Methods Being Integrated entries: Cosmos3, Hunyuan-worldplay2/HY-World-2.0, Solaris, GammaWorld, Ctrl-World, FantasyWorld, and MemFlow.
  • Add setup scripts, pipeline/operator/synthesis wrappers, mapping entries, inference test hooks, and installation docs for the new integrations.
  • Use external runtime wrappers that execute the upstream repositories and require real output artifacts.

Verified real inference

  • Cosmos3 original and OpenWorldLib integrated text-to-video output.
  • Ctrl-World original and OpenWorldLib integrated rollout video output.
  • MemFlow original and OpenWorldLib integrated video output.
  • Solaris original and OpenWorldLib integrated video output.
  • GammaWorld original and OpenWorldLib integrated 189-frame navigation video output.
  • Hunyuan-worldplay2 original and OpenWorldLib integrated gaussians/points PLY output.
  • FantasyWorld original and OpenWorldLib integrated video plus point-cloud PLY output.

Static checks

  • python -m py_compile on new/changed integration Python files.
  • bash -n on new setup and inference shell scripts.
  • git diff --check.

@VisionChengzhuo VisionChengzhuo force-pushed the main branch 2 times, most recently from 605451b to 19089ca Compare June 30, 2026 03:06
@zengbohan0217

Copy link
Copy Markdown
Collaborator

有些需要调整一下:

  1. data/test_case/hunyuan_worldplay2/teaser.png 这个应该是完全没有必要的,而且占用1MB
  2. data/test_case/fantasy_world/examples/ 这个路径下的数据看看能不能使用已有测例中的,比如 data/test_case/test_image_seq_case1,然后这个camera json有点太长了
  3. src/openworldlib/operators/ 路径下的新加的几个文件格式好像不太对,尽量参考一下 src/openworldlib/operators/yume_1p5_operator.py
  4. 同理 src/openworldlib/pipelines/ 下面新加的几个方法好想和已有的格式也不太一致,参考一下 src/openworldlib/pipelines/yume/pipeline_yume_1p5.py
  5. src/openworldlib/synthesis/ 下面新加的函数命名好像也有问题,可以参考一下 src/openworldlib/synthesis/visual_generation/cosmos/cosmos_predict2p5_synthesis.py
  6. src/openworldlib/synthesis/visual_generation/fantasy_world/FantasyWorld/diffsynth_wan21/ 、src/openworldlib/synthesis/visual_generation/fantasy_world/FantasyWorld/diffsynth_wan22/ 、 src/openworldlib/synthesis/visual_generation/fantasy_world/FantasyWorld/vggt/ 、 src/openworldlib/synthesis/visual_generation/fantasy_world/FantasyWorld/wan/ 等内容,在 src/openworldlib/base_models/diffusion_model/diffsynth/ 、src/openworldlib/base_models/diffusion_model/video/wan_2p1 、src/openworldlib/base_models/diffusion_model/video/wan_2p2 、src/openworldlib/base_models/three_dimensions/general_3d/eastern_journalist/utils3d、src/openworldlib/base_models/three_dimensions/point_clouds/vggt/vggt 等地方都有过,希望不要重复造轮子
  7. src/openworldlib/synthesis/visual_generation/gamma_world/data/buildHouse_flat/first_frame.png、src/openworldlib/synthesis/visual_generation/gamma_world/data/buildTower_normal/first_frame.png 这些数据,尽量不要这样提交,而且太大了,要放在 data/test_case/ 路径下
  8. 参考 6 7,其他的几个pipeline也是,重复的轮子尽量复用 src/openworldlib/base_models 已经有的,没有的可以在这里进行补充,数据也尽量使用 data/test_case/ 路径下的,如果实在需要提供新的数据,降低size,往 data/test_case/ 路径下进行提交,但是文件size一定要降低。另外就是一定不要吧图片之类的文件往 src/openworldlib/下面提交

@zengbohan0217

Copy link
Copy Markdown
Collaborator

改的很好,还有一些需要小小的微调一下:

  1. data/test_case/gamma_world/buildTower_normal/first_frame.jpg 以及 data/test_case/gamma_world/buildHouse_flat/first_frame.jpg 与 data/test_case/test_image_seq_case1/image_0001.jpg 以及 data/test_case/test_image_seq_case1/image_0002.jpg 好像是重复的,这个就别重复上传了,修改一下调用路径
  2. src/openworldlib/synthesis/visual_generation/hunyuan_world/hunyuan_worldplay2/hyworld2/worldgen/third_party/下的 gsplat_maskgaussian 以及 navmesh 应该考虑放在 submodules/ 路径下,这里是第三方安装包,另外类似于glm这种比较大的repo,尽量在 scripts/setup/ 下进行额外的下载
  3. src/openworldlib/synthesis/visual_generation/memflow/wan/ 这个路径也考虑往 src/openworldlib/base_models/diffusion_model/ 路径下迁移

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants