Merge SeFlow++ and ZOD extraction from HiMo Project

[Kin-Zhang]

This pull request introduces several improvements and updates to the OpenSceneFlow codebase, primarily focusing on new methods, documentation, Docker and environment setup, and training workflows.

Documentation and Training Workflow Updates from HiMo project:

• Major reorganization of the README.md to clearly separate supervised and self-supervised training instructions, update dataset support (added ZOD), and streamline pretrained weight download links and training commands. Also, the changelog section was removed for clarity. [1] [2] [3] [4] [5] [6] [7] [8]
• Updating 1) a new three-frame backbone; 2) seflowppLoss; 3) seflowpp_auto from HiMo (SeFlow++) part. Major reorganization of feed-forward self-supervised methods. Now we comment the ssl-loss config by default, it needs to add `+' with new parameters. The ssl_label can be changed for easy future exploration, etc.
• Training commands now consistently use the optimizer.lr parameter and improved loss function specification for all models. [1] [2] [3]

Environment and Installation:

• Dockerfile updated to install pytorch3d/torch-scatter via conda/pip to support running all methods inside Docker.
• Revert the environment to python=3.8, torch=2.0.0 and cuda=11.7 check more discussion for upgrading-issue in discussion CUDA utils #11

---

Old Message:

HiMo got accepted into T-RO now! 🔥 I will try my best to update all detailed codes so that everyone can reproduce all the tables/results/figures in the paper.
All Scene Flow-related parts will be merged into OpenSceneFlow as this pull request shows, and then we will update another repo for the HiMo task and downstream task codes.

• update seflow++ scripts.
• update av2 extraction code. lidar_id and lidar_dt etc.
• update seflow++ weights, seflow repo readme also update.
• update opensf demo data.
• update all baselines (NSFP, FastNSF, ICP-Flow), check these on Merge ICP-Flow into codebase #15 and Merge two optimization-based (NSFP, FastNSF) methods into codebase from HiMo project #14
• update ZOD data extraction.

The self-supervised learning SSL part has a big refactor. We added auto_label involved in cfg, etc, for afterward potential seflowpp hyper-paramters tuning by users, etc.

[Copilot]

Pull Request Overview

This PR merges SeFlow++ and ZOD extraction code from the HiMo project into OpenSceneFlow. It represents a major update to support self-supervised learning with automated labeling and extends the framework to handle three-frame processing for improved scene flow estimation.

• Updates training infrastructure to support new self-supervised loss functions (seflowLoss, seflowppLoss) with automated labeling
• Refactors model architectures to support three-frame processing and adds new DeFlowPP model
• Adds ZOD dataset extraction capabilities and updates AV2 extraction with additional metadata

Reviewed Changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
train.py	Updates training configuration validation and dataset initialization for SSL workflows
src/trainer.py	Extends training step to handle three-frame data and new loss functions
src/models/	Refactors models to use centralized point cloud processing and adds DeFlowPP
src/lossfuncs/	Restructures loss functions into separate modules for supervised and self-supervised learning
src/dataset.py	Updates dataset loading to support automated labeling and three-frame sequences
dataprocess/	Adds ZOD extraction and enhances AV2 extraction with lidar metadata
process.py	Major refactor to support multiple labeling strategies (DUFO, clustering, NND)

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[Copilot]

The wandb.finish() call is inside the conditional block but not properly indented. This will cause a syntax error as it appears to be at the same indentation level as the if statement.

[Kin-Zhang]

didn't get this point.... check later.

[Copilot]

[nitpick] Using torch.nanmean instead of torch.mean for consistency with seflowLoss function. However, this creates an inconsistency between the two loss functions where seflowLoss uses torch.mean. Consider standardizing the approach.

Suggested change

	chamfer_dis = torch.nanmean(est_dist0[est_dist0 <= TRUNCATED_DIST]) + torch.nanmean(est_dist1[est_dist1 <= TRUNCATED_DIST])
	chamfer_dis = torch.mean(est_dist0[est_dist0 <= TRUNCATED_DIST]) + torch.mean(est_dist1[est_dist1 <= TRUNCATED_DIST])

[Kin-Zhang]

SeFlow++ ckpt for av2 leaderboard (local results after fixed gt #5 ):

---

Some notes here:
SeFlow++ introduced by HiMo has three changes compared to the original SeFlow: a) a three-frame 'deflowpp' backbone; b) refine auto-label seflowpp_auto; c) new symmetric loss function seflowppLoss.

We observed that for Argoverse 2 data, SeFlow++ benefits from a) most and c). For a highway with less observed free-space on the DUFO label, b) can help. However, for low-speed environments like urban environments, it might break the cluster consistency and introduce noise, etc. Feel free to explore more. In the main READM, we present the one we trained on the paper AV2 results.

Here is for training seflowpp backbone (deflowpp) with seflow loss and auto-label etc:

# (for av2) if you don't want to auto-label again etc, seflow++ could use seflow loss on seflow++ three-frame backbone, their av2 performance are simialr.
python train.py model=deflowpp save_top_model=3 val_every=3 voxel_size="[0.2, 0.2, 6]" point_cloud_range="[-51.2, -51.2, -3, 51.2, 51.2, 3]" num_workers=16 epochs=9 optimizer.lr=1e-4 "+add_seloss={chamfer_dis: 1.0, static_flow_loss: 1.0, dynamic_chamfer_dis: 1.0, cluster_based_pc0pc1: 1.0}" +ssl_label=seflow_auto loss_fn=seflowLoss num_frames=3 batch_size=8