docs: Update RL entrypoint examples

[AntoineRichard]

Description

Updates the documentation to use the unified reinforcement learning train/play entrypoints with --rl_library instead of deprecated per-library script paths. Also documents this migration in the Isaac Lab 3.0 migration guide and updates related wording from scripts to commands/workflows where appropriate.

No new dependencies are required.

Issue: N/A

Type of change

• Documentation update

Screenshots

Not applicable.

Checklist

• I have read and understood the contribution guidelines
• I have run the pre-commit checks with ./isaaclab.sh --format
  • Full ./isaaclab.sh -f was attempted, but check-git-lfs-pointers failed because git-lfs is not installed in this environment.
  • SKIP=check-git-lfs-pointers ./isaaclab.sh -f passed all remaining hooks.
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
  • Not applicable: documentation-only change; no tests added.
• I have added a changelog fragment under source/<pkg>/changelog.d/ for every touched package (do not edit CHANGELOG.rst or bump extension.toml — CI handles that)
  • Not applicable: documentation-only change; no package was touched.
• I have added my name to the CONTRIBUTORS.md or my name already exists there

[isaaclab-review-bot]

🤖 Isaac Lab Review Bot

Summary

This PR updates all documentation examples to use the unified reinforcement learning CLI entrypoints (./isaaclab.sh train --rl_library <library> and ./isaaclab.sh play --rl_library <library>) instead of the deprecated per-library script paths (python scripts/reinforcement_learning/<library>/train.py). It also adds a new migration guide section documenting this change for Isaac Lab 3.0, renames references from "scripts" to "commands/workflows/entrypoints" where appropriate, and updates literalinclude references from train.py to train_sb3.py.

Findings

The changes are consistent and well-executed across all 33 files. Specific observations:

1. Consistency ✅ — All examples consistently use the new ./isaaclab.sh train --rl_library <lib> / ./isaaclab.sh play --rl_library <lib> pattern. Windows examples correctly use isaaclab.bat with the same syntax.

2. Multi-GPU/distributed examples ✅ — Correctly updated to use scripts/reinforcement_learning/train.py --rl_library <lib> (the unified script path) when launching via torch.distributed.run or skrl.utils.distributed.jax, since those need a direct Python script rather than the shell wrapper.

3. Migration guide addition ✅ — The new section in migrating_to_isaaclab_3-0.rst clearly explains the old vs new syntax with before/after examples, lists supported libraries, and documents the distributed launcher pattern.

4. Ray hyperparameter tuning ✅ — Correctly removes the now-unnecessary --workflow argument from the ray tuning example.

5. literalinclude references — Updated from train.py to train_sb3.py in configuring_rl_training.rst and run_rl_training.rst. The emphasize-lines directives were removed, which is fine since line numbers may have shifted with the refactor.

6. Minor wording updates ✅ — "scripts" → "commands"/"workflows"/"entrypoints" is applied consistently throughout.

No broken references, typos, or syntax issues found.

Verdict

No issues found — Clean, comprehensive documentation migration to the unified RL entrypoints.

---

Update (8202f07): Reviewed incremental changes. This commit:

1. Restores emphasize-lines directives in configuring_rl_training.rst and run_rl_training.rst — these highlight the key SB3 training code sections for readers. Line ranges look correct for the updated train_sb3.py file. ✅

2. Re-adds --workflow to the Ray tuning example in ray.rst with a new explanatory paragraph clarifying that Ray launches training via its --workflow script path. The note about using the unified entrypoint for custom configs is a helpful addition. ✅

These changes address the missing highlight ranges noted in the previous review and correct the Ray tuner documentation. No new issues found.

[greptile-apps]

Greptile Summary

This PR updates documentation across 33 files to replace deprecated per-library RL script invocations (./isaaclab.sh -p scripts/reinforcement_learning/<lib>/train.py) with the new unified Isaac Lab 3.0 CLI entrypoints (./isaaclab.sh train --rl_library <lib>), and adds a dedicated migration guide section explaining the change.

• All five supported libraries (rsl_rl, rl_games, skrl, sb3, rlinf) are updated consistently across tutorials, how-tos, feature docs, migration guides, and policy deployment docs; Windows .bat equivalents are also updated.
• Tutorial literalinclude dropdowns pointing to sb3/train.py are migrated to the new canonical implementation file sb3/train_sb3.py; distributed multi-GPU examples correctly use scripts/reinforcement_learning/train.py --rl_library <lib> since torch.distributed.run requires a direct Python script path.
• The Ray hyperparameter tuner example silently drops --workflow without providing migration guidance; tuner.py still defaults to the old per-library path and its --workflow flag only accepts a bare script path (no embedded --rl_library argument), so readers adopting the new entrypoints have no documented path for the Ray workflow.

Confidence Score: 4/5

Documentation-only change that is safe to merge; all referenced script paths and files exist in the repository and the functional behavior of every updated command is correct.

The bulk of the changes are straightforward find-and-replace migrations that are accurate and internally consistent. The one area that warrants attention is the Ray tuner section, where the --workflow argument was dropped without documenting how users can adopt the new unified entrypoint through the tuner — the underlying tuner.py still points to the old per-library path by default. The tutorial emphasize-lines removal is also a minor quality regression. Neither issue blocks the PR from being merged, but both leave readers with less guidance than before.

docs/source/features/ray.rst (incomplete migration story for the Ray tuner workflow argument) and docs/source/tutorials/03_envs/run_rl_training.rst (tutorial code highlights were removed without replacement)

Important Files Changed

Filename	Overview
docs/source/features/ray.rst	Removes the --workflow argument from the Ray tuner example without documenting how the Ray integration should adopt the new unified entrypoint
docs/source/migration/migrating_to_isaaclab_3-0.rst	Adds a new migration section documenting the unified RL CLI entrypoints and the path for distributed launcher scenarios; well-structured and accurate
docs/source/tutorials/03_envs/run_rl_training.rst	Updates literalinclude to train_sb3.py and CLI commands to new entrypoint; removes emphasize-lines that previously highlighted key code sections for tutorial readers
docs/source/tutorials/03_envs/configuring_rl_training.rst	Switches literalinclude to train_sb3.py and updates CLI commands; removes emphasize-lines and updates the argument explanation for --rl_library vs --agent
docs/source/features/multi_gpu.rst	Correctly updates distributed commands to use scripts/reinforcement_learning/train.py --rl_library instead of per-library paths; mixed style vs isaaclab.sh train is appropriate for torch.distributed.run context
docs/source/overview/reinforcement-learning/rl_existing_scripts.rst	Comprehensive update across all RL libraries (rl_games, rsl_rl, skrl, sb3, rlinf) to unified entrypoint syntax; file title updated from Scripts to Workflows
docs/source/features/population_based_training.rst	Correct update from ./isaaclab.sh -p to ./isaaclab.sh train --rl_library for the PBT worker launch command
docs/source/setup/quickstart.rst	Updates quickstart training commands and refreshes introductory prose; changes are consistent and correct

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[User runs training] --> B{Entrypoint style}
    B -->|Isaac Lab 3.0 new| C["./isaaclab.sh train --rl_library <lib>"]
    B -->|Deprecated compat| D["./isaaclab.sh -p scripts/rl/<lib>/train.py"]
    B -->|Distributed launcher| E["python -m torch.distributed.run scripts/rl/train.py --rl_library <lib>"]
    B -->|Ray tuner| F["./isaaclab.sh -p scripts/rl/ray/tuner.py\n(default workflow = rl_games/train.py)"]

    C --> G[scripts/reinforcement_learning/train.py]
    D --> H["scripts/rl/<lib>/train.py\n(emits DeprecationWarning, still functional)"]
    E --> G
    F --> H

    G --> I[Selects library via --rl_library]
    I --> J["scripts/rl/<lib>/train_<lib>.py"]
    H --> J

    style F fill:#ffe0b2,stroke:#e65100
    style H fill:#fff9c4,stroke:#f9a825

Loading

Comments Outside Diff (1)

1. docs/source/features/ray.rst, line 130-141 (link)

   Ray tuner --workflow removed without a migration replacement

   The PR drops --workflow scripts/reinforcement_learning/rl_games/train.py from this example, but tuner.py still hardcodes WORKFLOW = "scripts/reinforcement_learning/rl_games/train.py" as its default and its --workflow CLI argument only accepts a bare script path (no embedded arguments). This means the Ray tuner cannot yet specify the new unified entrypoint (scripts/reinforcement_learning/train.py --rl_library <lib>) through the --workflow flag, so the integration quietly stays on the deprecated per-library path. A reader following the migration guide will have no guidance on how to update their Ray tuner workflows to the new style.

_{Reviews (1): Last reviewed commit: "docs: Update RL entrypoint examples" | Re-trigger Greptile}

[greptile-apps]

Tutorial code highlights dropped without replacement

The old literalinclude of train.py had emphasize-lines: 57, 66, 68-70, 81, 90-98, 100, 105-113, 115-116, 121-126, 133-136 that directed tutorial readers to the wrapper creation, environment setup, and agent training sections. train_sb3.py is a restructured file with different line numbers, so the old values don't apply — but leaving no emphasize-lines at all means the "The Code Explained" section no longer has visual anchors pointing readers to the relevant code blocks. New emphasis lines matching train_sb3.py's structure would preserve the tutorial's learning value.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[AntoineRichard]

Restored the SB3 tutorial highlight ranges against train_sb3.py in 8202f071ea1.

[isaaclab-review-bot]

Perfect, thanks for restoring the highlights! The new ranges (97-100, 104-109, etc.) look like they now correctly point to the wrapper creation and training sections in train_sb3.py. 👍

[AntoineRichard]

Addressed the Greptile feedback in 8202f071ea1: restored the SB3 tutorial :emphasize-lines: ranges and clarified the Ray tuner workflow behavior. The Ray example now stays on the executable RL-Games workflow path for the built-in configs and documents how custom Ray configs can use the unified train.py entrypoint by passing --rl_library through runner_args.