Support Dpsk4.1 SFT and RL - #10179
Open
tastelikefeet wants to merge 9 commits into
Open
tastelikefeet wants to merge 9 commits into
tastelikefeet wants to merge 9 commits into
Conversation
- Register deepseek_v41 model type (LLM + MLLM) and model arch (vision/aligner) - Add V4.1 config classes (text/vision composite) and vLLM compat patches (hc_head init, deep_gemm bundled swap, bf16 o_proj fallback, weight skip) - Add template support for V4.1 (multimodal) - Megatron: engram freezing for on-policy GRPO, RL weight-sync skips unsupported engram export - Add GRPO colocate examples (full + lora) and V4.1 template align test - Add FSDP sharded LoRA conversion util
LoRA SFT and colocate GRPO starting points for DeepSeek-V4.1-Flash. Both record the parallelism support boundary (CP/PP/EP/DP/VPP work, training TP and SP do not), why --bf16 must stay, and that vLLM TP for rollout is unrelated to the unsupported training TP. The GRPO script also notes that LoRA weight sync ships adapters only, so the Engram tables stay resident in the rollout engine.
…rsions in the examples DeepSeek-V4.1 does not run on any released megatron-core wheel: it needs the hybrid stack, Engram and CSA2 code, so both example scripts now spell out the source checkout that was validated -- the dev branch with NVIDIA/Megatron-LM PR modelscope#7224 merged in -- and say to expose it through PYTHONPATH so it shadows an installed megatron-core. Same for the two Python packages: ms-swift and mcore-bridge both need source installs, because v4.1 support landed after their latest releases and requirements/megatron.txt still pins mcore-bridge >= 1.6.3, which predates it.
…ted TP/SP, document CP recipe) The two colocate GRPO examples set --tensor_model_parallel_size 8 and --sequence_parallel true (to satisfy AbsorbedMLA's "SP required when TP>1" guard), but TP/SP are not a validated path for DeepSeek-V4.1 (TP is design-doc only; SP depends on TP). This contradicts the support matrix already documented in the models/ SFT+GRPO examples. Set TP=1 and drop --sequence_parallel so all four examples agree with the validated CP/PP/EP/DP/VPP support set. EP=8/PP=4 now matches examples/models/deepseek_v41/megatron_grpo.sh. Also document the non-obvious context-parallel recipe in all four examples: a naive --context_parallel_size>1 fails because DSv4's hybrid attention (CSA2/Engram) rejects the default zigzag CP layout and contiguous CP needs THD (packed) inputs from a sequence-packing scheduler. The correct combo is: --cp_partition_mode contiguous --packing true --sequence_packing_scheduler default_dynamic_cp No code change; comments and flag values only.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR type
PR information
Write the detail information belongs to this PR.
Experiment results
Paste your experiment result here(if needed).