Skip to content

Handle prompt embedding concat in Qwen dreambooth example#13387

Open
chenyangzhu1 wants to merge 6 commits intohuggingface:mainfrom
chenyangzhu1:qwen-image-concat-dimension
Open

Handle prompt embedding concat in Qwen dreambooth example#13387
chenyangzhu1 wants to merge 6 commits intohuggingface:mainfrom
chenyangzhu1:qwen-image-concat-dimension

Conversation

@chenyangzhu1
Copy link
Copy Markdown

@chenyangzhu1 chenyangzhu1 commented Apr 2, 2026

What does this PR do?

When running the train_dreambooth_lora_qwen_image.py script with the --with_prior_preservation flag, the training crashes during the text embedding extraction phase. Fixing two distinct bugs related to tensor concatenation at L1323 and L1324.

Solution

To resolve these issues, the tensor concatenation logic needs to handle dynamic padding and missing masks dynamically:

  1. Reconstruct Masks: If a mask is missing (None), reconstruct it as an all-ones tensor based on its corresponding embedding's sequence length.
  2. Pad to Max Length: If the sequence lengths differ, pad the shorter tensor(s) to match the longest sequence.
  3. Fold Back to None: After concatenation, if the merged mask consists entirely of 1s, fold it back to None.

Fixes #13386

Before submitting

Who can review?

@sayakpaul

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

return prompt_embeds_mask.to(device=prompt_embeds.device)


def _pad_prompt_embedding_pair(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would maybe also add a comment on how this is only relevant during prior preservation.

Copy link
Copy Markdown
Author

@chenyangzhu1 chenyangzhu1 Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

I added the following comment

# These helpers only matter for prior preservation, where instance and class prompt
# embedding batches are concatenated and may not share the same mask/sequence length.

Copy link
Copy Markdown
Member

@sayakpaul sayakpaul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Just one comment.

@sayakpaul sayakpaul requested a review from linoytsaban April 3, 2026 05:46
@sayakpaul
Copy link
Copy Markdown
Member

@bot /style

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 3, 2026

Style bot fixed some files and pushed the changes.

@HuggingFaceDocBuilderDev
Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] train_dreambooth_lora_qwen_image.py crashes with --with_prior_preservation due to tensor concatenation errors

3 participants