Fix Bugs #101

danielhanchen · 2025-03-26T12:08:59Z

Solves:

Gemma vision requires grad failed
Pixtral & Mistral3 SDPA does NOT work
Qwen Vision inference gibberish
finetune_vision_layers **kwargs
UNSLOTH_DISABLE_AUTO_UPDATES
Fix Mistral V3 Data Collator not working

* gemma require gradients fix * Update peft_utils.py --------- Co-authored-by: Daniel Han <[email protected]>

…pointing-warning Fix gradient checkpointing warning filter implementation

Fix bugs in generate_batches.py.Original output = [] will result in duplication of results.

* Support Iterable Datasets * Update dataset_utils.py * Update dataset_utils.py * Update dataset_utils.py * Update dataset_utils.py * Preserve batch size from iterable dataset * Preserve batch size from iterable dataset * Support train_on_response_only with IterableDataset * Support train_on_response_only with IterableDataset * Support train_on_response_only with IterableDataset * Support train_on_response_only with IterableDataset

…to nightly

danielhanchen added 30 commits March 12, 2025 21:52

Update dataset_utils.py

a7c257a

Update dataset_utils.py

6556f13

Update compiler.py

9a05b2f

Update compiler.py

0f6fc7a

Update compiler.py

4bb152a

Update compiler.py

3ccdf86

Update loss_utils.py

1783ba1

Update loss_utils.py

b2983f4

Merge branch 'main' into nightly

3c9c731

gpu_memory_utilization

4c5c77d

Update temporary_patches.py

b918327

Update vision_utils.py

e8f561c

Update vision_utils.py

4459ef8

Update vision_utils.py

62e0e14

Update vision_utils.py

9f4b729

Update vision_utils.py

29a7abf

Update vision_utils.py

9830edd

Update vision_utils.py

be53fda

Update vision_utils.py

28f4df4

train on completions VLMs

ad13d0a

Update dataset_utils.py

370cbd7

Update dataset_utils.py

bd60d26

Update dataset_utils.py

29ed559

Update dataset_utils.py

e0a4416

VLM train only on completions

d6e55ca

Update loss_utils.py

adf8307

Update dataset_utils.py

98d5885

Update compiler.py

967c2ba

Update compiler.py

ddf2b8e

Update compiler.py

cb2f6c7

rolandtannous and others added 29 commits March 24, 2025 15:39

Fix gradient checkpointing warning filter implementation

a62e4c6

Input grads fix for gemma3 (#96)

d115cea

* gemma require gradients fix * Update peft_utils.py --------- Co-authored-by: Daniel Han <[email protected]>

Merge pull request #97 from rolandtannous/fix/suppress-gradient-check…

454757c

…pointing-warning Fix gradient checkpointing warning filter implementation

Update vision_utils.py

c50123a

Vision requires grad

b199491

Check SDPA for Mistral / Pixtral

1670fa6

Update compiler.py

e32f797

Update vision_utils.py

b9d9cc5

Update vision_utils.py

5e3c88f

Update vision_utils.py

5c4086c

Update __init__.py

0599242

Update vision_utils.py

8da5939

Update vision_utils.py

db90dca

Update vision_utils.py

51cefe5

Update vision_utils.py

20b42ce

Update vision_utils.py

b03ded6

Update vision_utils.py

65469c2

Update vision_utils.py

7f4eb00

Update vision_utils.py

8584b5d

Update vision_utils.py

8bb6b55

Update vision_utils.py

20a61b0

Update vision_utils.py

86ca6d5

Update vision_utils.py

0221094

Update vision_utils.py

d13ebf7

Update vision_utils.py

f6c4b2e

Update vllm_utils.py (#99)

2d1e506

Fix bugs in generate_batches.py.Original output = [] will result in duplication of results.

Update vision_utils.py

23e018f

Merge branch 'nightly' of https://github.com/unslothai/unsloth-zoo in…

affb9d8

…to nightly

danielhanchen merged commit 4a66f8b into main Mar 26, 2025
2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix Bugs #101

Fix Bugs #101

danielhanchen commented Mar 26, 2025

Fix Bugs #101

Fix Bugs #101

Conversation

danielhanchen commented Mar 26, 2025