Skip to content

Fix training issues and compatability with transformers v4.x and v5#326

Merged
Ingvarstep merged 8 commits into
mainfrom
fix/training
Jan 29, 2026
Merged

Fix training issues and compatability with transformers v4.x and v5#326
Ingvarstep merged 8 commits into
mainfrom
fix/training

Conversation

@Ingvarstep

@Ingvarstep Ingvarstep commented Jan 29, 2026

Copy link
Copy Markdown
Collaborator

The following issues were fixed:

  1. Incorrect loading of spm.model tokenizers in the case of transformers v5;
  2. Data processing issues for token-level labels preparation;
  3. Trainer compatibility between v4.x and v5.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant