KItCAT — Knowledge Injection via Token Corruption and Augmented Training. [Paper]
KItCAT is a lightweight knowledge injection pipeline for decoder-only LLMs. It augments standard next-token prediction by randomly replacing input tokens with other vocabulary tokens at training time, while keeping the original labels unchanged. This generates diverse training inputs from each sample at negligible cost — no paraphrasing, no extra models required.
Code is coming soon