Pinned Loading

1. RWKV-Infer RWKV-Infer Public

   A large-scale RWKV v7(World, PRWKV, Hybrid-RWKV) inference. Capable of inference by combining multiple states(Pseudo MoE). Easy to deploy on docker. Supports true multi-batch generation and dynamic…

   Python 52 2

2. RWKV-LM-RLHF RWKV-LM-RLHF Public

   Reinforcement Learning Toolkit for RWKV.(v6,v7,ARWKV) Distillation,SFT,RLHF(DPO,ORPO), infinite context training, Aligning. Exploring the possibilities for deeper fine-tuning of RWKV.

   Python 63 6

3. RWKV-infctx-trainer-LoRA RWKV-infctx-trainer-LoRA Public

   RWKV v5, v6 infctx LoRA trainer with 4bit quantization,Cuda and Rocm supported, for training arbitary context sizes, to 10k and beyond!

   Python 8 2

4. RWKV-LM-State-4bit-Orpo RWKV-LM-State-4bit-Orpo Public

   State tuning with Orpo of RWKV v6 can be performed with 4-bit quantization. Every model can be trained with Orpo on Single 24GB GPU!

   Python 7 1

5. RWKV5-LM-LoRA RWKV5-LM-LoRA Public

   RWKV v5,v6 LoRA Trainer on Cuda and Rocm Platform. RWKV is a RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN an…

   Python 13 4

6. RWKV-LM-RLHF-DPO-LoRA RWKV-LM-RLHF-DPO-LoRA Public

   Forked from Triang-jyed-driung/RWKV-LM-RLHF-DPO

   Direct Preference Optimization LoRA for RWKV, aiming for RWKV-5 and 6.

   Python 1