Administered by:
Deepseek R1 Distill 8B Q40 on 4 x Raspberry Pi 5, 6.43 tok/s (eval 11.68 tok/s)
L: https://github.com/b4rtaz/distributed-llama/discussions/162
C: https://news.ycombinator.com/item?id=43059579
posted on 2025.02.15 at 11:11:29 (c=0, p=13)
GitHub[v0.12.2] Deepseek R1 Distill 8B Q40 on 4 x Raspberry Pi 5 8GB · b4rtaz distributed-llama · Discussion #162Model: deepseek_r1_distill_llama_8b_q40 Version: 0.12.2 Evaluation Prediction 2 x Raspberry Pi 5 8GB 7.70 tok/s 3.54 tok/s 4 x Raspberry Pi 5 8GB 11.68 tok/s 6.43 tok/s 2 x Raspberry Pi 5 8GB ... 🔶...