Introducing 1.58bit DeepSeek-R1 GGUFs! 🐋
DeepSeek-R1 can now run in 1.58-bit, while being fully functional. We shrank the 671B parameter model from 720GB to just 131GB - a 80% size reduction.
Naively quantizing all layers breaks the model entirely, causing endless loops &