You do seem to know a thing or 2! ???? Real beginners asked questions here so the simple install and lower hardware requirements were taken into account... (GGUF for highly quantized R1 models.) But ONNX can - indeed - greatly improve inference speed: medium.com/@vimalkansal...
Oh the sensitive data issue is only if you use the API or online chat interface. Fine if you're running it locally, but then you're limited to the smaller versions of the model – slightly less capable
Yeah that's fair enough. The API and chat seem fine if you're asking for general help without anything involving any sensitive, personal data. But running the 14B model locally on a Macbook Pro works just fine too.
Yup, got it working locally on a Macbook M1 with Ollama. There are helpful links from others in this thread, but I also updated the original post with more details on local use ????
"It won't discuss Tian—" ???? Ask ChatGPT about Alexander Hanff. Or Guido Scorza. Or Brian Hood. They both do censorship. But one is open source and can be modified however you want.
You can run distilled versions like qwen 3.2b r1 locally. Personally I won't trust their API despite being darn cheap. Because user prompts are used for training and their privacy policy is kinda shady Overall, it is a great model. Good thing is, they got MAGA Sam Hypeman coping
Let me go into a few questions asked here... You can run smaller 'distilled' R1 versions locally. Smaller means lower quality too. Most easy way is using Ollama ollama.com/library/deep... Online chat is free, for now, but it will not go into 'sensitive subjects' chat.deepseek.
Online/API: Do not use your own sensitive data as input. You need enough memory locally for reasonable quality and speed. If you have enough VRAM you can use one of the larger models, like: huggingface.co/bartowski/de... That is an uncensored variation you can use with Ollama
Thanks for the write up! I think I’ll follow this how to: For those asking, looks like you can start with smaller models. www.reddit.com/r/selfhosted...
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。