minus-squaretau@lemmings.worldtoTechnology@lemmy.world•Local AI is one step closer through Mistral-NeMo 12BlinkfedilinkEnglisharrow-up4·4 months agoJust beware that like AMD, Intel GPUs suffer a performance hit when using LLMs because of the CUDA specific optimizations in frameworks like llama.cpp linkfedilink
minus-squaretau@lemmings.worldtoPrivacy@lemmy.ml•What is the cost of privacy?linkfedilinkarrow-up4·6 months agoI’ve heard good things about Instander, but it may not satisfy all the requirements of an alternative frontend. linkfedilink
Just beware that like AMD, Intel GPUs suffer a performance hit when using LLMs because of the CUDA specific optimizations in frameworks like llama.cpp