通过LM STUDIO在本地部署大语言模型  

https://lmstudio.ai/

LM Studio – Discover, download, and run local LLMs

下载一个LM STUDIO,只有一个文件,运行,搜索大模型名字,就能看到模型名称,右侧列出了本地兼容可以运行的版本。一般都是量化够后的模型,可以根据自己的电脑配置选择下载。(因为国内屏蔽了huggingface等等大模型网站,系统需要科学上网)
下载完之后,点左侧对话图标,可以选择加载已经下载的大模型了。右侧有选项,有GPU的同学可以选择加载到显存的层数,具体设置可以试,放不下会报错。设置为-1就是全部放进显存。我的P40显存有24g,32g大小的模型层数可设置到16。
实际运行这几天最火的dolphin mixtral 7b模型非常流畅,出字后每秒可以8个英文字符左右。

For Windows 链接:https://pan.baidu.com/s/1c35ICKyhfHCocHr5qmJ68A?pwd=we2n     提取码:we2n

For MAC M1/M2/M3  链接:https://pan.baidu.com/s/1pAIfl7M7JSMt4fzUCsbZVg?pwd=u7b4    提取码:u7b4 

For Linux 链接:https://pan.baidu.com/s/1CYDziO5Lw9TpGSCUTRd29A?pwd=gilc     提取码:gilc

With LM Studio, you can …
🤖 – Run LLMs on your laptop, entirely offline
👾 – Use models through the in-app Chat UI or an OpenAI compatible local server
📂 – Download any compatible model files from HuggingFace
🤗 repositories
🔭 – Discover new & noteworthy LLMs in the app’s home page

LM Studio supports any ggmlLlama, MPT, and StarCoder model on Hugging Face (Llama 2, Orca, Vicuna, Nous Hermes, WizardCoder, MPT, etc.)

Minimum requirements: M1/M2/M3 Mac, or a Windows PC with a processor that supports AVX2. Linux is available in beta.

Made possible thanks to thellama.cpp project.

LM Studio Beta ReleasesHere you may find from time to time beta releases of LM Studio, provided under the terms of use.Join the community on Discord to discuss these beta releases.🐧 Linux Beta (Based on 0.2.8)Includes0.2.8 build for Linux (AppImage)Does not support GGML; only GGUFKnown issues on Ubuntu 20If you are getting errors: run the app from the terminal and watch for any error logsDoes NOT support auto-update; please keep an eye on this page for updates or join the LM Studio Discord server for announcements[Bug] Missing icons on some Linux distributions
DownloadsLM+Studio-0.2.8-beta-v1.AppImage
AVX Support (Based on 0.2.8)IncludesFor older PCs without AVX2 instruction set
Downloads
WindowsLM-Studio-0.2.8-Setup-avx-beta-2.exe

Categories:

Tags:

Comments are closed