{"id":44,"date":"2024-05-09T21:34:08","date_gmt":"2024-05-09T21:34:08","guid":{"rendered":"https:\/\/dashbenton.com\/blog\/?p=44"},"modified":"2024-05-09T21:58:38","modified_gmt":"2024-05-09T21:58:38","slug":"how-to-run-your-ai-locally-in-windows-and-linux-no-internet","status":"publish","type":"post","link":"https:\/\/dashbenton.com\/blog\/2024\/05\/09\/how-to-run-your-ai-locally-in-windows-and-linux-no-internet\/","title":{"rendered":"How To: Run Your AI Locally in Windows and Linux (No Internet)"},"content":{"rendered":"\n

<\/p>\n\n\n\n

In taday’s “How To,” we will be running my own AI locally; no internet required. To start, go to ollama.ai<\/a> or ollama.com<\/a>. If you are on Windows, do not<\/strong> click Windows (Preview). Instead, Install WSL (Windows Subsystem for Linux) using these commands (in CMD or PS):<\/p>\n\n\n\n

ps\nwsl --install<\/code><\/pre>\n\n\n\n
After that, in the Start Menu, search “Ubuntu” and open it.<\/p>\n\n\n\n
1. Install Ollama<\/h2>\n\n\n\n
1a. Windows<\/h3>\n\n\n\n
Open WSL and type: curl -fsSL https:\/\/ollama.com\/install.sh | sh<\/p>\n\n\n\n
1c. Linux<\/h3>\n\n\n\n
Open Terminal and type: curl -fsSL https:\/\/ollama.com\/install.sh | sh<\/p>\n\n\n\n
2. Is Ollama running?<\/h2>\n\n\n\n
To test if it is running, go to https:\/\/localhost:11434\/<\/a> and if it says “Ollama is running”, then it’s running!<\/p>\n\n\n\n
3. Install a model<\/h2>\n\n\n\n
After installing, open the Terminal\/WSL and type: ollama pull llama3<\/p>\n\n\n\n
This will download llama3 (4.8GB), the latest Llama model.<\/p>\n\n\n\n
4. Test<\/h2>\n\n\n\n
After downloading, type ollama run llama3<\/p>\n\n\n\n
Now, you can type with an AI in the command line.<\/p>\n\n\n\n
5. Install Open WebUI<\/h2>\n\n\n\n
To install Open WebUI, a UI on the web, type these three commands to install Docker…<\/p>\n\n\n\n
for pkg in docker.io docker-doc docker-compose podman-docker containerd runc; do sudo apt-get remove $pkg; done\ncurl -fsSL https:\/\/get.docker.com -o get-docker.sh\nsudo sh get-docker.sh<\/code><\/pre>\n\n\n\n
…and now install Open WebUI using this one command…<\/p>\n\n\n\n
sudo docker run -d -p 3000:8080 -e OLLAMA_API_BASE_URL=http:\/\/127.0.0.1:11434\/api -v open-webui:\/app\/backend\/data --name open-webui --restart always ghcr.io\/open-webui\/open-webui:main<\/code><\/pre>\n\n\n\n
Now, go to https:\/\/127.0.0.1:3000<\/a> to go to Open WebUI.<\/p>\n\n\n\n
6. Set Up Open WebUI<\/h2>\n\n\n\n
To start go to https:\/\/127.0.0.1:3000<\/a> and click “Sign Up”. Enter your details and click Sign Up. For other users, you, the first one signing up, is automatically an admin. The other users can only have access if you go to the “admin panel” (user -> Admin Panel) and turn the user from “pending” to “user” or “admin”.<\/p>\n\n\n\n
Optional: Make it Accessible from All Locations<\/h2>\n\n\n\n
Optionally, install ngrok to make localhost:3000 a website that anyone could visit anywhere, anytime.<\/p>\n\n\n\n
Conclusion<\/h2>\n\n\n\n
In conclusion, you can run an AI locally downloading files and using the Ollama service.<\/p>\n","protected":false},"excerpt":{"rendered":"
In taday’s “How To,” we will be running my own AI locally; no internet required. To start, go to ollama.ai or ollama.com. If you are on Windows, do not click Windows (Preview). Instead, Install WSL (Windows Subsystem for Linux) using these commands (in CMD or PS): After that, in the Start Menu, search “Ubuntu” and […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5],"tags":[],"class_list":["post-44","post","type-post","status-publish","format-standard","hentry","category-tutorials"],"_links":{"self":[{"href":"https:\/\/dashbenton.com\/blog\/wp-json\/wp\/v2\/posts\/44","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dashbenton.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dashbenton.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dashbenton.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dashbenton.com\/blog\/wp-json\/wp\/v2\/comments?post=44"}],"version-history":[{"count":3,"href":"https:\/\/dashbenton.com\/blog\/wp-json\/wp\/v2\/posts\/44\/revisions"}],"predecessor-version":[{"id":47,"href":"https:\/\/dashbenton.com\/blog\/wp-json\/wp\/v2\/posts\/44\/revisions\/47"}],"wp:attachment":[{"href":"https:\/\/dashbenton.com\/blog\/wp-json\/wp\/v2\/media?parent=44"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dashbenton.com\/blog\/wp-json\/wp\/v2\/categories?post=44"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dashbenton.com\/blog\/wp-json\/wp\/v2\/tags?post=44"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}