Dash's Code Camp

How To: Run Your AI Locally in Windows and Linux (No Internet)

In taday’s “How To,” we will be running my own AI locally; no internet required. To start, go to ollama.ai or ollama.com. If you are on Windows, do not click Windows (Preview). Instead, Install WSL (Windows Subsystem for Linux) using these commands (in CMD or PS):

ps
wsl --install

After that, in the Start Menu, search “Ubuntu” and open it.

1. Install Ollama

1a. Windows

Open WSL and type: curl -fsSL https://ollama.com/install.sh | sh

1c. Linux

Open Terminal and type: curl -fsSL https://ollama.com/install.sh | sh

2. Is Ollama running?

To test if it is running, go to https://localhost:11434/ and if it says “Ollama is running”, then it’s running!

3. Install a model

After installing, open the Terminal/WSL and type: ollama pull llama3

This will download llama3 (4.8GB), the latest Llama model.

4. Test

After downloading, type ollama run llama3

Now, you can type with an AI in the command line.

5. Install Open WebUI

To install Open WebUI, a UI on the web, type these three commands to install Docker…

for pkg in docker.io docker-doc docker-compose podman-docker containerd runc; do sudo apt-get remove $pkg; done
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh

…and now install Open WebUI using this one command…

sudo docker run -d -p 3000:8080 -e OLLAMA_API_BASE_URL=http://127.0.0.1:11434/api -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Now, go to https://127.0.0.1:3000 to go to Open WebUI.

6. Set Up Open WebUI

To start go to https://127.0.0.1:3000 and click “Sign Up”. Enter your details and click Sign Up. For other users, you, the first one signing up, is automatically an admin. The other users can only have access if you go to the “admin panel” (user -> Admin Panel) and turn the user from “pending” to “user” or “admin”.

Optional: Make it Accessible from All Locations

Optionally, install ngrok to make localhost:3000 a website that anyone could visit anywhere, anytime.

Conclusion

In conclusion, you can run an AI locally downloading files and using the Ollama service.

Tutorials