Run Ollama & Open WebUI on Windows Without Docker: Easy Python Setup for Local LLMs

The world of Artificial Intelligence is rapidly evolving, bringing sophisticated Large Language Models (LLMs) from cloud servers directly to our local machines. Running LLMs locally offers unparalleled privacy, control, and performance, freeing you from internet dependencies and subscription fees. While Docker has become a popular choice for deploying many AI applications, its learning curve and “voodoo, gremlin-filled” nature (as one user eloquently put it) can be a significant barrier for many Windows users.

This comprehensive guide cuts through that complexity. We’ll show you how to set up Ollama, a powerful tool for running LLMs, and Open WebUI, a beautiful and intuitive interface for interacting with them, all on your Windows PC using a simple Python-centric approach – no Docker required. By the end of this tutorial, you’ll have your very own local AI chatbot ready to assist you, powered by models like Llama 2, Mixtral, and more, running directly on your hardware. Get ready to unlock the full potential of local AI, simplified for everyone.

Why Choose This Docker-Free Python Setup?

Many tutorials lean heavily on Docker for deployment due to its containerization benefits. However, for users who simply want to get a local AI up and running without diving into container orchestration, Docker can introduce unnecessary overhead and complexity. This Python-based approach offers:

• Simplicity: A more direct installation process that leverages familiar Windows tools and Python.
• Native Performance: Potentially better performance by running applications directly on your host OS.
• Full Control: Easier access to configuration files and direct integration with your Windows environment.
• Resource Efficiency: Avoids the overhead of running a full Docker daemon and containers.

Ready to build your local AI powerhouse? Let’s begin!

Prerequisites: What You’ll Need

Before we dive into the installation steps, ensure your Windows machine meets the following requirements:

• Operating System: Windows 10 or 11 (64-bit).
• Processor: A modern multi-core CPU.
• RAM: At least 8GB of RAM is recommended. For larger models (e.g., 13B and above), 16GB, 32GB, or even 64GB will significantly improve performance.
• Graphics Card (GPU): While Ollama can run on CPU, a dedicated NVIDIA GPU (with CUDA support) or an AMD GPU (with ROCm support, though Windows support for ROCm is more limited) will dramatically accelerate LLM inference. The more VRAM, the better.
• Disk Space: At least 20-50 GB of free disk space for Ollama models (each model can range from 4GB to over 40GB).
• Internet Connection: Required for downloading installers and LLM models.

Step 1: Install Python (If Not Already Installed)

Python is the backbone of our “Docker-free” setup. Many Windows systems might already have Python installed, but it’s crucial to have a recent version (3.8+) and ensure it’s correctly added to your system’s PATH.

1. Check for Existing Python:
Open your Command Prompt or PowerShell and type:

python --version

If you see a version number (e.g., Python 3.10.x), you might be good to go. If not, or if the version is old, proceed with the installation.

2. Download Python:
Visit the official Python website: https://www.python.org/downloads/windows/
Download the latest stable “Windows installer (64-bit)”.

3. Run the Installer:

• Double-click the downloaded .exe file.
• CRITICAL STEP: On the first screen, check the box that says “Add Python.exe to PATH”. This is essential for running Python commands from any directory in your terminal.
• Select “Install Now” (recommended for new users).
• Follow the prompts to complete the installation.

4. Verify Python Installation:
Open a new Command Prompt or PowerShell window (to ensure PATH changes are recognized) and type:

python --version
pip --version

You should see the installed Python version and the pip version (Python’s package installer).

Step 2: Install Git (If Not Already Installed)

Git is a version control system that we’ll use to download the Open WebUI source code from GitHub.

1. Check for Existing Git:
Open Command Prompt or PowerShell and type:

git --version

If you see a version number, Git is installed. Otherwise, continue.

2. Download Git:
Go to the official Git website: https://git-scm.com/download/win
Download the appropriate installer for your Windows version.

3. Run the Installer:

• Double-click the downloaded .exe file.
• Follow the installation prompts. The default options are usually sufficient for most users. Ensure that “Git from the command line and also from 3rd-party software” is selected during the PATH configuration step.

4. Verify Git Installation:
Open a new Command Prompt or PowerShell window and type:

git --version

You should see the installed Git version.

Step 3: Install Ollama

Ollama is the engine that allows you to run open-source large language models locally on your computer. It handles the complexities of model loading, memory management, and inference.

1. Download Ollama for Windows:
Visit the official Ollama website: https://ollama.com/download
Click on the “Download for Windows” button.

2. Run the Ollama Installer:

• Double-click the downloaded OllamaSetup.exe file.
• The installation is straightforward; just follow the on-screen instructions. Ollama will automatically configure itself and start a background service upon completion. You’ll typically see an Ollama icon in your system tray.

3. Verify Ollama Installation and Run Your First Model:
Open Command Prompt or PowerShell. Ollama automatically adds itself to your PATH.
Let’s pull and run a popular model, Llama 2:

ollama run llama2

• The first time you run this, Ollama will download the llama2 model, which can take a few minutes depending on your internet speed and the model size (Llama 2 is about 3.8GB).
• Once downloaded, llama2 will start, and you’ll see a prompt where you can type questions.
• Type /? for help or bye to exit the chat.

To see a list of all models you’ve downloaded:

ollama list

4. Pulling Other Models:
Ollama supports a wide variety of models. You can explore them on the Ollama library page: https://ollama.com/library
To download any model, use the ollama pull command, for example:

ollama pull mixtral

This will download the Mixtral 8x7B model, a powerful alternative to Llama 2.

Step 4: Set Up Open WebUI with Python

Open WebUI provides a beautiful, user-friendly interface that makes interacting with your local Ollama models a breeze. While its backend is Python, its frontend requires Node.js and npm for building. We will install these briefly as a dependency for the frontend build.

1. Create a Directory for Open WebUI:
It’s good practice to keep your projects organized. Open Command Prompt and create a new directory:

mkdir C:\OpenWebUI
cd C:\OpenWebUI

You can choose any directory, e.g., D:\AI\OpenWebUI.

2. Clone the Open WebUI Repository:
Use Git to download the Open WebUI source code into your new directory:

git clone https://github.com/open-webui/open-webui.git .

The . at the end means “clone into the current directory,” so all files will be directly inside C:\OpenWebUI.

3. Install Node.js and npm (for Frontend Build):
Open WebUI’s user interface is built using modern web technologies (React), which requires Node.js and its package manager, npm, to compile. This is a one-time setup for building the frontend.

• Go to the Node.js website: https://nodejs.org/en/download
• Download the “Windows Installer (.msi)” for the LTS (Long Term Support) version.
• Run the installer. Accept the license agreement, choose default installation paths, and ensure “Add to PATH” is selected. Restart your Command Prompt after installation.
• Verify installation:

  node --version
  npm --version

4. Build the Open WebUI Frontend:
Navigate back to your Open WebUI directory (C:\OpenWebUI in our example) in Command Prompt:

cd C:\OpenWebUI

Now, install the frontend dependencies and build the UI:

npm install
npm run build

This process might take several minutes, depending on your internet speed and system performance. It downloads all necessary JavaScript packages and then compiles the web interface.

5. Install Python Backend Dependencies:
Once the frontend build is complete, we need to install the Python packages required for the Open WebUI backend. Still in the C:\OpenWebUI directory:

pip install -r backend/requirements.txt

This command reads the requirements.txt file in the backend folder and installs all the necessary Python libraries.

6. Run the Open WebUI Backend:
Finally, you can start the Open WebUI backend server:

python backend/app.py

You should see output indicating that the server is starting, typically on http://localhost:8080. Keep this Command Prompt window open as long as you want Open WebUI to run.

Step 5: Access Open WebUI and Start Chatting

With both Ollama and Open WebUI running, you’re ready to interact with your local LLMs!

1. Open Your Web Browser:
Go to http://localhost:8080 (or the address shown in your Command Prompt).

2. Create Your First Account:
The first time you access Open WebUI, you’ll be prompted to create an account. This is for local access management; it doesn’t connect to any external service. Fill in your desired username and password.

3. Select a Model and Chat:

• After logging in, you’ll see the Open WebUI interface.
• In the top left corner, you might see a dropdown or a prompt to “Select a model.” Choose llama2 (or mixtral, or any other model you pulled with Ollama). If you don’t see any models, ensure Ollama is running in the background. You might need to refresh the page or restart the python backend/app.py process if Ollama was started after Open WebUI.
• Type your prompt into the chat box at the bottom and hit Enter.
• Congratulations! You’re now chatting with a powerful LLM running entirely on your Windows PC, without Docker!

Advanced Tips and Troubleshooting

Running Open WebUI in the Background

Keeping the Command Prompt window open for python backend/app.py can be inconvenient. For a more persistent setup:

• Batch Script: You can create a simple batch file (start_webui.bat) with the command start python backend/app.py (ensure you cd into the directory first). This will open a new, minimized console window.
• NSSM (Non-Sucking Service Manager): For a truly robust background service, consider using NSSM to install python backend/app.py as a Windows service. This is beyond the scope of this “easy setup” but offers enterprise-grade reliability.

Updating Ollama and Open WebUI

Keeping your tools updated is important for new features and bug fixes.

• Ollama: Simply download and run the latest OllamaSetup.exe from ollama.com/download. It will automatically update your existing installation.
• Open WebUI:
  1. Stop the python backend/app.py process (close its Command Prompt).
  2. Navigate to your C:\OpenWebUI directory in Command Prompt.
  3. Pull the latest changes: git pull
  4. Re-run frontend build (if updates might affect frontend): npm install && npm run build
  5. Re-install backend dependencies (if requirements.txt changed): pip install -r backend/requirements.txt
  6. Start the backend again: python backend/app.py

Common Issues

• ” ‘python’ is not recognized…”: This means Python is not in your system’s PATH. Re-run the Python installer and ensure “Add Python to PATH” is checked, or manually add it.
• ” ‘git’ is not recognized…”: Similar to Python, Git is not in your PATH. Re-run the Git installer.
• ” ‘npm’ is not recognized…”: Node.js/npm is not installed or not in your PATH. Install Node.js or ensure the PATH is correctly set.
• Port 8080 already in use: If another application is using port 8080, Open WebUI might fail to start. You can change the port in Open WebUI’s configuration (check the official documentation for environment variables, usually OLLAMA_WEBUI_PORT).
• Firewall blocking access: Ensure your Windows Firewall isn’t blocking python.exe or port 8080. You might need to add an exception.
• Ollama models not appearing in WebUI: Make sure the Ollama service is running in the background. You can check the system tray icon or try ollama list in a new terminal. Restarting python backend/app.py after Ollama is confirmed running usually resolves this.

Conclusion

You’ve successfully set up a powerful local AI environment on your Windows PC, complete with Ollama for running diverse LLMs and Open WebUI for a fantastic conversational interface, all without the complexities of Docker. This setup empowers you with privacy, control, and the ability to experiment with cutting-edge AI models directly on your hardware.

The journey into local LLMs is just beginning. As you explore different models and features within Open WebUI, you’ll discover new possibilities for productivity, creativity, and learning. Enjoy your personalized, private AI assistant!

Frequently Asked Questions

What is Ollama?

Ollama is a platform that simplifies running large language models (LLMs) locally on your computer. It handles model downloading, loading, and inference efficiently.

What is Open WebUI?

Open WebUI is a free, open-source, and highly customizable web interface that provides a user-friendly chat experience for local LLMs powered by Ollama. It mimics popular AI chatbots but runs entirely on your system.

Why run without Docker?

Running without Docker eliminates the need for Docker Desktop, container management, and its associated overhead, simplifying the setup process for many Windows users and potentially offering better native performance.

Do I need a powerful GPU?

While Ollama can run models on your CPU, a powerful NVIDIA GPU (with CUDA) or AMD GPU (with ROCm) with sufficient VRAM will significantly speed up inference, especially for larger or more complex LLMs.

Can I use different LLMs?

Yes! Ollama supports a wide range of models. You can pull any model from the Ollama library using ollama pull [model_name] and select it within Open WebUI.

How do I update Ollama and Open WebUI?

Ollama is updated by re-running its installer. Open WebUI is updated by git pull from its repository, re-running npm install && npm run build for the frontend, and pip install -r backend/requirements.txt for the backend.