Local Ai (OLLAMA) Installation and Setup Guide

Date: June 5, 2024
Author: Joseph El-Kadi

Introduction

This guide provides step-by-step instructions for installing and setting up Ollama on Linux and Windows systems using WSL (Windows Subsystem for Linux). Follow the instructions carefully to ensure a smooth installation process.

1. Download Ollama

Visit ollama.com to download the necessary files.
This guide is for Linux and Windows users.

1a. Windows WSL Installation

Open CMD or Terminal and type:

wsl --install

1b. After Installation and Setup

Type:

wsl

1c. Best Practice: System Update

Type:

sudo apt update
sudo apt upgrade -y

2. Install Ollama

Open Terminal, paste, and run the following command:

curl -fsSL https://ollama.com/install.sh | sh

2a. Verify Installation

Open your browser and go to:

localhost:11434

If you see the "Ollama is running" message, you are successfully running LLAMA!

3. Access AI Libraries

Visit huggingface.co
or
Visit ollama.com/library

4. Get Llama 3 (by Meta)

Type one of the following commands:

ollama pull llama3

or

ollama run llama3

5. WebUI - Docker

5a. Docker Installation
Type the following commands:

sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

5b. Add Docker Repository to Apt Sources

Type the following command:

echo \ "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \ $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \ sudo tee /etc/apt/sources.list.d/docker.list > /dev/null sudo apt-get update

5c. Full Docker Installation

Type:

sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

6. Host Docker Container for WebUI

Type the following command to run the Docker container:

sudo docker run -d --network=host -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=http://127.0.0.1:11434 --name open-webui --restart always ghcr.io/open-webui/open-webui:main

6a. Verify Docker Container

Type:

sudo docker ps

7. Access WebUI

Open your browser and go to:
localhost:8080
or
127.0.0.1:8080

8. Create SuperAdmin User

Your first user is your SuperAdmin user. Remember these credentials.
Click on "Sign Up" and complete the registration.

9. Add More AI Models to WebUI

Type:

ollama pull {ai-model}

10. Fine-Tune Your AI

  1. Click on Workspace.
  2. Click on "Create a model".
  3. Name your model.
  4. Set a model tag name.
  5. Add a description of what your model does.
  6. In Model Params, set your AI's boundaries.