🚀 Setting Up Docker on macOS and Ubuntu 22.04 for Open-WebUI and Ollama

Welcome 👋

Overview

Setting up Docker on macOS and Linux for GPU-accelerated workloads or lightweight services can be tricky, especially with platform-specific constraints. For instance, macOS M1/M2 GPUs aren’t natively supported by Docker, making tools like OrbStack and native installations of some applications a necessity. Meanwhile, Ubuntu users benefit from Nvidia GPU integration for Docker but require additional configuration.

In this guide, we’ll walk through the process of:

1. Setting up Docker with OrbStack on macOS for lightweight and fast performance.
2. Configuring Nvidia GPU support for Docker on Ubuntu 22.04.
3. Deploying Open-WebUI and Ollama with Docker on both platforms.

Setting Up Docker on macOS

1. Install OrbStack for Docker Services

OrbStack is a lightweight alternative to Docker Desktop, offering faster performance and lower resource usage. Here’s how to get started:

• Download and install OrbStack from OrbStack’s official website.
• Once installed, OrbStack automatically manages your Docker services. No additional setup is required for basic Docker functionality.

Why OrbStack? Unlike Docker Desktop, OrbStack runs more efficiently on macOS, especially on Apple Silicon (M1/M2), saving both memory and CPU.

2. Deploy Open-WebUI Services with Docker

After OrbStack is set up, you can deploy Open-WebUI using Docker commands in terminal:

docker run -d -p 3000:8080 -v open-webui:/app/backend/data --name open-webui ghcr.io/open-webui/open-webui:main

Access Open-WebUI via http://localhost:3000.

3. Install Ollama Natively

Docker on macOS cannot currently utilise M-series GPUs. To use Ollama effectively:

If you want to install ollama with docker, specify an EXTERNAL disk to store models like

docker run -d \
  -v /Volumes/BenedictsT7/OllamaModels:/root/.ollama \
  -e OLLAMA_MODEL_PATH=/root/.ollama \
  -p 11434:11434 \
  --name ollama \
  ollama/ollama 

If you choose to download macOS client, the default model download path is $HOME/.ollama, so I recommend to softlink an external disk to this path before you install ollama. Then,

1. Download and install Ollama natively from Ollama’s official site to download.
2. Follow the installation instructions provided on the website.

Note: Installing Ollama natively ensures optimal performance since Docker cannot take advantage of macOS hardware acceleration for GPU workloads.

Setting Up Docker on Linux (Ubuntu 22.04)

1. Install Docker-CE

For a robust Docker setup on Ubuntu, install the Docker Community Edition (Docker-CE):

# Add Docker's official GPG key:
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to Apt sources:
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update

# Install the Docker packages(latest).
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin 

# Verify that the installation is successful 
sudo docker run hello-world

Common docker commands:

sudo docker ps -a 
sudo docker stop xxx
sudo docker remove xxx
sudo docker restart xxx

If you want to remove docker completely

To completely remove Docker from Ubuntu 22.04, including all configurations, containers, images, and networks, follow these steps:

Step 1: Stop all running containers

Run the following command to stop any running Docker containers:

sudo docker stop $(docker ps -q)

Step 2: Remove all Docker containers, images, and volumes

Remove all containers:

sudo docker rm $(docker ps -a -q)

Remove all images:

sudo docker rmi $(docker images -q)

Remove all volumes:

sudo docker volume rm $(docker volume ls -q)

Step 3: Uninstall Docker packages

Run the following commands to uninstall Docker and related packages:

sudo apt-get purge -y docker-ce docker-ce-cli containerd.io docker-compose-plugin
sudo apt-get autoremove -y --purge

Step 4: Delete Docker-related files

Remove Docker’s configuration files, images, containers, and networks:

sudo rm -rf /var/lib/docker
sudo rm -rf /etc/docker

Remove any remaining files in /var/run or /var/lib related to Docker:

sudo rm -rf /var/run/docker
sudo rm -rf /var/run/containerd
sudo rm -rf /var/lib/containerd

Step 5: Verify removal

Check if Docker services still exist:

systemctl list-units --type=service | grep docker

If any service is listed, disable and remove it:

sudo systemctl stop docker
sudo systemctl disable docker

Ensure the docker command is no longer available:

docker --version

If the command returns an error, Docker has been successfully removed.

Step 6: Optional - Remove Docker group

If you created a Docker group during the installation, you can remove it:

sudo groupdel docker

Step 7: Clean up unused dependencies (Optional)

Run the following command to ensure all unused packages are cleaned up:

sudo apt-get autoremove -y
sudo apt-get autoclean

After completing these steps, Docker will be completely removed from your Ubuntu 22.04 system.

2. Enable Nvidia GPU Support

To leverage Nvidia GPUs, install the Nvidia Container Toolkit:

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
# optionally
# sudo sed -i -e '/experimental/ s/^#//g' /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt-get update 
sudo apt-get install -y nvidia-container-toolkit 

# config docker 
sudo nvidia-ctk runtime configure --runtime=docker 
# restart docker daemon 
sudo systemctl restart docker

Verify GPU availability:

sudo docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

3. Install Ollama with Nvidia GPU Support

Make sure you’ve successfully setup NVIDIA Containers Docker. To install Ollama on Ubuntu:

sudo docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

If you want to add disks as .ollama storage, you can refer to macos setup sections above for the commands. Access Ollama at http://localhost:11434 to see whether it’s running.

4. Deploy Open-WebUI with Nvidia GPU Support

Deploy Open-WebUI in a similar fashion to Ollama:

sudo docker run -d -p 3000:8080 --gpus=all -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:ollama

Access Open-WebUI at http://localhost:3000.

Tips for Using Docker on macOS and Linux

1. On macOS: Use OrbStack’s integrated features to manage containers effortlessly. If a specific application doesn’t run well in Docker (e.g., Ollama), consider native installation.
2. On Linux: Nvidia GPUs offer significant performance gains. Ensure drivers and the container toolkit are updated regularly for compatibility.
3. General Tip: Regularly clean up unused images and containers to save disk space:

   docker system prune -af
   

Conclusion

Setting up Docker on macOS and Linux has never been easier with tools like OrbStack and Nvidia Container Toolkit. Whether you’re running services like Open-WebUI or exploring AI tools like Ollama, these step-by-step instructions will help you optimise your setup for performance and efficiency.

Ready to dive into Docker? Follow these instructions and get your development environment running smoothly today!