Private AI Workspace Demo

What this demo does

This demo runs a lightweight private AI assistant on your own Ubuntu VM using FastAPI and Ollama. Browser access happens through an SSH tunnel, so the app does not need public IPv4.

The demo repository is https://github.com/rdekarmakar/gangacloud-customer-ai-demo.

• Runs on a private Ubuntu VM
• Uses Ollama with qwen2.5:0.5b
• Serves a small browser demo through FastAPI
• Uses an SSH tunnel for local browser access

Install Ollama

curl -fsSL https://ollama.com/install.sh | sh

ollama --version

Pull the tiny model

ollama pull qwen2.5:0.5b

ollama run qwen2.5:0.5b

Clone and run the demo

git clone https://github.com/rdekarmakar/gangacloud-customer-ai-demo.git

cd gangacloud-customer-ai-demo

make venv

make test-ollama

make run

Open the SSH tunnel

Run this from your own computer, then keep the terminal open while using the browser demo.

ssh -J <PROXMOX_USER>@<PROXMOX_PUBLIC_IP> -L 8000:localhost:8000 ubuntu@<VM_PRIVATE_IP>

http://localhost:8000/demo