Deploying a Private LLM on Azure | Docker + Ollama + FastAPI + VNet Architecture

Description

In this video, I deploy a distributed LLM Chat Application on Microsoft Azure using a secure VNet architecture. Private LLM Server (Ollama + TinyLlama) running in Docker Public FastAPI backend on a separate VM Secure inter-VM communication using Azure Virtual Network Reverse proxy using Nginx Network Security Group configuration SSH jump host setup Model serving via REST API Architecture Overview: User → Public VM (FastAPI) → Private VM (Ollama) → TinyLlama → Response This project demonstrates real-world cloud architecture principles including: Public vs Private infrastructure separation Secure networking design Containerized model deployment API integration Azure cloud fundamentals Tech Stack: Microsoft Azure Ubuntu 24.04 Docker Ollama TinyLlama FastAPI Virtual Networks (VNet) Network Security Groups (NSG) This is an intermediate-level cloud + AI infrastructure project.