Building and Scaling an AI Startup with Async Django

Schedule

Room: Room 111

April 3

12:00–12:25

Abstract

Django's async capabilities and batteries-included tooling make it an ideal framework for quickly building MVPs and iterating. This talk demonstrates building a document search MVP with Django templates, ChromaDB, and hosted large language models. It then shows how to refactor and scale it using Elasticsearch, Celery/RabbitMQ workers, React, self-hosted vLLM, and auth. With Django async, you can rapidly build, constantly improve, and deploy the latest AI models in your product.

Prerequisites

Python, Django

Description

Building and Scaling an AI Startup with Async Django

In the fast-moving world of AI, startups need to build and deploy new ideas quickly. AI models operate asynchronously and require streaming responses for the best user experience. Luckily, Django now incorporates async behavior while still providing out-of-the-box tooling, making it ideal for rapidly building an AI startup.

This talk demonstrates building an MVP where we can ask a Large Language Model (LLM) about the contents of documents (Retrieval Augmented Generation, RAG) usingDjango and then refactoring, scaling, and developing it into a full product. We'll build a simple RAG MVP using ChromaDB and stream responses from a hosted large language model in real-time to a chat interface using Django templates and streaming responses.

Next, we'll scale and improve the system by:

• Replacing ChromaDB with Elasticsearch
• Replacing Django Async with Celery workers and RabbitMQ for complex processing
• Replacing Django templates with a React app
• Self-hosting a large language model with vLLM
• Adding token-based authentication

AI models evolve exponentially - this talk shows how to rapidly integrate new models, build business logic, scale, and swap improved models. With Django's async capabilities, you can build and iterate an AI product quickly.