Python Setup for AI Engineering
Set up a professional Python environment for AI projects: pyenv, venv, pip, requirements.txt, .env files, python-dotenv, and a clean project structure.
Python Setup for AI Engineering
A clean environment is the foundation of every reliable AI project. This lesson walks you through the full setup: managing Python versions with pyenv, isolating dependencies with venv, pinning packages, loading secrets from .env files, and structuring an AI application so it is maintainable from day one.
1. Managing Python Versions with pyenv
AI libraries often require specific Python versions. pyenv lets you install and switch between multiple versions without touching your system Python.
# Install pyenv (macOS / Linux)
curl https://pyenv.run | bash
# Add to ~/.bashrc or ~/.zshrc
export PYENV_ROOT="$HOME/.pyenv"
export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"
# Install a specific version
pyenv install 3.12.3
# Set a version globally
pyenv global 3.12.3
# Set a version for a single project directory
cd ~/projects/my-ai-app
pyenv local 3.12.3 # writes .python-version file
# Verify
python --version # Python 3.12.3On Windows, use the official Python installer from python.org or pyenv-win:
# Install pyenv-win
Invoke-WebRequest -UseBasicParsing -Uri "https://raw.githubusercontent.com/pyenv-win/pyenv-win/master/pyenv-win/install-pyenv-win.ps1" -OutFile "./install-pyenv-win.ps1"
.\install-pyenv-win.ps1
pyenv install 3.12.3
pyenv global 3.12.32. Creating and Activating a Virtual Environment
Never install project dependencies into your global Python. Always use a virtual environment.
# Create a venv named .venv in the project root
python -m venv .venv
# Activate on Linux / macOS
source .venv/bin/activate
# Activate on Windows PowerShell
.\.venv\Scripts\Activate.ps1
# You should see (.venv) in your prompt
# Deactivate when done
deactivateWhy .venv as the name?
- Most
.gitignoretemplates already exclude.venv - IDEs (VS Code, PyCharm) auto-detect it
- Keeps the project root clean
3. Installing Packages with pip
# Always upgrade pip first
pip install --upgrade pip
# Install packages
pip install openai httpx pydantic python-dotenv
# Install a specific version (important for reproducibility)
pip install openai==1.30.1
# Install from a requirements file
pip install -r requirements.txt
# Install in editable mode (for your own packages)
pip install -e .4. Pinning Dependencies with requirements.txt
# After installing everything your project needs:
pip freeze > requirements.txtA typical requirements.txt for an AI project:
# requirements.txt
openai==1.30.1
httpx==0.27.0
pydantic==2.7.1
pydantic-settings==2.2.1
python-dotenv==1.0.1
structlog==24.2.0
tenacity==8.3.0
redis==5.0.4
sqlalchemy==2.0.30
alembic==1.13.1
fastapi==0.111.0
uvicorn[standard]==0.30.0
pytest==8.2.0
pytest-asyncio==0.23.7Separating dev from production dependencies
# requirements-dev.txt
-r requirements.txt # include all production deps
pytest==8.2.0
pytest-asyncio==0.23.7
black==24.4.2
ruff==0.4.4
mypy==1.10.0Install dev dependencies in development:
pip install -r requirements-dev.txt5. Loading Secrets from .env Files
Never hardcode API keys. Store them in a .env file and load them at runtime.
# .env (never commit this file!)
OPENAI_API_KEY=sk-proj-...
AZURE_OPENAI_ENDPOINT=https://my-resource.openai.azure.com/
AZURE_OPENAI_API_KEY=...
AZURE_OPENAI_DEPLOYMENT=gpt-4o
REDIS_URL=redis://localhost:6379
DATABASE_URL=postgresql+asyncpg://user:pass@localhost/mydb
APP_ENV=development
LOG_LEVEL=DEBUGAdd .env to .gitignore immediately:
echo ".env" >> .gitignore
echo ".env.local" >> .gitignoreProvide a template others can copy:
# .env.example (commit this)
OPENAI_API_KEY=your-key-here
AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
AZURE_OPENAI_API_KEY=your-azure-key-here
AZURE_OPENAI_DEPLOYMENT=gpt-4o
REDIS_URL=redis://localhost:6379
DATABASE_URL=postgresql+asyncpg://user:pass@localhost/mydb
APP_ENV=development
LOG_LEVEL=INFO6. python-dotenv
# config.py
from dotenv import load_dotenv
import os
# Load .env from the current directory (or parent dirs)
load_dotenv()
OPENAI_API_KEY = os.environ["OPENAI_API_KEY"]
APP_ENV = os.getenv("APP_ENV", "production") # with default
LOG_LEVEL = os.getenv("LOG_LEVEL", "INFO")
print(f"Running in {APP_ENV} mode")Load a specific file:
load_dotenv(".env.local") # override with local file
load_dotenv(override=True) # env vars in the file override existing env vars7. Pydantic Settings (Recommended for AI Apps)
Pydantic Settings gives you type-safe config with automatic validation:
# settings.py
from pydantic_settings import BaseSettings, SettingsConfigDict
from pydantic import Field
class Settings(BaseSettings):
model_config = SettingsConfigDict(
env_file=".env",
env_file_encoding="utf-8",
case_sensitive=False,
)
# Required ā will raise if missing
openai_api_key: str
# Optional with defaults
azure_openai_endpoint: str = ""
azure_openai_deployment: str = "gpt-4o"
redis_url: str = "redis://localhost:6379"
database_url: str = ""
app_env: str = "production"
log_level: str = "INFO"
max_tokens: int = Field(default=1024, ge=1, le=8192)
# Singleton ā import this everywhere
settings = Settings()Usage:
# main.py
from settings import settings
import openai
client = openai.OpenAI(api_key=settings.openai_api_key)
print(f"Using model: {settings.azure_openai_deployment}")8. Project Structure for an AI Application
my-ai-app/
āāā .env # secrets ā never commit
āāā .env.example # template ā commit this
āāā .gitignore
āāā .python-version # pyenv version pin
āāā requirements.txt
āāā requirements-dev.txt
āāā pyproject.toml # optional: modern packaging
āāā README.md
ā
āāā src/
ā āāā my_ai_app/
ā āāā __init__.py
ā āāā settings.py # Pydantic Settings
ā āāā main.py # FastAPI app or entry point
ā ā
ā āāā agents/ # Agent logic
ā ā āāā __init__.py
ā ā āāā assistant.py
ā ā
ā āāā tools/ # Tool implementations
ā ā āāā __init__.py
ā ā āāā search.py
ā ā āāā calendar.py
ā ā
ā āāā models/ # Pydantic models / DB models
ā ā āāā __init__.py
ā ā āāā schemas.py
ā ā āāā db.py
ā ā
ā āāā services/ # Business logic
ā ā āāā __init__.py
ā ā āāā llm_service.py
ā ā
ā āāā utils/ # Shared utilities
ā āāā __init__.py
ā āāā logging.py
ā āāā retry.py
ā
āāā tests/
ā āāā conftest.py
ā āāā test_agents/
ā āāā test_tools/
ā
āāā docs/9. Makefile for Common Commands
# Makefile
.PHONY: install dev-install test lint format run
install:
pip install --upgrade pip
pip install -r requirements.txt
dev-install:
pip install --upgrade pip
pip install -r requirements-dev.txt
test:
pytest tests/ -v
lint:
ruff check src/
mypy src/
format:
black src/ tests/
ruff check --fix src/
run:
uvicorn src.my_ai_app.main:app --reload --port 800010. Verifying Your Setup
# verify_setup.py
import sys
import importlib
print(f"Python version: {sys.version}")
print(f"Python executable: {sys.executable}")
required = ["openai", "httpx", "pydantic", "dotenv", "structlog", "tenacity"]
for package in required:
try:
mod = importlib.import_module(package)
version = getattr(mod, "__version__", "unknown")
print(f" {package}: {version}")
except ImportError:
print(f" {package}: NOT INSTALLED")Run it:
python verify_setup.pyExpected output:
Python version: 3.12.3 (main, ...)
Python executable: /home/user/projects/my-ai-app/.venv/bin/python
openai: 1.30.1
httpx: 0.27.0
pydantic: 2.7.1
dotenv: 1.0.1
structlog: 24.2.0
tenacity: 8.3.011. Quick-Start Script
#!/usr/bin/env bash
# setup.sh ā run once after cloning
set -e
echo "Creating virtual environment..."
python -m venv .venv
echo "Activating..."
source .venv/bin/activate
echo "Upgrading pip..."
pip install --upgrade pip
echo "Installing dependencies..."
pip install -r requirements-dev.txt
echo "Copying .env template..."
if [ ! -f .env ]; then
cp .env.example .env
echo "Edit .env and fill in your API keys."
fi
echo "Done. Run: source .venv/bin/activate"Summary
| Tool | Purpose | |---|---| | pyenv | Manage multiple Python versions | | venv | Isolate project dependencies | | pip | Install packages | | requirements.txt | Pin exact versions for reproducibility | | python-dotenv | Load .env files into os.environ | | Pydantic Settings | Type-safe, validated config objects | | .env.example | Safe template to commit to git |
A proper setup takes under 10 minutes and saves hours of debugging environment-related issues later. Always activate your venv before running any Python command in a project.
Found this helpful?
Leave a comment
Have a question, correction, or just found this helpful? Leave a note below.