Learnixo
Back to blog
AI Systemsbeginner

Python Setup for AI Engineering

Set up a professional Python environment for AI projects: pyenv, venv, pip, requirements.txt, .env files, python-dotenv, and a clean project structure.

Asma Hafeez KhanMay 15, 20266 min read
PythonAI EngineeringSetupEnvironmentpyenvvenv
Share:š•

Python Setup for AI Engineering

A clean environment is the foundation of every reliable AI project. This lesson walks you through the full setup: managing Python versions with pyenv, isolating dependencies with venv, pinning packages, loading secrets from .env files, and structuring an AI application so it is maintainable from day one.


1. Managing Python Versions with pyenv

AI libraries often require specific Python versions. pyenv lets you install and switch between multiple versions without touching your system Python.

Bash
# Install pyenv (macOS / Linux)
curl https://pyenv.run | bash

# Add to ~/.bashrc or ~/.zshrc
export PYENV_ROOT="$HOME/.pyenv"
export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"

# Install a specific version
pyenv install 3.12.3

# Set a version globally
pyenv global 3.12.3

# Set a version for a single project directory
cd ~/projects/my-ai-app
pyenv local 3.12.3        # writes .python-version file

# Verify
python --version           # Python 3.12.3

On Windows, use the official Python installer from python.org or pyenv-win:

POWERSHELL
# Install pyenv-win
Invoke-WebRequest -UseBasicParsing -Uri "https://raw.githubusercontent.com/pyenv-win/pyenv-win/master/pyenv-win/install-pyenv-win.ps1" -OutFile "./install-pyenv-win.ps1"
.\install-pyenv-win.ps1

pyenv install 3.12.3
pyenv global 3.12.3

2. Creating and Activating a Virtual Environment

Never install project dependencies into your global Python. Always use a virtual environment.

Bash
# Create a venv named .venv in the project root
python -m venv .venv

# Activate on Linux / macOS
source .venv/bin/activate

# Activate on Windows PowerShell
.\.venv\Scripts\Activate.ps1

# You should see (.venv) in your prompt
# Deactivate when done
deactivate

Why .venv as the name?

  • Most .gitignore templates already exclude .venv
  • IDEs (VS Code, PyCharm) auto-detect it
  • Keeps the project root clean

3. Installing Packages with pip

Bash
# Always upgrade pip first
pip install --upgrade pip

# Install packages
pip install openai httpx pydantic python-dotenv

# Install a specific version (important for reproducibility)
pip install openai==1.30.1

# Install from a requirements file
pip install -r requirements.txt

# Install in editable mode (for your own packages)
pip install -e .

4. Pinning Dependencies with requirements.txt

Bash
# After installing everything your project needs:
pip freeze > requirements.txt

A typical requirements.txt for an AI project:

TEXT
# requirements.txt
openai==1.30.1
httpx==0.27.0
pydantic==2.7.1
pydantic-settings==2.2.1
python-dotenv==1.0.1
structlog==24.2.0
tenacity==8.3.0
redis==5.0.4
sqlalchemy==2.0.30
alembic==1.13.1
fastapi==0.111.0
uvicorn[standard]==0.30.0
pytest==8.2.0
pytest-asyncio==0.23.7

Separating dev from production dependencies

TEXT
# requirements-dev.txt
-r requirements.txt          # include all production deps
pytest==8.2.0
pytest-asyncio==0.23.7
black==24.4.2
ruff==0.4.4
mypy==1.10.0

Install dev dependencies in development:

Bash
pip install -r requirements-dev.txt

5. Loading Secrets from .env Files

Never hardcode API keys. Store them in a .env file and load them at runtime.

Bash
# .env  (never commit this file!)
OPENAI_API_KEY=sk-proj-...
AZURE_OPENAI_ENDPOINT=https://my-resource.openai.azure.com/
AZURE_OPENAI_API_KEY=...
AZURE_OPENAI_DEPLOYMENT=gpt-4o
REDIS_URL=redis://localhost:6379
DATABASE_URL=postgresql+asyncpg://user:pass@localhost/mydb
APP_ENV=development
LOG_LEVEL=DEBUG

Add .env to .gitignore immediately:

Bash
echo ".env" >> .gitignore
echo ".env.local" >> .gitignore

Provide a template others can copy:

Bash
# .env.example  (commit this)
OPENAI_API_KEY=your-key-here
AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
AZURE_OPENAI_API_KEY=your-azure-key-here
AZURE_OPENAI_DEPLOYMENT=gpt-4o
REDIS_URL=redis://localhost:6379
DATABASE_URL=postgresql+asyncpg://user:pass@localhost/mydb
APP_ENV=development
LOG_LEVEL=INFO

6. python-dotenv

Python
# config.py
from dotenv import load_dotenv
import os

# Load .env from the current directory (or parent dirs)
load_dotenv()

OPENAI_API_KEY = os.environ["OPENAI_API_KEY"]
APP_ENV = os.getenv("APP_ENV", "production")   # with default
LOG_LEVEL = os.getenv("LOG_LEVEL", "INFO")

print(f"Running in {APP_ENV} mode")

Load a specific file:

Python
load_dotenv(".env.local")       # override with local file
load_dotenv(override=True)      # env vars in the file override existing env vars

7. Pydantic Settings (Recommended for AI Apps)

Pydantic Settings gives you type-safe config with automatic validation:

Python
# settings.py
from pydantic_settings import BaseSettings, SettingsConfigDict
from pydantic import Field


class Settings(BaseSettings):
    model_config = SettingsConfigDict(
        env_file=".env",
        env_file_encoding="utf-8",
        case_sensitive=False,
    )

    # Required — will raise if missing
    openai_api_key: str

    # Optional with defaults
    azure_openai_endpoint: str = ""
    azure_openai_deployment: str = "gpt-4o"
    redis_url: str = "redis://localhost:6379"
    database_url: str = ""

    app_env: str = "production"
    log_level: str = "INFO"
    max_tokens: int = Field(default=1024, ge=1, le=8192)


# Singleton — import this everywhere
settings = Settings()

Usage:

Python
# main.py
from settings import settings
import openai

client = openai.OpenAI(api_key=settings.openai_api_key)
print(f"Using model: {settings.azure_openai_deployment}")

8. Project Structure for an AI Application

my-ai-app/
ā”œā”€ā”€ .env                    # secrets — never commit
ā”œā”€ā”€ .env.example            # template — commit this
ā”œā”€ā”€ .gitignore
ā”œā”€ā”€ .python-version         # pyenv version pin
ā”œā”€ā”€ requirements.txt
ā”œā”€ā”€ requirements-dev.txt
ā”œā”€ā”€ pyproject.toml          # optional: modern packaging
ā”œā”€ā”€ README.md
│
ā”œā”€ā”€ src/
│   └── my_ai_app/
│       ā”œā”€ā”€ __init__.py
│       ā”œā”€ā”€ settings.py     # Pydantic Settings
│       ā”œā”€ā”€ main.py         # FastAPI app or entry point
│       │
│       ā”œā”€ā”€ agents/         # Agent logic
│       │   ā”œā”€ā”€ __init__.py
│       │   └── assistant.py
│       │
│       ā”œā”€ā”€ tools/          # Tool implementations
│       │   ā”œā”€ā”€ __init__.py
│       │   ā”œā”€ā”€ search.py
│       │   └── calendar.py
│       │
│       ā”œā”€ā”€ models/         # Pydantic models / DB models
│       │   ā”œā”€ā”€ __init__.py
│       │   ā”œā”€ā”€ schemas.py
│       │   └── db.py
│       │
│       ā”œā”€ā”€ services/       # Business logic
│       │   ā”œā”€ā”€ __init__.py
│       │   └── llm_service.py
│       │
│       └── utils/          # Shared utilities
│           ā”œā”€ā”€ __init__.py
│           ā”œā”€ā”€ logging.py
│           └── retry.py
│
ā”œā”€ā”€ tests/
│   ā”œā”€ā”€ conftest.py
│   ā”œā”€ā”€ test_agents/
│   └── test_tools/
│
└── docs/

9. Makefile for Common Commands

MAKEFILE
# Makefile
.PHONY: install dev-install test lint format run

install:
	pip install --upgrade pip
	pip install -r requirements.txt

dev-install:
	pip install --upgrade pip
	pip install -r requirements-dev.txt

test:
	pytest tests/ -v

lint:
	ruff check src/
	mypy src/

format:
	black src/ tests/
	ruff check --fix src/

run:
	uvicorn src.my_ai_app.main:app --reload --port 8000

10. Verifying Your Setup

Python
# verify_setup.py
import sys
import importlib

print(f"Python version: {sys.version}")
print(f"Python executable: {sys.executable}")

required = ["openai", "httpx", "pydantic", "dotenv", "structlog", "tenacity"]

for package in required:
    try:
        mod = importlib.import_module(package)
        version = getattr(mod, "__version__", "unknown")
        print(f"  {package}: {version}")
    except ImportError:
        print(f"  {package}: NOT INSTALLED")

Run it:

Bash
python verify_setup.py

Expected output:

Python version: 3.12.3 (main, ...)
Python executable: /home/user/projects/my-ai-app/.venv/bin/python
  openai: 1.30.1
  httpx: 0.27.0
  pydantic: 2.7.1
  dotenv: 1.0.1
  structlog: 24.2.0
  tenacity: 8.3.0

11. Quick-Start Script

Bash
#!/usr/bin/env bash
# setup.sh — run once after cloning

set -e

echo "Creating virtual environment..."
python -m venv .venv

echo "Activating..."
source .venv/bin/activate

echo "Upgrading pip..."
pip install --upgrade pip

echo "Installing dependencies..."
pip install -r requirements-dev.txt

echo "Copying .env template..."
if [ ! -f .env ]; then
  cp .env.example .env
  echo "Edit .env and fill in your API keys."
fi

echo "Done. Run: source .venv/bin/activate"

Summary

| Tool | Purpose | |---|---| | pyenv | Manage multiple Python versions | | venv | Isolate project dependencies | | pip | Install packages | | requirements.txt | Pin exact versions for reproducibility | | python-dotenv | Load .env files into os.environ | | Pydantic Settings | Type-safe, validated config objects | | .env.example | Safe template to commit to git |

A proper setup takes under 10 minutes and saves hours of debugging environment-related issues later. Always activate your venv before running any Python command in a project.

Enjoyed this article?

Explore the AI Systems learning path for more.

Found this helpful?

Share:š•

Leave a comment

Have a question, correction, or just found this helpful? Leave a note below.