Learnixo
Back to blog
Backend Systemsbeginner

Python Environment Setup: venv, pip, and Project Structure

Master Python environment setup from scratch — virtual environments, pip, pyproject.toml, poetry, and the project layout every professional Python dev uses.

LearnixoMay 7, 20267 min read
Pythonvenvpippoetrypyproject.tomlenvironment
Share:𝕏

Why Environment Setup Matters

Every broken Python project starts the same way: pip install pandas run globally, three projects fighting over package versions, "it works on my machine" becoming a weekly complaint.

Professional Python work means one isolated environment per project. This lesson sets you up the right way from day one.

1. Python Version Management with pyenv

Before creating any environment, control which Python version you're running.

Install pyenv (Linux/macOS):

Bash
curl https://pyenv.run | bash

Install pyenv-win (Windows):

POWERSHELL
pip install pyenv-win --target $HOME\.pyenv

Common pyenv commands:

Bash
pyenv install 3.12.3        # install a specific version
pyenv global 3.12.3         # set default for your shell
pyenv local 3.11.9          # set version for this directory only (writes .python-version)
pyenv versions              # list all installed versions
python --version            # verify active version

A .python-version file checked into your repo pins the Python version for every collaborator automatically.

2. Virtual Environments with venv

venv is built into Python 3.3+. No install needed.

Creating and Activating

Bash
# create
python -m venv .venv

# activate — Linux/macOS
source .venv/bin/activate

# activate — Windows PowerShell
.venv\Scripts\Activate.ps1

# activate — Windows CMD
.venv\Scripts\activate.bat

# verify — you should see (.venv) in your prompt
which python          # Linux/macOS: /your/project/.venv/bin/python
where python          # Windows

Deactivating

Bash
deactivate

What's Inside .venv

.venv/
  bin/         # python, pip, and installed script entry points
  lib/         # site-packages — where installed packages live
  pyvenv.cfg   # metadata: which Python version, base prefix

Always add .venv/ to .gitignore. Never commit it.

3. pip: Installing and Managing Packages

Basic Operations

Bash
pip install requests                  # latest version
pip install requests==2.31.0          # pin exact version
pip install "requests>=2.28,<3"       # version range
pip install -r requirements.txt       # install from file
pip install -e .                      # editable install (local package)
pip uninstall requests                # remove
pip list                              # show installed packages
pip show requests                     # details on one package
pip outdated                          # packages with newer versions

requirements.txt

Bash
pip freeze > requirements.txt         # capture current state
pip install -r requirements.txt       # restore exact state

A requirements.txt locks your full dependency tree including transitive dependencies. Good for deployment, not great for library development (use version ranges in pyproject.toml instead).

requirements split by environment

requirements/
  base.txt        # runtime dependencies
  dev.txt         # -r base.txt, plus: pytest, black, ruff, mypy
  prod.txt        # -r base.txt, plus: gunicorn, sentry-sdk
Bash
pip install -r requirements/dev.txt

4. pyproject.toml: The Modern Standard

pyproject.toml replaces setup.py and setup.cfg. It's the single config file for your project.

TOML
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "my-pipeline-tool"
version = "0.1.0"
description = "Internal ETL pipeline toolkit"
requires-python = ">=3.11"
dependencies = [
    "requests>=2.28",
    "pandas>=2.0",
    "pydantic>=2.0",
    "typer>=0.12",
]

[project.optional-dependencies]
dev = [
    "pytest>=8.0",
    "pytest-cov",
    "ruff",
    "mypy",
    "black",
]

[project.scripts]
my-tool = "my_pipeline_tool.cli:app"   # CLI entry point

[tool.ruff]
line-length = 100
select = ["E", "F", "I", "UP"]

[tool.mypy]
strict = true
python_version = "3.11"

[tool.pytest.ini_options]
testpaths = ["tests"]

Install in editable mode with all dev extras:

Bash
pip install -e ".[dev]"

5. Poetry: Dependency Management Done Right

Poetry is the most popular tool for managing Python project dependencies with a lockfile.

Install Poetry

Bash
curl -sSL https://install.python-poetry.org | python3 -

Start a new project

Bash
poetry new my-project     # creates folder with structure
cd my-project
poetry shell              # activates the managed venv

Add dependencies

Bash
poetry add requests pandas          # add runtime deps
poetry add --group dev pytest ruff  # add dev-only deps
poetry add "fastapi>=0.110"         # with version constraint
poetry remove requests              # remove a dep

Install from lockfile

Bash
poetry install                      # install everything
poetry install --only main          # skip dev deps (CI/prod)

Key files Poetry manages

pyproject.toml        # your declared dependencies (human-edited)
poetry.lock           # exact pinned versions of everything (commit this!)

The lockfile guarantees identical installs across every machine. Always commit poetry.lock.

Common workflow

Bash
poetry install          # after cloning a repo
poetry add <pkg>        # add a new dep
poetry update           # update deps within constraints
poetry run pytest       # run a command inside the venv
poetry build            # build a distributable package
poetry publish          # publish to PyPI

6. Project Structure for Professional Python

This is the layout used for automation tools, pipelines, and internal packages:

my-project/
├── .python-version          # pins Python version (pyenv)
├── .venv/                   # virtual environment (git-ignored)
├── .env                     # local secrets (git-ignored)
├── .env.example             # template for .env (committed)
├── .gitignore
├── pyproject.toml           # project metadata + tool config
├── poetry.lock              # pinned dependency tree
├── README.md
│
├── src/
│   └── my_project/          # main package (use underscores)
│       ├── __init__.py
│       ├── cli.py           # Typer CLI entry point
│       ├── config.py        # Settings from environment
│       ├── pipeline.py      # core logic
│       └── utils.py         # shared helpers
│
├── tests/
│   ├── conftest.py          # pytest fixtures
│   ├── test_pipeline.py
│   └── test_utils.py
│
├── scripts/
│   └── seed_data.py         # one-off scripts, not part of package
│
└── Makefile                 # common task shortcuts

Minimal Makefile

MAKEFILE
.PHONY: install dev test lint typecheck

install:
	poetry install --only main

dev:
	poetry install

test:
	poetry run pytest --cov=src --cov-report=term-missing

lint:
	poetry run ruff check src tests

typecheck:
	poetry run mypy src

format:
	poetry run black src tests
	poetry run ruff check --fix src tests

7. Environment Variables and .env

Never hardcode secrets or environment-specific config.

.env file

Bash
DATABASE_URL=postgresql://user:pass@localhost/mydb
API_KEY=sk-secret-key
DEBUG=true
LOG_LEVEL=INFO

Loading .env with python-dotenv

Python
from dotenv import load_dotenv
import os

load_dotenv()   # reads .env from current directory

db_url = os.environ["DATABASE_URL"]
debug = os.getenv("DEBUG", "false").lower() == "true"

Typed settings with Pydantic

Python
from pydantic_settings import BaseSettings

class Settings(BaseSettings):
    database_url: str
    api_key: str
    debug: bool = False
    log_level: str = "INFO"

    class Config:
        env_file = ".env"

settings = Settings()   # validates and loads at startup
print(settings.database_url)

8. Code Quality Tools

Install these in every project:

| Tool | Purpose | Command | |------|---------|---------| | ruff | Linting + import sorting (fast) | ruff check . | | black | Code formatting | black . | | mypy | Static type checking | mypy src | | pytest | Testing | pytest | | pytest-cov | Coverage | pytest --cov=src |

Pre-commit hooks (automated quality gates)

Bash
pip install pre-commit

.pre-commit-config.yaml:

YAML
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.4.0
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format

  - repo: https://github.com/psf/black
    rev: 24.3.0
    hooks:
      - id: black
Bash
pre-commit install    # installs hooks into .git/hooks
pre-commit run --all-files    # run manually

Every git commit now auto-lints and formats your code.

9. Checklist: Starting a New Python Project

Copy this every time:

Bash
# 1. Set Python version
pyenv local 3.12.3

# 2. Create project with poetry
poetry new my-project && cd my-project

# 3. Add runtime deps
poetry add requests pandas pydantic typer

# 4. Add dev deps
poetry add --group dev pytest pytest-cov ruff black mypy

# 5. Create src layout
mkdir -p src/my_project tests
touch src/my_project/__init__.py tests/conftest.py

# 6. Create .env.example
echo "DATABASE_URL=\nAPI_KEY=" > .env.example
cp .env.example .env

# 7. Install pre-commit
poetry add --group dev pre-commit
poetry run pre-commit install

# 8. Verify
poetry run python -c "import sys; print(sys.version)"
poetry run pytest

Summary

| Concept | Tool | Key Command | |--------|------|------------| | Python version | pyenv | pyenv local 3.12.3 | | Isolated env | venv / poetry | python -m venv .venv | | Install packages | pip / poetry | pip install / poetry add | | Lock dependencies | poetry.lock | poetry install | | Project config | pyproject.toml | single file for all tools | | Secrets | python-dotenv | load_dotenv() | | Linting | ruff | ruff check . | | Type checking | mypy | mypy src |

You now have the foundation that every professional Python project needs. Next up: writing clean, typed Python functions.

Enjoyed this article?

Explore the Backend Systems learning path for more.

Subscribe for weekly tutorials

Found this helpful?

Share:𝕏

Leave a comment

Have a question, correction, or just found this helpful? Leave a note below.