Git Collaboration: Pull Requests, Code Reviews, and Branching Strategies

1. Create feature branch from main
2. Make commits
3. Push branch to remote
4. Open PR: describe changes, request reviewers
5. CI runs automatically (tests, lint, type check)
6. Code review: comments, requests for changes
7. Address feedback, push more commits
8. Approval from required reviewers
9. Squash + merge into main
10. Delete feature branch
11. CI deploys to staging/production

## What changed
- Added incremental ingestion for the orders table
- Switched from full reload to MERGE-based upsert
- Added row count validation between bronze and silver layers

## Why
The daily full reload was taking 45min and causing warehouse credits overuse.
Incremental reduces this to ~3min on average.

## How to test
1. Run `dbt run --select stg_orders --full-refresh` to rebuild
2. Run `dbt test --select stg_orders` — all tests should pass
3. Check the Snowflake warehouse in QUERY_HISTORY to verify query time

## Checklist
- [x] dbt tests pass
- [x] Row counts validated
- [x] Docs updated in schema.yml
- [x] No PII in logs

# Blocking (must fix before merge)
# Use a descriptive prefix to signal severity

BLOCKING: This will cause a divide-by-zero if quantity is 0.
Please add a guard: `WHERE quantity > 0` or handle in Python.

# Suggestion (non-blocking, your call)
NIT: We typically use `LOWER(TRIM(...))` for email normalization
across all our staging models. Feel free to keep as-is.

# Question (asking, not demanding)
QUESTION: Why are we filtering status = 'active' here?
The requirements doc said we should include 'pending' too.

# Praise (signal what's good)
NICE: Clean use of the SCD Type 2 merge pattern here.
This will serve as a good reference for the team.

main          ── always production-ready
develop       ── integration branch
feature/*     ── one branch per feature, branched from develop
release/*     ── stabilization before release
hotfix/*      ── urgent prod fixes, branched from main

# GitFlow workflow
git switch -c feature/orders-pipeline develop
# ... make commits ...
git switch develop
git merge --no-ff feature/orders-pipeline   # --no-ff preserves merge commit

# Release
git switch -c release/1.2.0 develop
# ... test, fix bugs ...
git switch main
git merge --no-ff release/1.2.0
git tag -a v1.2.0 -m "Release 1.2.0"
git switch develop
git merge --no-ff release/1.2.0

main          ── everyone commits here (or merges very short-lived feature branches)
feature/*     ── live max 1-2 days, then merged

# TBD workflow
git switch -c feature/add-validation main
# ... make focused commits in 1-2 days ...
git fetch origin && git rebase origin/main  # keep in sync daily
git push -u origin feature/add-validation
# Open PR, get fast review, merge

main branch protection:
  ✓ Require pull request reviews before merging
      Required approvals: 1-2
  ✓ Dismiss stale pull request approvals when new commits are pushed
  ✓ Require status checks to pass before merging
      Required status checks:
        - CI / test (ubuntu-latest)
        - dbt-ci / dbt-build
  ✓ Require branches to be up to date before merging
  ✓ Restrict who can push to matching branches
  ✓ Require signed commits (optional, high security)
  ✓ Include administrators

git fetch origin
git rebase origin/main   # or: git merge origin/main
# resolve conflicts in editor
git add conflicted_file.py
git rebase --continue    # or: git commit (for merge)
git push --force-with-lease origin feature/my-branch

git reset --soft HEAD~1           # undo commit, keep changes staged
git switch -c feature/proper-branch
git push origin feature/proper-branch

git cherry-pick a3f4c2d   # apply specific commit to current branch

git reflog                        # find the SHA of the last commit on that branch
git checkout -b recovered-branch SHA

# .git/hooks/pre-commit (runs before each commit)
#!/bin/bash
ruff check .            && \  # lint
mypy src/               && \  # type check
pytest tests/ -q -x         # tests (fast)

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.4.0
    hooks:
      - id: ruff
      - id: ruff-format

  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.6.0
    hooks:
      - id: trailing-whitespace
      - id: end-of-file-fixer
      - id: check-yaml
      - id: detect-private-key       # prevents committing API keys
      - id: check-merge-conflict     # catches unresolved <<<< markers

# Install
brew install gh   # macOS
winget install GitHub.cli  # Windows

gh auth login

# Create a PR
gh pr create \
  --title "feat: add orders incremental pipeline" \
  --body "See PR template" \
  --reviewer alice,bob \
  --label "data-engineering"

# Review PRs
gh pr list
gh pr view 42
gh pr checkout 42    # check out PR branch locally
gh pr review 42 --approve
gh pr merge 42 --squash --delete-branch

# Issues
gh issue create --title "Bug: null orders on weekends" --label bug
gh issue list --assignee @me

data-platform/
├── dbt/                  # dbt project
│   ├── models/
│   └── dbt_project.yml
├── airflow/              # Airflow DAGs
│   ├── dags/
│   └── plugins/
├── ingestion/            # Python ingestion scripts
│   ├── src/
│   └── tests/
├── infrastructure/       # Terraform
│   └── main.tf
├── .github/
│   └── workflows/
└── Makefile