Back to blog
Cloud & DevOpsbeginner

Linux Project: Harden & Automate a Production Server

A practical guide to hardening a Linux server: SSH security, firewall setup, automatic updates, fail2ban, log monitoring, and deployment automation with bash scripts.

Asma HafeezApril 17, 20265 min read
linuxsecuritybashsshfirewalldevops
Share:𝕏

Linux Project: Harden & Automate a Production Server

This project takes a fresh Ubuntu 24.04 LTS server and hardens it against common attacks while setting up automation for updates and monitoring.


1. Initial Server Setup

Bash
# Connect to your server
ssh root@your-server-ip

# Create a non-root user
adduser deploy
usermod -aG sudo deploy

# Test the new user
su - deploy
sudo whoami  # should print "root"

2. SSH Hardening

Bash
# Generate SSH key on your local machine
ssh-keygen -t ed25519 -C "deploy@myserver" -f ~/.ssh/myserver_ed25519

# Copy public key to server
ssh-copy-id -i ~/.ssh/myserver_ed25519.pub deploy@your-server-ip

# Test key authentication works
ssh -i ~/.ssh/myserver_ed25519 deploy@your-server-ip

# Now harden SSH config on the server
sudo nano /etc/ssh/sshd_config

/etc/ssh/sshd_config changes

# Disable root login
PermitRootLogin no

# Disable password authentication (keys only)
PasswordAuthentication no
PubkeyAuthentication yes

# Disable empty passwords
PermitEmptyPasswords no

# Change default port (optional but reduces noise)
Port 2222

# Limit login grace time
LoginGraceTime 30

# Only allow specific users
AllowUsers deploy
Bash
# Restart SSH
sudo systemctl restart sshd

# Test from another terminal before closing current session!
ssh -p 2222 -i ~/.ssh/myserver_ed25519 deploy@your-server-ip

3. Firewall with UFW

Bash
# Install UFW (usually pre-installed)
sudo apt install ufw -y

# Default: deny incoming, allow outgoing
sudo ufw default deny incoming
sudo ufw default allow outgoing

# Allow your custom SSH port
sudo ufw allow 2222/tcp

# Allow web traffic
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp

# Enable firewall
sudo ufw enable

# Check status
sudo ufw status verbose

4. Automatic Security Updates

Bash
sudo apt install unattended-upgrades -y
sudo dpkg-reconfigure --priority=low unattended-upgrades
Bash
# Configure /etc/apt/apt.conf.d/50unattended-upgrades
sudo nano /etc/apt/apt.conf.d/50unattended-upgrades
// Auto-update security packages
Unattended-Upgrade::Allowed-Origins {
    "${distro_id}:${distro_codename}-security";
};

// Auto-remove unused packages
Unattended-Upgrade::Remove-Unused-Packages "true";

// Reboot automatically if needed (at 2am)
Unattended-Upgrade::Automatic-Reboot "true";
Unattended-Upgrade::Automatic-Reboot-Time "02:00";

5. Fail2Ban — Block Brute Force Attacks

Bash
sudo apt install fail2ban -y

# Create local config (don't edit the original)
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
sudo nano /etc/fail2ban/jail.local
INI
[DEFAULT]
# Ban for 1 hour after 5 failures
bantime  = 3600
findtime = 600
maxretry = 5

[sshd]
enabled = true
port    = 2222
Bash
sudo systemctl enable fail2ban
sudo systemctl start fail2ban

# Check banned IPs
sudo fail2ban-client status sshd

6. System Monitoring Script

Bash
# /home/deploy/scripts/health_check.sh
#!/bin/bash

LOG_FILE="/var/log/health_check.log"
ALERT_THRESHOLD_CPU=80    # percent
ALERT_THRESHOLD_DISK=85   # percent
ALERT_THRESHOLD_MEM=90    # percent

log() {
    echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE"
}

# CPU usage
CPU_USAGE=$(top -bn1 | grep "Cpu(s)" | awk '{print int($2)}')
if [ "$CPU_USAGE" -gt "$ALERT_THRESHOLD_CPU" ]; then
    log "ALERT: CPU usage is ${CPU_USAGE}%"
else
    log "OK: CPU usage is ${CPU_USAGE}%"
fi

# Disk usage
DISK_USAGE=$(df / | tail -1 | awk '{print int($5)}')
if [ "$DISK_USAGE" -gt "$ALERT_THRESHOLD_DISK" ]; then
    log "ALERT: Disk usage is ${DISK_USAGE}%"
else
    log "OK: Disk usage is ${DISK_USAGE}%"
fi

# Memory usage
MEM_TOTAL=$(free | grep Mem | awk '{print $2}')
MEM_USED=$(free | grep Mem | awk '{print $3}')
MEM_PERCENT=$(( MEM_USED * 100 / MEM_TOTAL ))
if [ "$MEM_PERCENT" -gt "$ALERT_THRESHOLD_MEM" ]; then
    log "ALERT: Memory usage is ${MEM_PERCENT}%"
else
    log "OK: Memory usage is ${MEM_PERCENT}%"
fi

# Check if critical services are running
for SERVICE in nginx postgresql; do
    if systemctl is-active --quiet "$SERVICE"; then
        log "OK: $SERVICE is running"
    else
        log "ALERT: $SERVICE is NOT running"
    fi
done
Bash
chmod +x /home/deploy/scripts/health_check.sh

# Run every 5 minutes via cron
crontab -e
# Add:
# */5 * * * * /home/deploy/scripts/health_check.sh

7. Deployment Automation Script

Bash
# /home/deploy/scripts/deploy.sh
#!/bin/bash
set -euo pipefail  # exit on error, undefined var, pipe failure

APP_DIR="/var/www/myapp"
REPO="https://github.com/yourname/myapp.git"
BRANCH="${1:-main}"
BACKUP_DIR="/var/backups/myapp"

log() { echo "[$(date '+%H:%M:%S')] $1"; }

log "Starting deployment of branch: $BRANCH"

# Backup current version
if [ -d "$APP_DIR" ]; then
    BACKUP="$BACKUP_DIR/backup_$(date '+%Y%m%d_%H%M%S')"
    mkdir -p "$BACKUP_DIR"
    cp -r "$APP_DIR" "$BACKUP"
    log "Backed up to $BACKUP"
fi

# Pull latest code
if [ -d "$APP_DIR/.git" ]; then
    cd "$APP_DIR"
    git fetch origin
    git checkout "$BRANCH"
    git pull origin "$BRANCH"
else
    git clone --branch "$BRANCH" "$REPO" "$APP_DIR"
    cd "$APP_DIR"
fi

log "Code updated to $(git rev-parse --short HEAD)"

# Install dependencies and build
if [ -f "package.json" ]; then
    npm ci --production
    log "Node dependencies installed"
fi

if [ -f "requirements.txt" ]; then
    pip install -r requirements.txt
    log "Python dependencies installed"
fi

# Restart service
sudo systemctl restart myapp
sudo systemctl is-active --quiet myapp && log "Service restarted OK" || {
    log "ERROR: Service failed to start — rolling back"
    cp -r "$BACKUP/." "$APP_DIR/"
    sudo systemctl restart myapp
    exit 1
}

log "Deployment complete!"

8. Log Rotation

Bash
# /etc/logrotate.d/myapp
/var/log/myapp/*.log {
    daily
    missingok
    rotate 14
    compress
    delaycompress
    notifempty
    create 0640 deploy deploy
    sharedscripts
    postrotate
        systemctl reload myapp > /dev/null 2>&1 || true
    endscript
}

Security Checklist

[ ] SSH: key-only auth, root login disabled, non-default port
[ ] Firewall: UFW enabled, only necessary ports open
[ ] Fail2ban: installed and monitoring SSH
[ ] Unattended upgrades: security patches auto-installed
[ ] Non-root user: app runs as dedicated user, not root
[ ] File permissions: web root not world-writable
[ ] HTTPS: TLS certificate (Let's Encrypt via certbot)
[ ] Monitoring: health checks running via cron
[ ] Log rotation: logs don't fill disk
[ ] Backups: automated and tested

Key Takeaways

  1. Disable password SSH immediately — key-only auth eliminates most automated attacks
  2. UFW default deny — only open ports you explicitly need
  3. Fail2ban handles brute force without manual IP banning
  4. Use set -euo pipefail in deployment scripts — fail fast and loud
  5. Always test rollback before you need it in production

Enjoyed this article?

Explore the Cloud & DevOps learning path for more.

Found this helpful?

Share:𝕏

Leave a comment

Have a question, correction, or just found this helpful? Leave a note below.