Troubleshooting

Troubleshooting

When something breaks in your homelab, systematic diagnosis beats random reboots. This guide walks through common issues and how to read the logs to find the real problem.

Note

Always check logs first. Most issues leave a trail. The trick is knowing where to look and what to read.

Systemd Services

Check Service Status

sudo systemctl status docker

Output shows if the service is running, when it started, and recent log lines.

View Full Logs

# Last 50 lines
sudo journalctl -u docker -n 50

# Follow logs in real-time
sudo journalctl -u docker -f

# Logs since last boot
sudo journalctl -u docker -b

# Logs for the last hour
sudo journalctl -u docker --since "1 hour ago"

Restart a Service

sudo systemctl restart docker

Wait a few seconds, then check status:

sudo systemctl status docker

Docker Container Issues

Container Won’t Start

Check the logs:

docker logs container-name

Common causes and fixes:

Issue	Check	Fix
Port in use	sudo lsof -i :8080	Change port or stop conflicting service
Missing image	docker images	Run docker pull image-name
Permission denied	docker logs container	Fix volume ownership with chown
Out of memory	docker stats	Increase memory or reduce container limits

Container Exits Immediately

docker compose logs app

Look for error messages in the output. Common patterns:

Error: Cannot find config file at /app/config.yaml
Error: Database connection refused
Error: Port 8080 already in use

Network Connectivity Issues

Test if containers can reach each other:

docker compose exec app ping db

If this fails:

# Check networks
docker network ls

# Inspect the network
docker network inspect app-network

# Check if containers are on the network
docker inspect app | grep -A 20 NetworkSettings

Network Diagnostics

Check Network Interfaces

ip addr show

Look for your server’s IP address and interface status.

Test Connectivity

# Ping a remote host
ping -c 4 8.8.8.8

# Check DNS resolution
nslookup google.com
dig google.com

# Trace route to a host
traceroute google.com

Check Open Ports

# Show all listening ports
sudo ss -tlnp

# Check specific port
sudo lsof -i :8080

Firewall Rules

# Check UFW status
sudo ufw status verbose

# List all rules
sudo ufw show added

# Test if port is open
sudo ufw allow 8080/tcp

Disk Space Issues

Check Disk Usage

# Overall disk usage
df -h

# Directory size
du -sh /srv/apps/*

# Find large files
find /srv -type f -size +1G

Clean Up Docker

# Show disk usage
docker system df

# Remove unused images
docker image prune -a

# Remove unused volumes
docker volume prune

# Remove everything unused
docker system prune -a

Caution

docker system prune -a removes all unused images, containers, and networks. This is usually safe but can be destructive if you rely on old images.

Memory and CPU Issues

Check Resource Usage

# System-wide
free -h
top

# Docker containers
docker stats

Identify Resource Hogs

# Find processes using most CPU
ps aux --sort=-%cpu | head -10

# Find processes using most memory
ps aux --sort=-%mem | head -10

Limit Container Resources

Edit compose.yaml:

services:
  app:
    image: myapp:latest
    deploy:
      resources:
        limits:
          cpus: '1'
          memory: 512M

SSH and Remote Access

Can’t Connect via SSH

# Check SSH is running
sudo systemctl status ssh

# Check SSH is listening
sudo ss -tlnp | grep ssh

# Try verbose connection
ssh -v user@server

Permission Denied

# Check key permissions
ls -la ~/.ssh/

# Should be:
# -rw------- id_ed25519
# -rw-r--r-- id_ed25519.pub
# drwx------ .ssh

# Fix permissions
chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_ed25519
chmod 644 ~/.ssh/id_ed25519.pub

Locked Out

If you can’t connect via SSH:

1. Use console access or recovery mode
2. Check /etc/ssh/sshd_config for errors
3. Verify ~/.ssh/authorized_keys has your public key
4. Check file permissions (see above)
5. Restart SSH: sudo systemctl restart ssh

Database Issues

PostgreSQL Won’t Connect

# Check if container is running
docker compose ps

# Check logs
docker compose logs db

# Try connecting from another container
docker compose exec app psql -h db -U postgres -c "SELECT 1"

Database Corruption

# Check database integrity
docker compose exec db pg_checksum -D /var/lib/postgresql/data

# Rebuild indexes
docker compose exec db psql -U postgres -d myapp -c "REINDEX DATABASE myapp"

Backup and Restore

# Backup
docker compose exec db pg_dump -U postgres myapp > backup.sql

# Restore
docker compose exec -T db psql -U postgres myapp < backup.sql

Performance Tuning

Slow Docker Commands

# Check Docker daemon
docker info

# Restart Docker
sudo systemctl restart docker

# Check disk I/O
iostat -x 1 5

Slow Network

# Check bandwidth
iperf3 -c remote-host

# Monitor network
iftop

# Check for packet loss
ping -c 100 remote-host | grep loss

Incident Response Template

When something breaks, follow this template:

Time: [when did it happen?]
Service: [what broke?]
Impact: [what's not working?]

Symptoms:
- [what did you observe?]
- [what error messages?]

Diagnosis:
- Checked: [what did you look at?]
- Found: [what was wrong?]

Fix:
- Action: [what did you do?]
- Result: [did it work?]

Prevention:
- Root cause: [why did this happen?]
- Next steps: [how to prevent this?]

Tip

Keep these notes in a shared location. When the same issue happens again, you’ll have the solution ready.

Common Error Messages

Error	Cause	Fix
Connection refused	Service not running or port wrong	Check service status, verify port
Permission denied	File permissions or user access	Fix ownership with chown or chmod
Out of memory	Container needs more RAM	Increase memory limit or reduce workload
Disk full	No space left on device	Clean up old files or add storage
DNS resolution failed	Network or DNS issue	Check /etc/resolv.conf, test DNS

Getting Help

When you’re stuck:

1. Check the logs — Most issues are documented there
2. Search the error message — Someone has probably hit this before
3. Simplify the test — Isolate the problem to one component
4. Document what you tried — This helps others help you

See the Ubuntu Server First Steps for basic setup issues.