Cron Jobs

Cron Jobs#


1. Create a backup Scheduler every hour between 8:00 and 22:00#

8..22/1:00

1.1. Check if it was triggered#

grep CRON /var/log/syslog | tail -n 5

Crontab commands#

contab -e

2. Cronjob for MacMini#

#https://crontab.guru/
#@reboot (sleep 60 && echo "ondemand" | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor)
@reboot setpci -s 0:1f.0 0xa4.b=0

#PVEBackup
30 3 * * * /bin/bash /mnt/pve/Software/Software/Scripts/Proxmox/proxmox_backup.sh > /dev/null 2>&1

#hp1
50 7 * * * /usr/bin/wakeonlan 94:57:a5:65:4e:90
55 7 * * * /usr/sbin/pvesm set HP1BackupServer -disable false
00 22 * * * /usr/sbin/pvesm set HP1BackupServer -disable true

#HP2
00 18 * * * /usr/bin/wakeonlan 94:57:a5:65:7e:88
#HP3
#50 8 * * 7 /usr/bin/wakeonlan ec:b1:d7:7c:a9:48
#HP4
00 13 * * * /usr/bin/wakeonlan 20:67:7c:ee:63:80
#HP6
50 8 * * * /usr/bin/wakeonlan ac:16:2d:6e:64:fc
#HP7
50 8 * * * /usr/bin/wakeonlan 80:30:e0:3b:a5:74 ​

#HP8OLD
#45 7 * * * /usr/bin/wakeonlan 3c:d9:2b:03:d8:f2

#HP9OLD
#45 7 * * * /usr/bin/wakeonlan 68:b5:99:77:17:c2

#HP10OLD
#30 8 * * * /usr/bin/wakeonlan 00:26:55:29:0d:cc

#MP5.2TrueNAS
#00 08 * * 7 /usr/sbin/pvesm set MP5.2Backups -disable false
#00 11 * * 7 /usr/sbin/pvesm set MP5.2Backups -disable true

3. Cronjob HP4#

30 18 * * * /usr/sbin/pvesm set HP4BackupServer -disable true                                      
15 12 * * * /usr/sbin/shutdown -h now
00 19 * * * /usr/sbin/shutdown -h now

4. Cron Job to Auto Start After Power Failure for MacMini (autostart)#

crontab -e (crontab -l for a list of jobs)

Disk Passthrough - fstab

Proxmox Disk Passthrough Guide#

The HP7TrueNAS passthrough entry below is VM 207 and remains validation-pending. The former HP6Proxmox host was decommissioned; 10.1.1.16 is now assigned to HP6TrueNAS, formerly HP7TrueNAS. Do not apply the VM 207 commands without revalidating the deployment. HP7BackupServer is a separate VM 217, now at 10.1.1.217.


1. Discovery Commands#

Use these commands to identify physical disks on the Proxmox host.

# Install hardware lister
apt install lshw

# List disks and storage controllers
lshw -class disk -class storage

# List all Physical Disk IDs (The ID needed for passthrough)
ls -l /dev/disk/by-id/
# Advanced list: Maps Device to ID and filters out partitions/LVM
lsblk |awk 'NR==1{print $0" DEVICE-ID(S)"}NR>1{dev=$1;printf $0" ";system("find /dev/disk/by-id -lname \"*"dev"\" -printf \" %p\"");print "";}'|grep -v -E 'part|lvm'

2. Configuration Requirements#

Adding Serials (Crucial for TrueNAS/ZFS)#

In TrueNAS manually edit the VM config file to ensure disk serials persist:

GPU Passthrough Guide

Proxmox GPU Passthrough Guide (NVIDIA Tesla P4)#


1. Proxmox Host Configuration#

Pre-requisites#

  • Ensure PCIe Bifurcation is set to OFF in the server BIOS.

Enable IOMMU#

  1. Open the GRUB configuration:
nano /etc/default/grub
  1. Modify the line GRUB_CMDLINE_LINUX_DEFAULT="quiet" to:
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"
  1. Update GRUB:
update-grub
  1. Reboot the host.

Verify IOMMU Status#

dmesg | grep -e DMAR -e IOMMU

Note: Look for the line: DMAR: IOMMU enabled.


2. VM Configuration (Ubuntu 24.04)#

VM Hardware Settings#

  • Machine: q35
  • BIOS: SeaBIOS
  • Note: Do NOT run apt update or upgrade yet.

Add PCIe Device#

  1. Shut down the VM.
  2. Go to Hardware > Add > PCI Device.
  3. Select Raw Device > Select your GPU.
  4. Ensure All Functions and PCI-Express are checked.

3. Driver Installation in VM#

Verify Hardware Detection#

lspci -v
lspci -nnk | grep -iA2 nvidia

Prepare Build Environment#

Required for version 550+ drivers:

Grafana & InfluxDB Installation

Grafana & InfluxDB Installation Guide#


1. Install Grafana#

Documentation: Grafana Debian Installation
Video Reference: Grafana Setup Guide

Install Prerequisites#

apt-get install -y apt-transport-https software-properties-common wget

Import GPG Key#

mkdir -p /etc/apt/keyrings/
wget -q -O - [https://apt.grafana.com/gpg.key](https://apt.grafana.com/gpg.key) | gpg --dearmor | sudo tee /etc/apt/keyrings/grafana.gpg > /dev/null

Add Stable Repository#

echo "deb [signed-by=/etc/apt/keyrings/grafana.gpg] [https://apt.grafana.com](https://apt.grafana.com) stable main" | tee -a /etc/apt/sources.list.d/grafana.list

Install Grafana OSS#

apt-get update
apt-get install grafana -y

Enable and Start Service#

systemctl daemon-reload
systemctl enable grafana-server
systemctl start grafana-server
systemctl status grafana-server

2. Install InfluxDB#

Documentation: InfluxDB Installation

Linux - Various Command Lines

Linux & Proxmox Command Line Toolbox#


1 Storage & Disk Management#

1.1 Clean Up & Space Recovery#

# Purge unused packages and clean cache
apt autoremove --purge && apt autoclean
# Reclaim journal log space (limit to 500MB)
journalctl --vacuum-size=500M

1.2 Identify Large Files & Folders#

# List all files with human-readable sizes
ls -lah
# Summary of folder sizes in current directory
du -sh *
# Identify top 10 largest folders on root
sudo du -xh / | sort -rh | head -n 10
# Disk space usage summary (filtered for MB/GB)
du -cha --max-depth=1 / | grep -E "M|G"

1.3 Hugo Project Specifics (Size Checks)#

# Find files larger than 25MB in project
find /home/marc/hugo/canada2argentina -type f -size +25M -exec ls -lh {} +
# Count total files in the public build folder
find /home/marc/hugo/canada2argentina/public -type f | wc -l
# List top 25 largest files in project
find /home/marc/hugo/canada2argentina -type f -exec du -h {} + | sort -rh | head -n 25

1.4 Disk Health & Partitions#

# Wipe all partitions/signatures from a disk (CAUTION)
sgdisk --zap-all /dev/sdX
# Detailed SMART health report
smartctl -a /dev/sda | less
# Deep scan for bad blocks (Takes a long time)
badblocks -v /dev/sda -s

2 Proxmox & Virtualization#

2.1 VM / Container Management#

# Unlock a stuck VM or Container
qm unlock <vmid>
pct unlock <ctid>
# Resize a container disk (Final size, cannot shrink)
pct resize 103 rootfs 16G
# List running LXC containers
pct list | awk 'NR>1'

2.2 High Availability (HA) Fixes#

# Disable HA for a specific VM/CT to allow manual fixes
ha-manager set vm:200 --state disabled
ha-manager set ct:103 --state disabled

2.3 Repository & Templates#

# Update and list available CT templates
pveam update
pveam available
# List all active apt sources
grep -r '' /etc/apt/sources.list*

2.4 Proxmox Boot & Kernel#

# List and pin a specific kernel
proxmox-boot-tool kernel list
proxmox-boot-tool kernel pin 6.8.8-4-pve

3 Networking & Connectivity#

3.1 Troubleshooting & Discovery#

# Check DNS resolution
nslookup google.com
nslookup pi.hole
# Check open listening ports
ss -lntu
netstat -plant
# Performance test (Client mode, 8 parallel streams)
iperf3 -c 10.1.1.11 -P8

3.2 SSH & Remote Access#

# View real-time system logs for USB/Kernel errors
dmesg -w
dmesg -T | grep usb
# Clear "Remote Host Identification Changed" error
nano ~/.ssh/known_hosts

3.3 File Transfers (SCP)#

# Copy local images from Mac to Hugo site on Server
scp -v /Users/marc/Downloads/to_convert/* root@10.1.2.202:/home/marc/hugo/canada2argentina/static/images/
# Backup and transfer a folder between servers
tar -czvf "$(date +%Y-%m-%d_%Hh%M)_PangolinBackup.tar.gz" pangolin/
scp pangolin.tar.gz root@72.11.147.220:/home/marc/docker-compose

4 System & Hardware Info#

4.1 RAM & CPU#

# Clear RAM cache (Drop caches)
sync && echo 3 | tee /proc/sys/vm/drop_caches
# Check ZFS ARC summary and free RAM
arc_summary -s arc && free -h

4.2 BIOS & Hardware#

# Get BIOS version and serial info
sudo dmidecode -t bios
# List SCSI and PCIe hardware
lsscsi -u
lspci

4.3 Legacy iLO3 Fix (Firefox)#

  1. Type about:config in address bar.
  2. Search: security.tls.version.enable-deprecated.
  3. Set to: true.

5 Miscellaneous Utilities#

5.1 Permissions & Scripts#

# Make script executable and run
chmod +x ./unifi-9.0.114.sh && ./unifi-9.0.114.sh

5.2 Scheduling & Processes#

# Instant test of cron behavior (Sends message to all terminals)
* * * * * echo "Hello from cron" | wall
# Cancel a pending shutdown
shutdown -c

5.3 Print the path of the folder you are in#

# Print the path of the folder you are in
pwd

6 SSD SMART Testing on Proxmox#

6.1 Identify the drive#

# Display the drive model, serial number, capacity, and type.
lsblk -d -o NAME,MODEL,SERIAL,SIZE,TYPE /dev/sdX

6.2 View the current SMART report#

# Display the complete SMART health report.
smartctl -x /dev/sdX

6.3 Run a short self-test#

# Start a short, read-only SSD self-test.
smartctl -t short /dev/sdX

6.4 Check the test status#

# Check whether the self-test is still running or has completed.
smartctl -c /dev/sdb
echo "===== /dev/sdb ====="; smartctl -c /dev/sdb | grep -E "Self-test execution|remaining"

6.5 Check the test status if many disks tests running#

#One disk check of test
echo "===== /dev/sdk ====="
smartctl -x /dev/sdk | grep -iE "Self-test execution status|remaining|Background long"
# Multi Check whether the self-test is still running or has completed.
for d in /dev/sdb /dev/sdc /dev/sdd /dev/sde; do
  echo "===== $d ====="
  smartctl -x "$d" | grep -iE "Self-test execution status|remaining|Background long"
done

6.6 Run an extended self-test#

# Start an extended, read-only SSD self-test.
smartctl -t long /dev/sdX
#many
for d in /dev/sda /dev/sdb /dev/sdc /dev/sdd /dev/sde; do
  echo "===== Starting long test on $d ====="
  smartctl -t long "$d"
done

6.7 Check the extended test status#

# Check whether the extended test is complete.
smartctl -c /dev/sdX

6.8 Abort#

# Stop the test (this will stop all test if multiple started together)
smartctl -X
#If multiple
for d in /dev/sda /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf /dev/sdg /dev/sdh; do
  echo "===== $d ====="
  smartctl -X "$d"
done

6.9 View the final report#

# Display the final SMART report, including self-test results.
smartctl -x /dev/sdX

6.10 Save the report#

# Save the SMART report to a text file.
smartctl -x /dev/sdX > sdX-smart-report.txt

Pangolin Backup Strategy

1 VPS BACKUP SYSTEM: Pangolin Docker#

Manual Backup Verification — 2026-10-02#

  • Triggered the existing backup script as vps-backup on VPSBackups LXC 102 (10.1.2.102) for RN-VPS (72.11.147.220); it completed successfully.
  • The current script stops the Pangolin stack, creates the archive on the VPS, restarts the stack, transfers the archive to LXC 102, and applies retention on both hosts.
  • Verified archive: /home/vps-backup/Backups/2026-10-02_08h27_PangolinBackup.tar.gz (313,781,104 bytes). The timestamp is the backup host’s local time.
  • The archive passed gzip -t; its file listing contains docker-compose.yml, config/db/db.sqlite, and config/traefik/traefik.yaml. A restore test was not performed.
  • All six Pangolin stack containers were running afterward; Pangolin and CrowdSec reported healthy. No component versions were changed.
  • Documentation discrepancy: /mnt/VPSBackups does not currently exist on LXC 102. The older storage and live-mirror references below have not been revalidated; the verified archive destination is /home/vps-backup/Backups/.

Target VPS: 72.11.147.220
VPS Provider: RackNerd (RN-VPS)
Backup LXC: Proxmox 102
Retention: 28 Files (14 Days)

Proxmox - Configuration

Proxmox - Configuration#

For fresh installations and initial server builds, please refer to the Proxmox - Initial Setup guide for automated provisioning steps.


1. Storage Management#

1.1 Storage: ZFS Pool Creation#

Create a simple mirrored or striped ZFS pool via CLI.

# Create a pool named 'Storage' using two disks
zpool create Storage sdb sdc

# The pool will now appear in Node > Storage > ZFS

1.2 Storage Specific Tests#

# Test SMB Share Throughput
dd if=/dev/zero of=/mnt/pve/HP1Storage/test.file bs=1M count=4000

# Test local-lvm Performance
dd if=/dev/zero of=/var/lib/vz/test.file bs=1M count=4000

2. Networking#

2.1 Add Second Network Bridge (Isolated VM Traffic)#

Use this for dedicated VM traffic on a specific physical port (e.g., eno6).

Proxmox - Node Backup and Restore

Proxmox VE Host Backup & Restore Guide#

This guide covers the automated backup and restoration process for all Proxmox VE nodes using the dedicated PVE-only scripts.

Proxmox VE 9 Network Interface Disconnection Warning

On fresh Proxmox VE 9 installations, interfaces are pinned automatically (e.g., nic0–nic5). Overwriting /etc/network/interfaces with a legacy backup containing older naming schemes (such as eno5, eno6, or eno49) will cause the network interfaces to fail on reboot, resulting in a total loss of Web UI and SSH access.

Proxmox Backup Server - Setup

Proxmox Backup Server (PBS)#


1 Installation & Post-Install#

1.1 Post-Install Optimization#

Run this script to configure repositories and optimize settings:

bash -c "$(wget -qLO - [https://github.com/community-scripts/ProxmoxVE/raw/main/misc/post-pbs-install.sh](https://github.com/community-scripts/ProxmoxVE/raw/main/misc/post-pbs-install.sh))"

1.2 Install File Browser#

bash -c "$(wget -qLO - [https://github.com/tteck/Proxmox/raw/main/misc/filebrowser.sh](https://github.com/tteck/Proxmox/raw/main/misc/filebrowser.sh))"

2 Setup VM with SMB Datastore#

2.1 Create Mount Directory#

mkdir /mnt/HP1ProxmoxBackupServer

2.2 Configure Credentials#

nano /etc/samba/.smbcreds

File Content:

username=proxmox
password=private

2.3 Set Permissions#

Change permission to Read-Only by root:

chmod 400 /etc/samba/.smbcreds

2.4 Manual Mount Test#

mount -t cifs -o rw,vers=3.0,credentials=/etc/samba/.smbcreds,uid=34,gid=34 //10.1.1.225/HP1ProxmoxBackupServer /mnt/HP1ProxmoxBackupServer

2.5 Permanent Mount (fstab)#

nano /etc/fstab

Add line:

Tailscale Installation & Subnet Routing

Tailscale Installation & Subnet Routing Guide#


1. Proxmox LXC Container Preparation#

If you are installing Tailscale inside an LXC container, you must allow TUN device access from the Proxmox host.

Modify Container Config#

On the Proxmox host, edit the configuration file for the specific container (e.g., ID 119):

nano /etc/pve/lxc/119.conf

Add these lines to the bottom:

lxc.cgroup2.devices.allow: c 10:200 rwm
lxc.mount.entry: /dev/net/tun dev/net/tun none bind,create=file

Reboot the container:

pct reboot 119

2. Installation by OS#

Debian 12 (Bookworm)#

curl -fsSL [https://pkgs.tailscale.com/stable/debian/bookworm.noarmor.gpg](https://pkgs.tailscale.com/stable/debian/bookworm.noarmor.gpg) | sudo tee /usr/share/keyrings/tailscale-archive-keyring.gpg >/dev/null
curl -fsSL [https://pkgs.tailscale.com/stable/debian/bookworm.tailscale-keyring.list](https://pkgs.tailscale.com/stable/debian/bookworm.tailscale-keyring.list) | sudo tee /etc/apt/sources.list.d/tailscale.list
apt update && apt install tailscale -y

Ubuntu 24.04 (Noble)#

curl -fsSL [https://pkgs.tailscale.com/stable/ubuntu/noble.noarmor.gpg](https://pkgs.tailscale.com/stable/ubuntu/noble.noarmor.gpg) | sudo tee /usr/share/keyrings/tailscale-archive-keyring.gpg >/dev/null
curl -fsSL [https://pkgs.tailscale.com/stable/ubuntu/noble.tailscale-keyring.list](https://pkgs.tailscale.com/stable/ubuntu/noble.tailscale-keyring.list) | sudo tee /etc/apt/sources.list.d/tailscale.list
apt update && apt install tailscale -y

Ubiquiti UDM-SE#

# Install the latest version of Tailscale UDM
curl -sSLq https://raw.github.com/SierraSoftworks/tailscale-udm/main/install.sh | sh

3. Subnet Router & Exit Node Setup#

Enable IP Forwarding#

This is required if you want this node to act as a bridge to your local network.