Proxmox

Links:

Post-install

  • Remove unwanted languages from apt update
    • echo 'Acquire::Languages "none";' > /etc/apt/apt.conf.d/99translations
  • Enable IOMMU
    • sed -i 's/GRUB_TIMEOUT=5/GRUB_TIMEOUT=3/' /etc/default/grub;
      sed -i 's/GRUB_CMDLINE_LINUX_DEFAULT="quiet"/GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt"/' /etc/default/grub;
    • run update-grub
  • Run the community post install script link
    • bash -c "$(curl -fsSL https://raw.githubusercontent.com/community-scripts/ProxmoxVE/main/tools/pve/post-pve-install.sh)"

postfix email config

  • install extra modules
    • apt-get install libsasl2-modules postfix-pcre;
  • disable any default relayhost/smtp_sasl_security_options in main.cf before adding our own
    • sed -i 's/^relayhost/#relayhost/' /etc/postfix/main.cf;
      sed -i 's/^smtp_sasl_security_options/#smtp_sasl_security_options/' /etc/postfix/main.cf;
  • add relay host to /etc/postfix/main.cf
    • cat >> /etc/postfix/main.cf << EOF
      relayhost = [igly.one]:587
      smtp_use_tls = yes
      smtp_sasl_auth_enable = yes
      smtp_sasl_security_options = noanonymous
      smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
      smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
      smtp_header_checks = pcre:/etc/postfix/smtp_header_checks
      EOF
  • populate sasl_passwd
    • echo '[igly.one]:587  USER:PASSWORD' > /etc/postfix/sasl_passwd;
      chmod 600 /etc/postfix/sasl_passwd;
      postmap /etc/postfix/sasl_passwd;
  • Customize From field
    • echo "/^From:.*/ REPLACE From: ${HOSTNAME}-alert <${HOSTNAME}-alert@${HOSTNAME}.lair.lan>" > /etc/postfix/smtp_header_checks;
        postmap /etc/postfix/smtp_header_checks;
  • Restart postfix service
    • systemctl restart postfix.service;
  • Send test mail
    • echo "Test mail from postfix" | mail -s "$HOSTNAME Test Postfix" some@valid.email

Internal custom CA authority

  • Internal Lair.lan CA:
    • mkdir /usr/local/share/ca-certificates/lairlan-ca;
      mv lairlan.internal.root.crt /usr/local/share/ca-certificates/lairlan-ca/rootCA.crt
      update-ca-certificates
  • Internal Fail.pm CA:
    • mkdir /usr/local/share/ca-certificates/failpm-ca;
      mv failpm.internal.root.crt rootCA.crt /usr/local/share/ca-certificates/failpm-ca/rootCA.crt
      update-ca-certificates
  • Internal ACME SERVER:
    • pvenode acme account register default somemail@validornot.pm
      pvenode config set --acme domains="pve01.lair.lan;10.193.141.15"
      pvenode acme cert order

Storage

idmap

  • Create a user which will have ownership of the shared data
  • Add to /etc/subuid:
    • root:2000:1
  • Add to /etc/subgid:
    • root:2000:1
  • for each container needing to map data we add the following to its config /etc/pve/lxc/<container_id>.conf
    •   lxc.idmap: u 0 100000 2000
        lxc.idmap: g 0 100000 2000
        lxc.idmap: u 2000 2000 1
        lxc.idmap: g 2000 2000 1
        lxc.idmap: u 2001 102001 63535
        lxc.idmap: g 2001 102001 63535

Add a hostpath as mpX device mounted at vmpath inside CT

pct set VMID -mp[0-9] [hostpath],mp=[vmpath]

VM disk import/passt

qm import 900 /path/to/file
qm set 901--scsi1 /dev/disk/by-label/[label]

CT create disk and mount

creates a new disk image on storageid local size 30gb and maps into CT at /srv/www

pct set VMID -mp0 local:30,mp=/srv/www

Delayed Samba mount

In the case where you need to mount a samba share from a container/vm running on top of pve. You can use x-systemd.automount,x-systemd.idle-timeout=30 in fstab and have it mounted when something accesses the path.

//10.0.10.20/share /mnt/share cifs _netdev,x-systemd.automount,x-systemd.requires=network-online.target,x-systemd.device-timeout=10,noserverino,rw,users,nounix,password2=fake,user=data,password=<PASSWORD>,uid=2000,gid=2000,file_mode=0644,dir_mode=0755,vers=3 0 0

Test with

systemctl daemon-reload
systemctl restart remote-fs.target

Apply SDN config changes from cli

pvesh set /cluster/sdn

lxc.cap.*

This is not a working example

lxc.cap.keep:
lxc.cap.keep: net_raw net_admin
lxc.apparmor.profile: unconfined

To see caps inside CT use

capsh --print

Proxmox Backup Server

proxmox-backup-manager cert info | grep Fingerprint
proxmox-backup-manager datastore create offlinedata /mnt/offlinedata/pbsdata

Debugging

lxc-start -n 603 -F -l DEBUG -o /tmp/lxc-ID.log
lxc-start -n 601 -F -l DEBUG -o /tmp/lxc-ID.log  -l trace -o /dev/stderr

Thunderbolt 3 Network card

apt install bolt policykit-1
# Change to your ID
boltctl enroll d1030000-0082-8098-2027-f21a6402bb22
echo thunderbolt >> /etc/modules
# Change to your driver
echo atlantic >> /etc/modules
# Probably should rename nic before using its name as input below
echo 'SUBSYSTEM=="net", ACTION=="move", KERNEL=="enp9s0", RUN+="/usr/sbin/ip link set mtu 9000 txqlen 10000 dev '%k'", RUN+="/usr/sbin/ifup %k", RUN+="/usr/sbin/brctl addif vmbr0 %k"' > /etc/udev/rules.d/10-tb-en.rules

Notes on VLAN

I will list interesting commands here

bridge -compressvlans vlan show

Container Autologin

Only works if root has no passwd

GETTY_OVERRIDE="/etc/systemd/system/container-getty@1.service.d/override.conf"
mkdir -p $(dirname $GETTY_OVERRIDE)
cat <<EOF >$GETTY_OVERRIDE
  [Service]
  ExecStart=
  ExecStart=-/sbin/agetty --autologin root --noclear --keep-baud tty%I 115200,38400,9600 \$TERM
EOF
systemctl daemon-reload
systemctl restart $(basename $(dirname $GETTY_OVERRIDE) | sed 's/\.d//')

GPU Accel for plex/jellyfin/etc

insert into relevant /etc/pve/lxc/PVID.conf

dev0: /dev/dri/card0,gid=44
dev1: /dev/dri/renderD128,gid=104

Remove ghost node after pve hostname change

Do remember to copy over any PVID.confs to the current cluser /etc/pve/lxc/ or /etc/pve/qm/ before deleting

  • rm -rf /etc/pve/nodes/old-hostname/

Guest Support

  • virtio-win - VirtIO windows drivers compiled by fedora

Image/Disk tools

  • Disk2vhd (Windows) - Disk2vhd is a utility that creates VHD (Virtual Hard Disk - Microsoft’s Virtual Machine disk format) versions of physical disks for use in Microsoft Virtual PC or Microsoft Hyper-V virtual machines (VMs)
  • ntfsclone2vhd - Utility to convert ntfsclone “special images” to dynamic VHD virtual disks
  • VDI Studio (Windows) - VDI Studio is a free editor for VirtualBox disk image files
  • VHD Resizer tool (Mirrored, Windows) - expand the size of a VHD

Optimizations for running on CephFS cluster

NOT TESTED !


Set SAS HDD Write Cache Enable (WCE) (sdparm -s WCE=1 -S /dev/sd[x]) Set VM Disk Cache to None if clustered, Writeback if standalone Set VM Disk controller to VirtIO-Single SCSI controller and enable IO Thread & Discard option Set VM CPU Type to ‘Host’ Set VM CPU NUMA on servers with 2 more physical CPU sockets Set VM Networking VirtIO Multiqueue to number of Cores/vCPUs Set VM Qemu-Guest-Agent software installed Set VM IO Scheduler to none/noop on Linux Set Ceph RBD pool to use ‘krbd’ option


## __BUGS and Errors__

### __ifupdown2 bug__
This is currently an issue with ifupdown2 [1]<https://bugzilla.proxmox.com/show_bug.cgi?id=5406> [2]<https://github.com/CumulusNetworks/ifupdown2/issues/124>

/usr/sbin/ifup[903]: error: enp7s0f0np0.141: cmd '/bin/ip route replace default via 10.193.141.1 proto kernel dev enp7s0f0np0.141 onlink' failed: returned 2 (Error: Nexthop device is not up.)

A temp fix is to edit `/etc/network/ifupdown2/ifupdown2.conf` and change `link_master_slave` to `0`

`sed -i 's/link_master_slave=1/link_master_slave=0/' sed -i 's/GRUB_TIMEOUT=5/GRUB_TIMEOUT=3/' /etc/default/grub;`

### EDID block 0 is all zeroes<a name="EDID_block_0"></a>

    echo " i915.fastboot=1 drm.edid_firmware=edid/1920x1080.bin" >> /etc/kernel/cmdline
    update-initramfs -u -k all && pve-efiboot-tool refresh

Sadly this fix didn't not make the error go away, using the nuclear method for now

    i915.disable_display=1

In my case it didn't fix the problem, your milage may vary

### RRDC update error
Log output shows

Feb 09 23:37:22 pve01 rrdcached[2038]: handle_request_update: Could not read RRD file. Feb 09 23:37:22 pve01 pmxcfs[2052]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-vm/311: -1 Feb 09 23:37:22 pve01 pmxcfs[2052]: [status] notice: RRD update error /var/lib/rrdcached/db/pve2-vm/311: mmaping file ‘/var/lib/rrdcached/db/pve2-vm/311’: Invalid argument Feb 09 23:37:32 pve01 rrdcached[2038]: handle_request_update: Could not read RRD file. Feb 09 23:37:32 pve01 pmxcfs[2052]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-vm/311: -1 Feb 09 23:37:32 pve01 pmxcfs[2052]: [status] notice: RRD update error /var/lib/rrdcached/db/pve2-vm/311: mmaping file ‘/var/lib/rrdcached/db/pve2-vm/311’: Invalid argument Feb 09 23:37:42 pve01 rrdcached[2038]: handle_request_update: Could not read RRD file. Feb 09 23:37:42 pve01 pmxcfs[2052]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-vm/311: -1 Feb 09 23:37:42 pve01 pmxcfs[2052]: [status] notice: RRD update error /var/lib/rrdcached/db/pve2-vm/311: mmaping file ‘/var/lib/rrdcached/db/pve2-vm/311’: Invalid argument


The solution is simply to delete the RRDC db dir and restart the service. Files will automatically be regenerated
```bash
rm -r /var/lib/rrdcached/db;
systemctl restart rrdcached.service;

Source

Rescue options

https://pve.proxmox.com/pve-docs/pve-admin-guide.html#chapter_pmxcfs


Last modified: Mon Dec 8 14:34:22 2025