Proxmox

TOC:

Post install

  • Remove unwanted languages from apt update 
    • echo 'Acquire::Languages "none";' > /etc/apt/apt.conf.d/99translations
  • Enable IOMMU 
    • sed -i 's/GRUB_TIMEOUT=5/GRUB_TIMEOUT=3/' /etc/default/grub;
sed -i 's/GRUB_CMDLINE_LINUX_DEFAULT="quiet"/GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt"/' /etc/default/grub;
# Intel Version
sed -i 's/GRUB_CMDLINE_LINUX_DEFAULT="quiet"/GRUB_CMDLINE_LINUX_DEFAULT="quiet Intel_iommu=on iommu=pt"/' /etc/default/grub;
    • run update-grub
  • Run the community post install script link
    • bash -c "$(curl -fsSL https://raw.githubusercontent.com/community-scripts/ProxmoxVE/main/tools/pve/post-pve-install.sh)"
  • Optional Edit /etc/bash.bashrc
    • Uncomment "# Commented out, don't overwrite xterm -T "title" -n "icontitle" by default." section
    • Uncomment "# enable bash completion in interactive shells" section

SSL

  • Internal Lair.lan CA: 
    • mkdir /usr/local/share/ca-certificates/lairlan-ca;
mv lairlan.internal.root.crt /usr/local/share/ca-certificates/lairlan-ca/rootCA.crt
update-ca-certificates
  • Internal Fail.pm CA: 
    • mkdir /usr/local/share/ca-certificates/failpm-ca;
mv failpm.internal.root.crt rootCA.crt /usr/local/share/ca-certificates/failpm-ca/rootCA.crt
update-ca-certificates

Using internal CA see Proxmox SSL

Postfix email config

  • install extra modules
    • apt-get install libsasl2-modules postfix-pcre;
  • disable any default relayhost/smtp_sasl_security_options in main.cf before adding our own 
    • sed -i 's/^relayhost/#relayhost/' /etc/postfix/main.cf;
sed -i 's/^smtp_sasl_security_options/#smtp_sasl_security_options/' /etc/postfix/main.cf;
  • add relay host to /etc/postfix/main.cf 
    • cat >> /etc/postfix/main.cf << EOF
relayhost = [igly.one]:587
smtp_use_tls = yes
smtp_sasl_auth_enable = yes
smtp_sasl_security_options = noanonymous
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
smtp_header_checks = pcre:/etc/postfix/smtp_header_checks
EOF
  • populate sasl_passwd 
    • echo '[igly.one]:587  USER:PASSWORD' > /etc/postfix/sasl_passwd;
chmod 600 /etc/postfix/sasl_passwd;
postmap /etc/postfix/sasl_passwd;
  • Customize From field 
    • echo "/^From:.*/ REPLACE From: ${HOSTNAME}-alert <${HOSTNAME}-alert@${HOSTNAME}.lair.lan>" > /etc/postfix/smtp_header_checks;
  postmap /etc/postfix/smtp_header_checks;
  • Restart postfix service
    • systemctl restart postfix.service;
  • Send test mail
    • echo "Test mail from postfix" | mail -s "$HOSTNAME Test Postfix" some@valid.email

idmapping

  • Create a user which will have ownership of the shared data
  • Add to /etc/subuid:
    • root:2000:1
  • Add to /etc/subgid:
    • root:2000:1
  • for each container needing to map data we add the following to its config /etc/pve/lxc/<container_id>.conf 
    •   lxc.idmap: u 0 100000 2000
  lxc.idmap: g 0 100000 2000
  lxc.idmap: u 2000 2000 1
  lxc.idmap: g 2000 2000 1
  lxc.idmap: u 2001 102001 63535
  lxc.idmap: g 2001 102001 63535

Storage

Add PBS namespace

add pbs namespace to pve from cli:

pvesm add pbs [storage-id] --server pbs.lair.lan --datastore [id] --content backup --username [id] --password

Delayed Samba mount

In the case where you need to mount a samba share from a container/vm running on top of pve. You can use x-systemd.automount,x-systemd.idle-timeout=30 in /etc/fstab and have it mounted when something accesses the path.

//10.0.10.20/share /mnt/share cifs _netdev,x-systemd.automount,x-systemd.requires=network-online.target,x-systemd.device-timeout=10,noserverino,rw,users,nounix,password2=fake,user=data,password=<PASSWORD>,uid=2000,gid=2000,file_mode=0644,dir_mode=0755,vers=3 0 0

Test with

systemctl daemon-reload
systemctl restart remote-fs.target

Network

Reload interfaces

reload /etc/network/interfaces from cli

ifreload -a

SDN

Apply SDN config changes from cli

pvesh set /cluster/sdn

Bridges and VLAN

I will list interesting commands here

list which interfaces are members of bridges:

brctl show

list which interfaces/bridges are members of VLANs:

bridge -compressvlans vlan show

Thunderbolt 3 Network card

apt install bolt policykit-1
# Change to your ID
boltctl enroll d1030000-0082-8098-2027-f21a6402bb22
echo thunderbolt >> /etc/modules
# Change to your driver
echo atlantic >> /etc/modules
# Probably should rename nic before using its name as input below
echo 'SUBSYSTEM=="net", ACTION=="move", KERNEL=="enp9s0", RUN+="/usr/sbin/ip link set mtu 9000 txqlen 10000 dev '%k'", RUN+="/usr/sbin/ifup %k", RUN+="/usr/sbin/brctl addif vmbr0 %k"' > /etc/udev/rules.d/10-tb-en.rules

Container Storage

Hostpath

Add a hostpath as mpX device mounted at vmpath inside CT

pct set VMID -mp[0-9] [hostpath],mp=[vmpath]

VM disk import

first scp or otherwise load the disk file to the node

qm import 900 /path/to/file

VM disk passthrough

qm set 901--scsi1 /dev/disk/by-label/[label]

CT create disk and mount

creates a new disk image on storageid local size 30gb and maps into CT at /srv/www

pct set [VMID] -mp0 local:30,mp=/srv/www

GPU Acceleration

To be able to use a GPU inside a container add the following to /etc/pve/lxc/[VMID].conf:

dev0: /dev/dri/card0,gid=984
dev1: /dev/dri/renderD128,gid=988
lxc.cgroup2.devices.allow: c 226:1 rwm
lxc.cgroup2.devices.allow: c 226:128 rwm

Do remember to adjust GIDs to match inside the CT and also whatever software is a member of those groups

Container Autologin

Only works if root has no passwd

GETTY_OVERRIDE="/etc/systemd/system/container-getty@1.service.d/override.conf"
mkdir -p $(dirname $GETTY_OVERRIDE)
cat <<EOF >$GETTY_OVERRIDE
  [Service]
  ExecStart=
  ExecStart=-/sbin/agetty --autologin root --noclear --keep-baud tty%I 115200,38400,9600 \$TERM
EOF
systemctl daemon-reload
systemctl restart $(basename $(dirname $GETTY_OVERRIDE) | sed 's/\.d//')

Debugging

To figure out why a container won’t start:

lxc-start -n [VMID] -F -l DEBUG -o /tmp/lxc-ID.log
lxc-start -n [VMID] -F -l DEBUG -o /tmp/lxc-ID.log  -l trace -o /dev/stderr

To see CAPS inside CT use

capsh --print

BUGS and Errors

Remove ghost node after pve hostname change

Do remember to copy over any PVID.confs to the current cluser /etc/pve/lxc/ or /etc/pve/qm/ before deleting

rm -rf /etc/pve/nodes/old-hostname/

RRDC update error

Log output shows

Feb 09 23:37:22 pve01 rrdcached[2038]: handle_request_update: Could not read RRD file.
Feb 09 23:37:22 pve01 pmxcfs[2052]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-vm/311: -1
Feb 09 23:37:22 pve01 pmxcfs[2052]: [status] notice: RRD update error /var/lib/rrdcached/db/pve2-vm/311: mmaping file '/var/lib/rrdcached/db/pve2-vm/311': Invalid argument
Feb 09 23:37:32 pve01 rrdcached[2038]: handle_request_update: Could not read RRD file.
Feb 09 23:37:32 pve01 pmxcfs[2052]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-vm/311: -1
Feb 09 23:37:32 pve01 pmxcfs[2052]: [status] notice: RRD update error /var/lib/rrdcached/db/pve2-vm/311: mmaping file '/var/lib/rrdcached/db/pve2-vm/311': Invalid argument
Feb 09 23:37:42 pve01 rrdcached[2038]: handle_request_update: Could not read RRD file.
Feb 09 23:37:42 pve01 pmxcfs[2052]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-vm/311: -1
Feb 09 23:37:42 pve01 pmxcfs[2052]: [status] notice: RRD update error /var/lib/rrdcached/db/pve2-vm/311: mmaping file '/var/lib/rrdcached/db/pve2-vm/311': Invalid argument

The solution is simply to delete the RRDC db dir and restart the service. Files will automatically be regenerated

rm -r /var/lib/rrdcached/db;
systemctl restart rrdcached.service;

Source

Links:

Last modified: Thu May 7 22:09:06 2026