The Web UI and whatever is nice, but it's just not my thing.
Now that I have zero Windows in my house, it might be better to just run regular Linux on them with libvirtd ontop. Then manage them all from my desktop or laptop using virt-manager
The drawback would be that I lose the clustering features, such as automatic config sync, shared block storage via LVM (at least without risk) and automatic recovery of virtual machines if one host dies.
Because then I can set it up with libvirt, migrate virutal machines there from the Proxmox cluster, and once everything is on the third host, I can shut down my Proxmox machines and reinstall them with Linux+libvirt, and spread servers back out again afterwards.
And web ui barely matters when I connect to VMs over SPICE using remote-viewer anyways. Backups and High Availability are also automated with Proxmox. Not to mention that you can add LVM storage like iSCSI and Proxmox just handles making sure there's no collisions on storage that would cause corruption.
If using Libvirt it'd simplify the setup but I'd have to write scripts to do backups, and I'd pretty much give up on HA and storage management.
@quad For me, Proxmox is basically LXC and KVM with a nice web UI on top. I use the web UI rarely, but you can do anything KVM and LXC can do, with zero issues. I even automated some stuff via Ansible and it's a bliss. I don't see what benefit would it have to give up on the web UI, despite using it rarely. I mean, it's not like the web UI is a resource hog anyway.
What I miss, honestly, is nftables support, as Proxmox is still on iptables. But it's not that critical, I can wait.
At that point it'd be better to just use virt-manager rather than a Web UI
@quad That's odd. I've never experienced those issues, then again I don't cluster Proxmox at all. In my previous job I used to, and it worked relatively fine, but it was a little painful to manage it all.
But I don't really need clustering that much, yet managing two, soon three entirely separate Proxmox machines and migrating machines between them is a bit clunky if they're not in a cluster.
@quad I've grown accustomed to Proxmox, and I really like it. But I've been moving towards Docker, Docker-compose and Ansible (a homegrown Kubernetes of sorts, I suppose), and I am pretty sure I could move away from my current "one big server" infrastructure and into tiny AWS/Linode/Vultr nodes with autoscaling. But the uncertainty of price makes me a little anxious, even if I'd probably end up saving in the long term.
Worked well when I hosted individual proxmox nodes on Online.net or Hetzner boxes though. Heck even on the old Core 2 Duo pile of garbage with 8GB of RAM 5+ years ago when I was in school.
But I have two hosts now, a third coming soon, and clustering hasn't served me well. Yet doing stuff manually all the time kind of defeats the purpose of Proxmox in my eyes.
I keep hoping that the worst is behind us and it'll just get more solid from here on out, but it's a bit like waiting for the Year of Linux on the Desktop.
So clustering seems to cause more downtime than it saves for me. And running Proxmox seems pointless if I do everything manually behind the scenes anyways, since the Web UI mostly handles cross-node stuff when you're clustered.
It’s not a common occurrence but for example in December the pvesr service decided to die on a host and VMs were restarted on another.
pvesr stayed broken until I reinstalled the whole machine and rejoined it into the cluster. And I dunno why because all storage was perfectly functional when checking it via command line. NFS for backups worked fine and LVM over iSCSI showed up fine with lv commands.
Outside of power outages, all virtual machine downtime on my cluster has been caused by services failing whose names all start with “pve”