If you have an nvidia card, which is very common and can’t always be avoided, there is significant benefit in newer drivers, for one

Huh.

There’s a time and place for a DIY solution and academia can well be like that sometimes.

The latest Mac Mini can’t run Linux though. It’s M4 and asahi doesn’t even support M3 chips yet. But if you actually got the previous model with M1/M2 you can do Linux if desired. I might not attempt, and just use the Mac as a server as-is. It’s not too different from Linux. Asking the duck for “how to xx on Mac” when you already know the Linux equivalents should make your life tolerable.

Me, because someone at my work picked it for servers back in the day

I’d expect so, but you’ll need to test with your exact router model how it behaves. Some have a ‘DMZ’ function that you can use to pass all ports to a certain host. I use it to expose the WAN interface of my opnsense router to the internet through the ISP router. Then I can fine tune the open ports further in opnsense which is better designed for that than the usual ISP box.

This post considers the situation where you expose your ports to the internet, on the edge of your residential network, for example by setting your router to forward requests with port 443 to a certain host in your network. In this case you do have a public ip address and the configured port on your home server is now reachable from the internet. This is different from just exposing a port on a machine inside a residential network for local use.

I have an external storage unit a couple kilometers away and two 8TB hard drives with luks+btrfs. One of them is always in the box and after taking backups, when I feel like it, I detach the drive and bike to the box to switch. I’m currently researching btrbk for updating the backup drive on my pc automatically, it’s pretty manual atm. For most scenarios the automatic btrfs snapshots on my main disks are going to be enough anyway.

Oh yeah and I did enable Proxmox VM firewall for the TrueNAS, the NFS traffic goes via an internal interface. Wasn’t entirely convinced by NFS’s security posture when reading about it… At least restrict it to the physical machine 0_0 So I now need to intentionally pass a new NIC to any VM that will access the data, which is neat.

A wrap-up of what I ended up doing:

• Replaced the bare metal Ubuntu with Proxmox. Cool cool. It can do the same stuff but easier / comes with a lot of hints for best practices. Guess I’m a datacenter admin now
• Wiped the 2x960GB SSD pool and re-created it with ZFS native encryption
• Made a TrueNAS Scale VM, passed through the SSD pool disks, shared the datasets with NFS and made snapshot policies
• Mounted the NFS on the Ubuntu VM running my data related services and moved the docker bind mounts to that folder
• Bought a 1Gbps Intel network card to use instead of the onboard Realtek and maxed out the host memory to 16GB for good measure

I have achieved:

• 15min RPO for my data (as it sits on the NFS mount, which is auto-snapshotted in TrueNAS)
• Encryption at rest (ZFS native)

I have not achieved (yet…):

• Key fetch on boot. Now if the host machine boots I have to log in to TrueNAS to key in the ZFS passphrase. I will have to make some custom script for this anyway I guess to make it adapt to the situation as key fetching on boot is a paid feature in TrueNAS but it just makes managing the storage a bit easier so I wanna use it now. Disabled auto start on boot for the services VM that depends on the NFS share, so I’ll just go kick it up manually after unlocking the pool in TrueNAS.

Quite happy with the setup so far. Looking to automate actual backups next, but this is starting to take shape. Building the confidence to use this for my actual phone backups, among other things.

Really good to know. Planned to keep using very mainstream LTS versions anyway, but this solidifies the decision. Maybe on a laptop I’ll install something more experimental but that’s then throwaway style.

I guess I’ll give it a spin. There seems to be a big community around it. I initially thought I might migrate later so keeping the host OS layer as thin as possible. Ubuntu was mainly an easy start as I was familiar with it from before and the spirit in this initiative is DIY over framework - but if there’s a widely used solution for exactly this… Yeah.

Always a good reminder to test the backups, no I would not sleep properly if I didn’t test them :p

Aiming to keep it simple, too many moving parts in the VM snapshots / hard to figure out best practices and notice mistakes without work experience in the area, so I’ll just backup the data separately and call it a day. But thanks for the input! I don’t think any of my services have in-memory db’s.

Right, thanks for the heads up! On the desktops I have simply installed zfs as root via the Ubuntu 24.04 installer. Then, as the option was not available in the server variant I started to think maybe that is not something that should be done :p

Aight thank you so much, confirms I’m on the right path! This clarifies a lot, I’ll keep the ext4 boot drive :)

Right, so my aversion to live backups comes initially from Louis Rossmann’s guide on the FUTO wiki where he mentions it’s non trivial to reliably snapshot a running system. After a lot of looking elsewhere as well I haven’t gotten much hints that it would be bad advice and I want to err on the side of caution anyway. The hypervisor is QEMU/KVM so in theory it should be able to do live snapshots afaik. But I’m not familiar enough with the consistency guarantees to fully trust it. I don’t wanna wake up one day to a server crash and trying to mount the backed up qcow2 in a new system and suddenly it wouldn’t work and I just lost data.

It won’t matter though as I’ll just place all the important data on the zpool and back that up frequently as a simple data store. The VMs can keep doing their nightly shutdown and snapshot thing.

Ok so wrapping my head around this, what I think I need to be clear about is the separation between applications and data. Applications get the nightly VM snapshot way of backing up, and data will get the frequent zfs snapshots (and other backups). Kinda what I tried to do to begin with, so I will look more on how to do this separation for the applications I intend to use.

Still unsure if samba is the way to go for linking it together on the same physical machine.

Should I just run syncthing on the bare metal host…? Will sleep on it.

This is what I’m doing currently, but it’s not really feasible to have the services shut down hourly for snapshots. This is indeed why I started looking towards filesystem-level snapshotting Obviously I will have other types of backups as well, I’m simply looking to have the on-the-fly immutable snapshot capability here somehow.

Thanks! Can I ask what is your setup like? ZFS on bare metal? Do you have VMs?

Fair about the SSD life. How would you go about achieving the frequent backups without zfs? I wouldn’t want to implement it separately for every app I use, though I’m open to it if this doesn’t work out.

I’ll easily buy more memory if needed, the box now has 8GB and isn’t struggling in any way.