I run some services on machine A. I want to backup the data to machine B. It’s around 40 gigabytes in thousands of small files. I would prefer to compress the data, but there’s not enough space on machine A to keep the data twice (original + tar archive). I would like to avoid copying that many files via scp/rsync since it’s taking forever and a bit fragile.
Any solutions on how to solve this? Is it possible to compress live on machine A and stream into the archive on machine B without the need to keep the big archive file on A?
Eventually, it’s supposed to be automated and B has like the last 3 dailys, 1 weekly, 1 montly.
B can ssh into A.
I do have a someserver, but it’s a fediverse instance with around 60 monthly active users, and the server provider is more reliable than my ISP at home, so any downtime has a bit more impact. There’s also another admin who needs access. Not saying never, but at the moment I’m just looking for a different backup solution, moving the whole thing has some more aspects to consider.
The data would only be copied once a day. I have to check the limits though, good point.
You could set up syncthing to a pc at home and pull backups from there
That should work! I’ll have to make sure it only syncs one direction but that’s doable of course