That’s just…a bad idea then. You have every opportunity to not do that, and you’re using a solution like this as a patch to solve for other issues you had. Not a good use-case.
This is one of the most popular methods to handle apps that are using a storage backend directly over the internet for increased bandwidth, and separation of deployment and state.
If I’m hosting something like Nextcloud at a massive scale, it simply isn’t feasible to use internal networks because they don’t have enough bandwidth, and overlay/vpn network solutions have too much overhead. A common solution is to just run the services and directly connect to them over the internet. So I point my 10,000 node Nextcloud instance at S3, either my own cluster or somebody else’s and S3 handles encryption while remaining reasonably performant. And scalable.
Garage uses their software internally for something similar: https://garagehq.deuxfleurs.fr/documentation/design/goals/ . On this page, they describe using it as their matrix image/file storage cache. Same thing there, they probably have a large distributed matrix cluster that needs storage that scales with it, while also being secure over different networks.
Okay, as I said before, you had a technical issue you couldn’t fix for one specific use-case. Jumping down to a less efficient (which this is) abstract to solve for that problem isn’t a good solution, ESPECIALLY if you’re self-hosting as you describe.
If I go to a store to buy hot dogs, and they’re out of hot dogs, I wouldn’t buy hamburgers to cut in half and try to pass it off as hot dogs just because they fit in a hot dog bun.
That’s just…a bad idea then. You have every opportunity to not do that, and you’re using a solution like this as a patch to solve for other issues you had. Not a good use-case.
This is one of the most popular methods to handle apps that are using a storage backend directly over the internet for increased bandwidth, and separation of deployment and state.
If I’m hosting something like Nextcloud at a massive scale, it simply isn’t feasible to use internal networks because they don’t have enough bandwidth, and overlay/vpn network solutions have too much overhead. A common solution is to just run the services and directly connect to them over the internet. So I point my 10,000 node Nextcloud instance at S3, either my own cluster or somebody else’s and S3 handles encryption while remaining reasonably performant. And scalable.
Garage uses their software internally for something similar: https://garagehq.deuxfleurs.fr/documentation/design/goals/ . On this page, they describe using it as their matrix image/file storage cache. Same thing there, they probably have a large distributed matrix cluster that needs storage that scales with it, while also being secure over different networks.
Okay, as I said before, you had a technical issue you couldn’t fix for one specific use-case. Jumping down to a less efficient (which this is) abstract to solve for that problem isn’t a good solution, ESPECIALLY if you’re self-hosting as you describe.
If I go to a store to buy hot dogs, and they’re out of hot dogs, I wouldn’t buy hamburgers to cut in half and try to pass it off as hot dogs just because they fit in a hot dog bun.
Do you have an alternative solution to the technical challenges I have mentioned above?