@sir i don't think it's a problem. the web is large now and webservers need to account for that, unless there's some specific bug that's causing things to get hit more than there are eyeballs
@sneak every time you post a link on mastodon, every instance it gets federated to will automatically fetch it to generate a link preview
@sir @sneak what would be the solution?
A centralised proxy? Previews fetched from their origin (and DDossing that)? Some distribution and mirrorring; and then how to avoid poisoning?
Honestly curious.
What about a separate p2p 'linkpreview' network? Such a project could even leverage some 'reward' system, not unlike filecoin, dat or storj. Mastodon, matrix, signal etc could then fetch previews there, instead of all implementing their own.
@sneak Aside from the "snobbism" (not all can afford to have tuned servers/ CMSes, some may want to use a Pi or some spare hardware), this doesn't scale.
So, indeed: Today it might not be a problem at all.
But AP aims to grow and even manages to do so. Grow x4 and the mastodon side of the fediverse alone, hits a website potentially 10108 times[1] for each link.
My static-sites won't blink an eye. But your average shared-wordpress or Rails mudball will go down.
@berkes a rails or shared wordpress will go down if a link to it is shared on social media too, for the same reason. the web is big now and servers need to serve. nothing is broken here.
@berkes it won't exclude it from being shared on the fediverse. i would be extremely surprised if the links don't get posted even if preview generation fails. there is no problem here
@sneak I've owned and/or founded three webhosting companies. Please believe me when I say that your requirement will exclude over half of the "long tail" of the internet. If not more. Too many websites are on a constant brink of breakdown. From webshops that can handle 5 orders a week, but never more to blogs that crash at anything over 1rps.
I cherish that web. I certainly don't want to "exclude it" from being shared on fediverse. Not now, and not when the fediverse grows 4, 10, or 100x.