I didn’t realize this until I started self-hosting my own instance, but if you don’t join one of the 3 large instances (beehaw, world, ml) then you miss out on a LOT of historical content. The way federation works is that it only pulls in new post/comments after someone on your instance subscribes to a community on another instance. So if you find a cool new community on another instance, you can subscribe to see any new posts and comments, but you won’t see any of the old content at all unless you manually search for the post/comment.
Long winded way of saying, the best user experience (content wise) is always going to be on the largest instances unless Lemmy/ActivityPub changes how content backfilling works.
The best thing an admin can do is either subscribe to all the major instances before users pour in or ask the earliest members to do so. Just makes search and timeline stuff a bunch easier on folks new to federation
Maybe they should update the join-lemmy.org page to suggest joining smaller instances. They put popular instances at the top and presumably that’s what everyone wants to join.
Edit: and then randomize the list of smaller instances to further distribute the load.
And many uninformed newcomers (like myself) assume that they can only directly interact with people or communities on their home instance, so the large ones get hit even more.
I guess there has to be a lowest common denominator instance. Not at all a bad thing, it leaves the dedicated communities out of their inevitable implosion range which still having access.
Really everyone always wants to be on the most popular “site” instance to ensure it will just not go away suddenly. After that they go for ones that give them a cool @ domain name. This is how email and Jabber/XMPP worked for years. Modern fediverse should be using some form of modern distributed identity, not 1965 email style identities.
Yes, I figured. My domain name is not as cool as “shitjustworks” or whatever. But I can say that my instance is gonna stay for as long as Lemmy as software is supported, no matter if there are many users or not. I strongly believe that FOSS and the Fediverse are the future and I want to give something to the community by hosting the instance.
I went through the evolution of email… At first it was universities, then ISPs etc. Having your identity tied to them SUCKED every time you no longer qualified for an account, changed providers ETC. I was a hotmail user before Microsoft purchased it, and an early beta Gmail user… While this is some centralisation these two identities have lasted decades, where AT THE TIME AOL was the (this is the biggest, never going away) option, now almost no one has an @AOL.com address.
Point being that no matter the current promise your instance could DIE if you get ill or can’t afford to host it etc. The model is BAD. I have said it before and will say it again, Identity SHOULD NOT be tied to instances, AND it needs some form of bot and trust system built in.
I partially agree with you. But my plan is to hand over the entire thing should I fall ill or get tired of hosting and maintaining it.
But in the end, everything’s gonna go away. Even Reddit, like all the platforms before it. That’s just the way things work.
What would be better, though? Having a P2P-like system where everything is truly federated? Like… Everyone has all accounts and all content at all times? I don’t know how this would work.
At this point in time, there are clear advantages to the current federated system, but there are also clear disadvantes, like what you’re describing, as well as some other things, like the different rules and moderation techniques of instances, defederation, etc.
undefined> What would be better, though? Having a P2P-like system where everything is truly federated? Like… Everyone has all accounts and all content at all times? I don’t know how this would work
I think something mandatory in the server instances that runs a blockchain (not crypto to be clear but that is how it works) IE every instance server is a validator node. When you create an account you do it from an instance, it gets recorded into the blockchain but at that point you have a lemmy account. You can directly log in on any instance as YOU (kind of like how SAML/OAUTH lets you use a google / microsoft / steam account) and use the services. When you post it is signed with your blockchain info. You could get banned on a specific instance and that gets recorded in the block chain. Other instances could chose to look at that info and decide they don’t want users that have been banned on multiple other instances or on specific trusted instances. Over time your account essentially becomes more or less trusted but the key think is that your YOU and not bound to one instance.
How do you handle the current issues with open registration and federation? Having accounts based on the individual instance is kind of what enables Beehaw to exist in the way that it does.
Bad instances could put false data into the chain, some vigilance for kicking and untrusting them would be needed
If your account gets hijacked there isn’t much you can do about it, no recovery (problem / risk in crypto that is well known) You are personally responsible for your account not the instance and lots of users and not smart enough for this.
Typically to prevent abuse crypto chains specifically put in CPU/GPU compute load to prevent being able to rapidly abuse posting data to the network. This could be done client side in the browser but to some degree would be needed on the instances to prevent people just spinning up malicious instances without cost but if this overhead is limited to identity only IE user creation, sign in, update it would be manageable other than during rapit sign up events like Reddit going down, spikes in interest.
Nothing is free or perfect but the current system somehow didn’t learn from decades of spam,bots, abuse in Email, IM and Web Forums. I don’t even recall the last web forum phpBB, SMF, Invision, Discourse etc that don’t OUT OF THE BOX give server admins a tool box of anti bot and spam / abuse clean up tools.
I seem to remember some parts of the Fediverse being able to pick up your user and move to another instance? Is that Mastodon by any chance? If that’s so, sounds like something Lemmy should look into.
It doesn’t sound like it would be too hard, pack up your user stuff in a JSON and sign it (basically a JWT), and establish a protocol by which one instance passes the user to another. Or, even better, let the user take their JSON and import it manually into another instance – this could also double as personal backup.
I believe Mastodon has that (seem to remember seeing it when I setup my account) but I think the instance you are on has to still be alive at the time to do i so it doesn’t quite address the sudden shutdown of the instance that holds your identity.
Why don’t instances create user caps (at least temporarily) to help spread the load? Seems wild to have unlimited sign-ups when instances max out typical hardware at a couple thousand users.
Well, not every user uses the same amount of server resources. Some are very active, some only use the server rarely, some register an account, check out Lemmy and never come back for one reason or another.
Should maybe rate-limit concurrent connections or scale.
With lemmy.world I believe the issue was that it was handling that number of users fine, then version 0.18 came and when they upgraded to it their server started struggling.
This is true. I was having a lot of issues with lemmy.ml it’s getting overwhelmed. I wish there was an easy way to carry over subscriptions between accounts.
I’d rather be on a site that hugs you to death than one that wants to screw everyone over because they are afraid of AI and API calls. Besides, it’s nice seeing the communities come together.
Not a great day for social media. Twitter down, Reddit has not 3rd party apps, Lemmy is being hugged to death by people bailing Reddit and Twitter.
I guess I’ll go outside.
Lemmy isn’t hugged to death. The issue is that everyone is just heading to the same handful of instances.
I didn’t realize this until I started self-hosting my own instance, but if you don’t join one of the 3 large instances (beehaw, world, ml) then you miss out on a LOT of historical content. The way federation works is that it only pulls in new post/comments after someone on your instance subscribes to a community on another instance. So if you find a cool new community on another instance, you can subscribe to see any new posts and comments, but you won’t see any of the old content at all unless you manually search for the post/comment.
Long winded way of saying, the best user experience (content wise) is always going to be on the largest instances unless Lemmy/ActivityPub changes how content backfilling works.
Or at the very least, it’s proof that the best time to start a new Lemmy instance for yourself is now instead of later 😉
The best thing an admin can do is either subscribe to all the major instances before users pour in or ask the earliest members to do so. Just makes search and timeline stuff a bunch easier on folks new to federation
It also means that your server load / storage needs blow up with the more users you have.
Maybe they should update the join-lemmy.org page to suggest joining smaller instances. They put popular instances at the top and presumably that’s what everyone wants to join.
Edit: and then randomize the list of smaller instances to further distribute the load.
Yes, that’s most likely the cause.
And many uninformed newcomers (like myself) assume that they can only directly interact with people or communities on their home instance, so the large ones get hit even more.
Here’s the current usershare breakdown by instance, if anyone’s curious:
Source: https://github.com/tgxn/lemmy-explorer/tree/main/frontend/public/data
Lemmy.world is getting a very big chunk, but other than that it actually seems fairly distributed.
I guess there has to be a lowest common denominator instance. Not at all a bad thing, it leaves the dedicated communities out of their inevitable implosion range which still having access.
Amazing graph, shame it’s a bit jumbled at the bottom. Very informative.
Really everyone always wants to be on the most popular “site” instance to ensure it will just not go away suddenly. After that they go for ones that give them a cool @ domain name. This is how email and Jabber/XMPP worked for years. Modern fediverse should be using some form of modern distributed identity, not 1965 email style identities.
Yes, I figured. My domain name is not as cool as “shitjustworks” or whatever. But I can say that my instance is gonna stay for as long as Lemmy as software is supported, no matter if there are many users or not. I strongly believe that FOSS and the Fediverse are the future and I want to give something to the community by hosting the instance.
I went through the evolution of email… At first it was universities, then ISPs etc. Having your identity tied to them SUCKED every time you no longer qualified for an account, changed providers ETC. I was a hotmail user before Microsoft purchased it, and an early beta Gmail user… While this is some centralisation these two identities have lasted decades, where AT THE TIME AOL was the (this is the biggest, never going away) option, now almost no one has an @AOL.com address.
Point being that no matter the current promise your instance could DIE if you get ill or can’t afford to host it etc. The model is BAD. I have said it before and will say it again, Identity SHOULD NOT be tied to instances, AND it needs some form of bot and trust system built in.
I partially agree with you. But my plan is to hand over the entire thing should I fall ill or get tired of hosting and maintaining it.
But in the end, everything’s gonna go away. Even Reddit, like all the platforms before it. That’s just the way things work.
What would be better, though? Having a P2P-like system where everything is truly federated? Like… Everyone has all accounts and all content at all times? I don’t know how this would work.
At this point in time, there are clear advantages to the current federated system, but there are also clear disadvantes, like what you’re describing, as well as some other things, like the different rules and moderation techniques of instances, defederation, etc.
undefined> What would be better, though? Having a P2P-like system where everything is truly federated? Like… Everyone has all accounts and all content at all times? I don’t know how this would work
I think something mandatory in the server instances that runs a blockchain (not crypto to be clear but that is how it works) IE every instance server is a validator node. When you create an account you do it from an instance, it gets recorded into the blockchain but at that point you have a lemmy account. You can directly log in on any instance as YOU (kind of like how SAML/OAUTH lets you use a google / microsoft / steam account) and use the services. When you post it is signed with your blockchain info. You could get banned on a specific instance and that gets recorded in the block chain. Other instances could chose to look at that info and decide they don’t want users that have been banned on multiple other instances or on specific trusted instances. Over time your account essentially becomes more or less trusted but the key think is that your YOU and not bound to one instance.
How do you handle the current issues with open registration and federation? Having accounts based on the individual instance is kind of what enables Beehaw to exist in the way that it does.
Sounds good to me. Any downsides?
Nothing is free or perfect but the current system somehow didn’t learn from decades of spam,bots, abuse in Email, IM and Web Forums. I don’t even recall the last web forum phpBB, SMF, Invision, Discourse etc that don’t OUT OF THE BOX give server admins a tool box of anti bot and spam / abuse clean up tools.
I seem to remember some parts of the Fediverse being able to pick up your user and move to another instance? Is that Mastodon by any chance? If that’s so, sounds like something Lemmy should look into.
It doesn’t sound like it would be too hard, pack up your user stuff in a JSON and sign it (basically a JWT), and establish a protocol by which one instance passes the user to another. Or, even better, let the user take their JSON and import it manually into another instance – this could also double as personal backup.
You are correct, and it’s on the Lemmy roadmap as far as I know. It’s just Mastodon has had a bit longer in the oven. We’ll get there 😄
I believe Mastodon has that (seem to remember seeing it when I setup my account) but I think the instance you are on has to still be alive at the time to do i so it doesn’t quite address the sudden shutdown of the instance that holds your identity.
That’s a good point. 😃 All the more reason for them to offer the ability to download your account data.
Except unlike email, it is not a big deal to change accounts on lemmy, almost all interation happens in communites not user to user.
It matters if you create a community and are the only mod, or are a mod etc etc. You basically lose that if you lose your identity .
Why don’t instances create user caps (at least temporarily) to help spread the load? Seems wild to have unlimited sign-ups when instances max out typical hardware at a couple thousand users.
Well, not every user uses the same amount of server resources. Some are very active, some only use the server rarely, some register an account, check out Lemmy and never come back for one reason or another.
Should maybe rate-limit concurrent connections or scale.
With lemmy.world I believe the issue was that it was handling that number of users fine, then version 0.18 came and when they upgraded to it their server started struggling.
It’s just the fediverse. My timeline is nothing but hugs and love. :3
That’s another way to see it. 🥰
Yep, the local instance I’m using is working flawlessly
Like, which one?
If the platform can’t handle large numbers of users flocking to popular instances, then it is doomed to fail.
The instance I’m on is working fine, I think the problem is people are gravitating towards the largest 2-3 instances.
This is true. I was having a lot of issues with lemmy.ml it’s getting overwhelmed. I wish there was an easy way to carry over subscriptions between accounts.
2023 has been historical for social media, so much changing so fast.
There are worse ways to die. 😁
It’s coming from the Reddit apocalypse. You could call it “death by snoo’s news”.
I’d rather be on a site that hugs you to death than one that wants to screw everyone over because they are afraid of AI and API calls. Besides, it’s nice seeing the communities come together.
Working fine for me
Things were pretty bad this AM