A common problem of Lemmy compared to Reddit I see mentioned often is the lack of way to search for it. Lots of people add “reddit” to their Google queries to get better results, however this is not possible with Lemmy due to its decentralized nature.

Could this problem be solved with a read-only instance which would import all past and future content from every federated instance, with the sole purpose of being indexed by search engines? This way, one would add “lemmyindex” or whatever its name is to their search queries.

I suppose server capacity would be a problem; however, due to it being read-only, caching would significantly reduce load, and images would still be hosted by their source instances.

  •  sugar_in_your_tea   ( @sugar_in_your_tea@sh.itjust.works ) English
    2•1 year ago

    So is the intent for search engine optimization? Or for just for finding content within the fediverse?

    The first would be a tough nut to crack since SEO relies so heavily on links between indexed sites, so you’d pretty much have to have a bunch of instances and communities include links to this index for it to be visible.

    The second would be much more tractable, and perhaps a special instance that merely maintains an index (as in, not the full content, just an index of key words to content) would be viable. That option doesn’t require any kind of mass collaboration and could be done today, just like this list of instances.

    If we need to solve the SEO problem, I think it’ll be an uphill battle to keep it more relevant than the random tech blogs that will inevitably try to get that search share.