I’ve been using lemmy for a little while now. I feel like one of the strongest attractions I still have to reddit is the ability to find posts via a search engine. Typically I can type in whatever question I have and add “reddit” and get a far better answer than whatever google will give. I understand that lemmy doesn’t have this feature due to it being written in rust but I hope that sometime in the future we can find our way around this.

  •  lemmyng   ( @lemmyng@beehaw.org ) English
    12•1 year ago

    I understand that lemmy doesn’t have this feature due to it being written in rust

    The language has nothing to do with this. The way search engines work is they crawl the web and index everything they find. Then when you do a query for certain terms they use their magic sauce algorithm to find results they believe are most relevant and present them to you.

    Reddit is a convenient search term for three reasons:

    • it has predictable page locations, making it easy to crawl,
    • it has a lot of cross-links meaning it bubbles up to the top of search results easier, and
    • it gets crawled often because its content changes very fast.

    So for Lemmy to have equivalent indexability it needs lots of links, frequent changes, and it needs to have a predictable method for discovering content via crawling. The first two are entirely based on users - the more users contribute the more content gets posted, more comments, are made, and more links to the content show up organically on other sites. The last one is trickier - being federated Lemmy content cannot be crawled in the same way as standard websites. However the fediverse does have a deterministic way of getting content from federated servers, so it could well be that we’ll see the search engines create their own federation server to slurp up content instead of crawling the sites.