On the whole, I really like the search engine. But one small bug you may want to fix is that occasionally the wrong results appear under ‘Users’. For example, if you type ‘Will MacAskill’, the three results that show up are posts where the name ‘Will MacAskill’ appears in the title, rather than the user Will MacAskill.
EDIT: Mmh, this appears to happen because a trackback to Luke Muehlhauser’s post, ‘Will MacAskill on Normative Uncertainty’, is being categorized as the name of a user. So, not a bug with the search engine as such, but still something that the EA Forum tech team may want to fix.
Oh the joys of a long legacy of weird code. I’ve deleted those accounts, although I’m sad to report that our search engine is not smart enough to figure out that “Will MacAskill” should return “William_MacAskill”
Yeah, you can add lots of additional fields. It also has like 100 options for changing the algorithm (including things like changing the importance of spelling errors in search, and its eagerness to correct them), so playing around with that might make sense.
With a configuration change, the search engine now understands that karma is important in ranking posts and comments. (It unfortunately doesn’t have access to karma for users.)
Curious what the problem with the current search engine is? Agree that it’s important to be able to find forum posts via Google, which is currently an EA Forum specific issue, but improvements to the search likely also affect LessWrong, so I am curious in getting more detail on that.
Posts are not listed in order of relevance. You need to know exact words from the post you’re searching for in order to find it—preferably exact words from the title.
For example, if I wanted to find your post from four days ago on long term future grants and typed in “grants”, your post wouldn’t appear, because your post uses the word “grant” in the title instead.
For example, if I wanted to find your post from four days ago on long term future grants and typed in “grants”, your post wouldn’t appear, because your post uses the word “grant” in the title instead.
FYI, this was a very helpful concrete example.
On reflection your reasoning is false though—it’s not because the post uses the word ‘grant’. If I search ‘grant’ I get almost identical results, certainly the first 6 are the same. If I search ‘ltf grants’ I get the right thing even though neither ‘ltf’ or ‘grants’ is in the title. I also think that it’s not like there aren’t a lot of other posts you could be searching for with the word ‘grant’ - it isn’t just random other posts, there are *many* posts withing ~2x karma that have that word in the title.
Still, I share a vague sense that something about search is not quite right, though I can’t put my finger on it.
(Edit: This was written before Khorton edited a concrete example into their comment)
Interesting. I haven’t had many issues with the search. I mostly just wanted it to have more options that I can tweak (like restricting it to a specific time period and author). If you know of any site (that isn’t a major search engine provider) that has search that does better here, I would be curious to look into what technology they use (we use Algolia, which seems to be one of the most popular search providers out there, and people seem to generally be happy with it). It might also be an issue of configuration.
Speaking to the google search results – It’s pretty hard to just rise up the google rankings. We’ve done the basic advice: the crawled page contains the post titles and keywords, made sure google finds the mobile view is satisfactory. It’s likely there more we can do but it’s not straightforward. Complicating matters is that during the great spampocalypse in May, we were hit with a punitive action from google, because we were polluting their ranking algorithm with spam links. You may remember a time when there were no results linking to posts at all. We fixed it, but it’s possible (and I’d guess likely) that we’re still getting dinged for that. Unfortunately, google gives us no way of knowing.
Please fix the EA forum search engine and/or make it easier to find forum posts through Google.
On the whole, I really like the search engine. But one small bug you may want to fix is that occasionally the wrong results appear under ‘Users’. For example, if you type ‘Will MacAskill’, the three results that show up are posts where the name ‘Will MacAskill’ appears in the title, rather than the user Will MacAskill.
EDIT: Mmh, this appears to happen because a trackback to Luke Muehlhauser’s post, ‘Will MacAskill on Normative Uncertainty’, is being categorized as the name of a user. So, not a bug with the search engine as such, but still something that the EA Forum tech team may want to fix.
Oh the joys of a long legacy of weird code. I’ve deleted those accounts, although I’m sad to report that our search engine is not smart enough to figure out that “Will MacAskill” should return “William_MacAskill”
Is there a way to give Algolia additional information from the user’s profile so that it can fuzzy search it?
We could probably add a nickname field that we set manually.
Yeah, you can add lots of additional fields. It also has like 100 options for changing the algorithm (including things like changing the importance of spelling errors in search, and its eagerness to correct them), so playing around with that might make sense.
With a configuration change, the search engine now understands that karma is important in ranking posts and comments. (It unfortunately doesn’t have access to karma for users.)
This doesn’t fix the example I put forward, but it does make the search function more understandable and less frustrating. Thanks!
Oh, interesting. LessWrong always had that, and I never even thought about that maybe being a configuration difference between the two sites.
Curious what the problem with the current search engine is? Agree that it’s important to be able to find forum posts via Google, which is currently an EA Forum specific issue, but improvements to the search likely also affect LessWrong, so I am curious in getting more detail on that.
Posts are not listed in order of relevance. You need to know exact words from the post you’re searching for in order to find it—preferably exact words from the title.
For example, if I wanted to find your post from four days ago on long term future grants and typed in “grants”, your post wouldn’t appear, because your post uses the word “grant” in the title instead.
FYI, this was a very helpful concrete example.
On reflection your reasoning is false though—it’s not because the post uses the word ‘grant’. If I search ‘grant’ I get almost identical results, certainly the first 6 are the same. If I search ‘ltf grants’ I get the right thing even though neither ‘ltf’ or ‘grants’ is in the title. I also think that it’s not like there aren’t a lot of other posts you could be searching for with the word ‘grant’ - it isn’t just random other posts, there are *many* posts withing ~2x karma that have that word in the title.
Still, I share a vague sense that something about search is not quite right, though I can’t put my finger on it.
(Edit: This was written before Khorton edited a concrete example into their comment)
Interesting. I haven’t had many issues with the search. I mostly just wanted it to have more options that I can tweak (like restricting it to a specific time period and author). If you know of any site (that isn’t a major search engine provider) that has search that does better here, I would be curious to look into what technology they use (we use Algolia, which seems to be one of the most popular search providers out there, and people seem to generally be happy with it). It might also be an issue of configuration.
Speaking to the google search results – It’s pretty hard to just rise up the google rankings. We’ve done the basic advice: the crawled page contains the post titles and keywords, made sure google finds the mobile view is satisfactory. It’s likely there more we can do but it’s not straightforward. Complicating matters is that during the great spampocalypse in May, we were hit with a punitive action from google, because we were polluting their ranking algorithm with spam links. You may remember a time when there were no results linking to posts at all. We fixed it, but it’s possible (and I’d guess likely) that we’re still getting dinged for that. Unfortunately, google gives us no way of knowing.