We want these forums to show up in search results, which are created by bots crawling the web and obeying robots.txt, so excluding bots would be very costly
Yeah, I realised the morning after I posted this comment. Oh well, a little embarrassment is healthy every now and then. ^^
We want these forums to show up in search results, which are created by bots crawling the web and obeying robots.txt, so excluding bots would be very costly
Yeah, I realised the morning after I posted this comment. Oh well, a little embarrassment is healthy every now and then. ^^