Reddit announced on Tuesday that it’s updating its Robots Exclusion Protocol (robots.txt file), which tells automated web bots whether they are permitted to crawl a site. Historically, robots.txt file ...