Allow robots.txt to get exposed without authentication

(topic deleted by author)