# www.robotstxt.org/ # Allow crawling of all content User-agent: * Disallow: /user/sign/in Disallow: /users/sign/up Disallow: /admin