User-agent: * 
Allow: /
Disallow: /pdf/
Disallow: /login/
Disallow: /auth/