# allow all User-agent: * Disallow: # disallow some pages example # User-agent: * # Disallow: /private/ # Disallow: /restricted/