Identify a crawler from the Botscope catalogue, see how it is verified, and draft a robots.txt policy. With a Botscope account, review which agents reached your site.