How to create a robots.txt file

https://developers.google.com/search/docs/advanced/robots/create-robots-txt