robots.txt for AI bots means using the standard robots.txt file to allow or refuse specific AI crawlers by their published user-agent names.
The robots exclusion protocol was formalised as RFC 9309 in September 2022. Each AI company names its user agent, so a site can, for instance, disallow a training crawler while allowing a search fetcher.
The file is a request, not a lock. Reputable companies honour it, but it gives no technical protection on its own.
A common mistake is a blanket disallow copied from a template years ago that now quietly hides a site from every assistant. Checking the file is one of the first steps in any AI visibility review.