Google explains the use cases for its different crawler types April 24, 2023 There is Googlebot, special-case crawlers and user-triggered fetchers.