Google explains the use cases for its different crawler types

There is Googlebot, special-case crawlers and user-triggered fetchers.

You may also like...

Good Morning C(_)