\bafd\b|hamburg.*berall|neevabot|google web preview|google favicon|googlebot|feedfetcher|siteexplorer|www.wotbox.com/bot|publiclibraryarchive.org|siteexplorer.info|privacyawarebot|www.safedns.com|flippybearbot*|semanticbot|bingbot|bingpreview/1.0b|msnbot|mfibot|google-site-verification|uptimebot|ccbot|stratagems kumo|finbot|spider@seoscanners.net|domaincrawler|yahoo! slurp|jobboersebot|www.linguee.com/bot|megaindex.ru|duckduckgo-favicons-bot|duckduckbot|yacybot|grouphigh|smtbot|sogou web spider|netestate ne crawler|yandexbot|nmap scripting engine|bubing|dataprovider\.com|backlinkcrawler |barkrowler|007ac9.*crawler|b-l-i-t-z-b-o-t|adscanner|extlinksbot|aihitbot|nyu internet census|mauibot|linkedinbot|360spider|nsrbot|blexbot|gdpr-bot|iabtechlab.*crawler|nimbostratus-bot|ahrefsbot|tracemyfile|implisensebot|adsbot|pleskbot|spiderling|tagvisit|checkmarknetwork|ssearch_bot|compatible; qwantify|seekportbot|oppo a33|aspiegelbot|domainstatsbot|petalbot|komodiabot|niuebot|applebot|ioncrawl|zoominfobot|sitebot|webprosbot|expanse.*(paloalto|expanseinc)|netsystemsresearch|birdcrawlerbot|dataforseobot|serpstatbot|dnbcrawler|go-http-client|alittle client|speedy.*spider|orbbot|awariobot|mixrankbot|2ip bot|ezooms.bot|dotbot|webwikibot|bsbot|companyspotter|pinterestbot|www.80legs.com|ibusiness shopcrawler|t3versionsbot|siteauditbot|mj12bot|memorybot|kstandbot|domainsbot.*pandalytics|dmbot|crawler.*sistrix|sistrix.*crawler|urlsumabot|geedobot|phantombot|bytespider|velenpublicwebcrawler|rainbot|amazonbot|gptbot|claudebot|www.botje.com/plukkie.htm|test-bot|spawning-ai|wbsearchbot|baiduspider|psbot|pixraybot|icjobs|www.askpeter.info/bot.html|digext.*dts agent |semager-bots|checklink|catchbot|cliqzbot|semrushbot|xovibot|\bobot\b|wonderbot\/*|ru_bot|archive.org_bot|nutch|wevikabot|savetheworldheritage|\bmindupbot\b|netcraftsurveyagent|semalt.semalt.com/crawler.php|openlinkprofiler.org/bot|meanpathbot|exabot|oberforstmeister|rogerbot|linkdexbot|lssrocketcrawler|lightspeedsystemscrawler|compatible.*indy library|turnitinbot|bigfinder.de|netzcheckbot|ia_archiver