Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Server Farms 2026 Continuing discussion of hosting and data center IP ranges |
6 | blend27 | 3:27 pm Feb 21, 2026 | ||||
| Server Farms 2025 Continuing discussion of hosting and data center IP ranges |
5 | not2easy | 5:09 pm Mar 1, 2025 | ||||
| AI UAs - Identifying the crawlers AI uses |
1 | not2easy | 5:38 pm Oct 18, 2023 | ||||
| At Home Without the Robots: 2023 edition |
5 | lucy24 | 11:39 am Jul 29, 2023 | ||||
| At Home with the Law-Abiding Robots (2021) |
2 | lucy24 | 8:32 pm Sep 6, 2021 | ||||
| At Home with the Robots: 2020 edition |
11 | lucy24 | 2:48 pm Aug 13, 2020 | ||||
| At Home with the Robots: 2019 edition |
3 | lucy24 | 3:52 am Mar 13, 2019 | ||||
| At Home with the Robots: 2017 edition Critique of the Year's Active User Agents |
15 | lucy24 | 3:52 pm May 18, 2017 | ||||
| User Agents of Programming Libraries and Command Line Tools Common user agents |
14 | incrediBILL | 8:54 pm Apr 29, 2008 | ||||
| Reflectionbot |
2 | lucy24 | 6:57 pm Sep 12, 2026 | ||||
| 0 | 12:00 am Jan 1, 1970 | ||||||
| User Agent Client Hints Detection (UA-CH) Is anyone using this? |
1 | SumGuy | 12:41 pm Sep 7, 2026 | ||||
| Annoying Crawler, Bot, or what? Malicious traffic from a certain ISP/network[2] ( 1 2 ) |
60 | PeterTroyWilliams | 8:15 pm Sep 6, 2026 | ||||
| MathPicDatasetCrawler |
3 | lucy24 | 4:02 am Sep 2, 2026 | ||||
| Cloudflare now in the email pollution business 104.30.4.0/22 |
1 | SumGuy | 1:46 am Aug 28, 2026 | ||||
| Penetration testing using AI bots Did I just see an OpenAI example? |
3 | SumGuy | 12:55 am Aug 28, 2026 | ||||
| URLBuilder this month's winner |
2 | lucy24 | 1:53 am Aug 26, 2026 | ||||
| Scraper - Komi Inc 140.235.142.0/24 |
1 | Bewenched | 9:08 pm Aug 22, 2026 | ||||
| SkypeUriPreview - links pasted into MS 365 Outlook 52.112.39.133 |
5 | Bewenched | 5:32 pm Aug 15, 2026 | ||||
| IbouBot 217.113.196.90 |
2 | Bewenched | 5:21 pm Aug 9, 2026 | ||||
| 204.194.48.0/21 Bots hammering |
1 | Bewenched | 4:23 am Aug 6, 2026 | ||||
| Hits from 31.105.x.x (Mistralbot ?) |
1 | SumGuy | 12:12 am Jul 19, 2026 | ||||
| Interesting set of hits using forged searchbot User Agents Including DeepSeek and xAI (?!) |
4 | SumGuy | 12:34 pm Jul 2, 2026 | ||||
| Hacker spoofing as GoogleBot 136.234.222.64 |
8 | Bewenched | 3:23 pm May 28, 2026 | ||||
| Microsoft, AS8075. Google, AS396982. |
9 | shawnb61 | 6:13 pm May 24, 2026 |
