|
|
禁止所有爬虫搜索引擎蜘蛛收录网站任何页面的robots终极写法：
- User-agent: Baiduspider
- Disallow: /
- User-agent: Googlebot
- Disallow: /
- User-agent: MSNBot
- Disallow: /
- User-agent: Baiduspider-image
- Disallow: /
- User-agent: YoudaoBot
- Disallow: /
- User-agent: Sogou web spider
- Disallow: /
- User-agent: Sogou inst spider
- Disallow: /
- User-agent: Sogou spider2
- Disallow: /
- User-agent: Sogou blog
- Disallow: /
- User-agent: Sogou News Spider
- Disallow: /
- User-agent: Sogou Orion spider
- Disallow: /
- User-agent: ChinasoSpider
- Disallow: /
- User-agent: Sosospider
- Disallow: /
- User-agent: yisouspider
- Disallow: /
- User-agent: EasouSpider
- Disallow: /
- User-agent: *
- Disallow: /
robots.txt
(651 Bytes, 下载次数: 1)
阅读权限: 10
|
|
帐号登录
QQ登录
微信登录
发表于 2017-2-17 17:54:12
