1 <rule name="Block spider"> 2 <match url="(^robots.txt$)" ignoreCase="false" negate="true" /> 3 <conditions> 4 <add input="{HTTP_USER_AGENT}" pattern="Apache-HttpClient|SemrushBot|Webdup|AcoonBot|AhrefsBot|Ezooms| EdisterBot|EC2LinkFinder|jikespider|Purebot|MJ12bot|WangIDSpider|WBSearchBot|Wotbox|xbfMozilla|Yottaa|YandexBot| Jorgee|SWEBot|spbot|TurnitinBot-Agent|curl|perl|Python|Wget|Xenu|ZmEu|facebookexternalhit" ignoreCase="true" /> 5 </conditions> 6 <action type="AbortRequest" /> 7 </rule>
记录一下，简单测试可用
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。