通过日志文件查看百度蜘蛛是否抓取
cat /var/log/httpd/access_log-20170423|grep Baiduspider
cat /var/log/httpd/access_log-20170423|grep 360
---------------------------------------------------------------------------
cat /var/log/httpd/access_log-20170423|grep Baiduspider
cat /var/log/httpd/access_log-20170423|grep 360
---------------------------------------------------------------------------