Scrapy-selectors总结


Scrapy 自带的提取数据的机制,由XPath或CSS表达式指定的HTML文档的某些部分

例如:

<html>
 <head>
  <base href='http://example.com/' />
  <title>Example websitetitle>
 head>
 <body>
  <div id='images'>
   <a href='image1.html'>Name: My image 1 <br /><img src='image1_thumb.jpg' />a>
   <a href='image2.html'>Name: My image 2 <br /><img src='image2_thumb.jpg' />a>
   <a href='image3.html'>Name: My image 3 <br /><img src='image3_thumb.jpg' />a>
   <a href='image4.html'>Name: My image 4 <br /><img src='image4_thumb.jpg' />a>
   <a href='image5.html'>Name: My image 5 <br /><img src='image5_thumb.jpg' />a>
  div>
 body>
html>



XPath 选择标签中的文本:
>>> response.xpath('//title/text()').extract()
['Example website']


CSS 选择标签中的文本:
>>> response.css('title::text').extract()
['Example website']

XPath 或 CSS 选择标签的属性:
>>> response.xpath('//base/@href').extract()
['http://example.com/']

>>> response.css('base::attr(href)').extract()
['http://example.com/']