Scrapy : Another Method To Avoid A Lot Of Try Except

I want to ask a question When I use css selector,extract() will make the output thing a list So if the css selector didn't have value It will show error in terminal(like below),an

Solution 1:

You can use a separate function for extraction of data. e.g for text nodes, sample code is here

    def extract_text(node):
        if not node:
            return ''
        _text = './/text()'
        extracted_list = [x.strip() for x in node.xpath(_text).extract() if len(x.strip()) > 0]
        if not extracted_list:
            return ''
        return ' '.join(extracted_list)

and you can call this method like this

self.extract_text(sel.css("your_path"))