Mac Os X El Capitan - Scrapy/python Importerror: Cannot Import Name Xmlrpc_client

I am trying to use Scrapy on Mac OS X El Capitan. I have zsh installed and I have tried everything that I could find online to fix this issue. I have also looked at Scrapy throws I

Solution 1:

I've had nothing but pain mucking with the Mac OS X system Python libraries installed in the /Library/Python directory. What has worked well for me is a combination of MacPorts and virtualenv:

1. Install MacPorts

2. Install Python, pip, and virtualenv from MacPorts:

   /opt/local/bin/port install python27
   /opt/local/bin/port install py27-pip
   /opt/local/bin/port install py27-virtualenv
   

3. Setup virtualenv:

   /opt/local/bin/virtualenv-2.7 myenv
   

4. Activate virtualenv (don't forget the dot!)

   . myenv/bin/activate
   

5. Install scrapy

   pip install scrapy
   

This way, the system Python libraries are untouched and you can install whatever packages you like without having to remove or upgrade existing packages.

Solution 2:

What helped me was to uninstall six and scrapy and then install again:

pip uninstall six
pip uninstall scrapy

pip install six
pip install scrapy

Run with sudo if necessary.

---

Or, you can also try upgrading six and scrapy:

pip install --upgrade scrapy
pip install --upgrade six

Solution 3:

Try uninstalling via pip and then reinstall using easy_install command. I had the same trouble with another python module and doing it this way fixed the issue for me on Mac OS X El Capitan.

Solution 4:

I believe the best solution on OS X should be "Don’t use system python". It will make life easier. This link shows how to do this.

There’s a known issue that prevents pip from updating system packages. This has to be addressed to successfully install Scrapy and its dependencies. Here are some proposed solutions:

(Recommended) Don’t use system python, install a new, updated version that doesn’t conflict with the rest of your system. Here’s how to do it using the homebrew package manager:

1. Install homebrew following the instructions in http://brew.sh/
2. Update your PATH variable to state that homebrew packages should be used before system packages (Change .bashrc to .zshrc accordantly if you’re using zsh as default shell):

echo "export PATH=/usr/local/bin:/usr/local/sbin:$PATH" >> ~/.bashrc

3. Reload .bashrc to ensure the changes have taken place:

source ~/.bashrc

4. Install python:

brew install python

5. Latest versions of python have pip bundled with them so you won’t need to install it separately. If this is not the case, upgrade python:

brew update; brew upgrade python