Pip install scrapy ошибка

Before installing Scrapy on Python 3.5 on Windows 10 in virtualenv
Install lxml,PyDispatcher then simpy install Scrapy code is attached.

    (pymote_env) C:UsersDDSHARMADocumentspymote_env>pip install PyDispatcher
Collecting PyDispatcher
Installing collected packages: PyDispatcher
Successfully installed PyDispatcher-2.0.5

(pymote_env) C:UsersDDSHARMADocumentspymote_env>pip install Scrapy
Collecting Scrapy
  Using cached Scrapy-1.2.1-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): six>=1.5.2 in c:usersddsharmadocumentspymote_envlibsite-packages (from Scrapy)
Collecting service-identity (from Scrapy)
  Using cached service_identity-16.0.0-py2.py3-none-any.whl
Collecting Twisted>=10.0.0 (from Scrapy)
  Using cached Twisted-16.4.1.tar.bz2
Collecting parsel>=0.9.3 (from Scrapy)
  Using cached parsel-1.0.3-py2.py3-none-any.whl
Collecting pyOpenSSL (from Scrapy)
  Using cached pyOpenSSL-16.2.0-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): PyDispatcher>=2.0.5 in c:usersddsharmadocumentspymote_envlibsite-packages (from Scrapy)
Requirement already satisfied (use --upgrade to upgrade): lxml in c:usersddsharmadocumentspymote_envlibsite-packages (from Scrapy)
Collecting queuelib (from Scrapy)
  Using cached queuelib-1.4.2-py2.py3-none-any.whl
Collecting w3lib>=1.15.0 (from Scrapy)
  Using cached w3lib-1.15.0-py2.py3-none-any.whl
Collecting cssselect>=0.9 (from Scrapy)
  Using cached cssselect-1.0.0-py2.py3-none-any.whl
Collecting pyasn1-modules (from service-identity->Scrapy)
  Using cached pyasn1_modules-0.0.8-py2.py3-none-any.whl
Collecting pyasn1 (from service-identity->Scrapy)
  Using cached pyasn1-0.1.9-py2.py3-none-any.whl
Collecting attrs (from service-identity->Scrapy)
  Using cached attrs-16.2.0-py2.py3-none-any.whl
Collecting zope.interface>=4.0.2 (from Twisted>=10.0.0->Scrapy)
  Using cached zope.interface-4.3.2-cp35-cp35m-win32.whl
Collecting cryptography>=1.3.4 (from pyOpenSSL->Scrapy)
  Using cached cryptography-1.5.2-cp35-cp35m-win32.whl
Requirement already satisfied (use --upgrade to upgrade): setuptools in c:usersddsharmadocumentspymote_envlibsite-packages (from zope.interface>=4.0.2->Twisted>=10.0.0->Scrapy)
Collecting idna>=2.0 (from cryptography>=1.3.4->pyOpenSSL->Scrapy)
  Using cached idna-2.1-py2.py3-none-any.whl
Collecting cffi>=1.4.1 (from cryptography>=1.3.4->pyOpenSSL->Scrapy)
  Using cached cffi-1.8.3-cp35-cp35m-win32.whl
Collecting pycparser (from cffi>=1.4.1->cryptography>=1.3.4->pyOpenSSL->Scrapy)
  Downloading pycparser-2.16.tar.gz (230kB)
    100% |################################| 235kB 286kB/s
Building wheels for collected packages: Twisted, pycparser
  Running setup.py bdist_wheel for Twisted ... done
  Stored in directory: C:UsersDDSHARMAAppDataLocalpipCachewheelse5362e7b4cea7df9113fb2818b224eb5d143be981568d9c43057a0a
  Running setup.py bdist_wheel for pycparser ... done
  Stored in directory: C:UsersDDSHARMAAppDataLocalpipCachewheels34ccb534e805b130a9227f0681d5517cf554d950bb0361c3a373496f
Successfully built Twisted pycparser
Installing collected packages: pyasn1, pyasn1-modules, idna, pycparser, cffi, cryptography, pyOpenSSL, attrs, service-identity, zope.interface, Twisted, w3lib, cssselect, parsel, queuelib, Scrapy
Successfully installed Scrapy-1.2.1 Twisted-16.4.1 attrs-16.2.0 cffi-1.8.3 cryptography-1.5.2 cssselect-1.0.0 idna-2.1 parsel-1.0.3 pyOpenSSL-16.2.0 pyasn1-0.1.9 pyasn1-modules-0.0.8 pycparser-2.16 queuelib-1.4.2 service-identity-16.0.0 w3lib-1.15.0 zope.interface-4.3.2

(pymote_env) C:UsersDDSHARMADocumentspymote_env>

Источник

Errors exists about the Twisted when installing scrapy in window by using pip install scrapy. Any one know how to fix it.

 copying srctwistedinternettest_yieldfromtests.py.3only -> buildlib.win32-2.7twistedinternettest
    creating buildlib.win32-2.7twistedinternettestfake_CAs
    copying srctwistedinternettestfake_CAschain.pem -> buildlib.win32-2.7twistedinternettestfake_CAs
    copying srctwistedinternettestfake_CAsnot-a-certificate -> buildlib.win32-2.7twistedinternettestfake_CAs
    copying srctwistedinternettestfake_CAsthing1.pem -> buildlib.win32-2.7twistedinternettestfake_CAs
    copying srctwistedinternettestfake_CAsthing2-duplicate.pem -> buildlib.win32-2.7twistedinternettestfake_CAs
    copying srctwistedinternettestfake_CAsthing2.pem -> buildlib.win32-2.7twistedinternettestfake_CAs
    copying srctwistedmailtestrfc822.message -> buildlib.win32-2.7twistedmailtest
    copying srctwistedpythontest_deprecatetests.py.3only -> buildlib.win32-2.7twistedpythontest
    copying srctwistedwordsiminstancemessenger.glade -> buildlib.win32-2.7twistedwordsim
    copying srctwistedwordsxishxpathparser.g -> buildlib.win32-2.7twistedwordsxish
    running build_ext
    building 'twisted.test.raiser' extension
    error: INCLUDE environment variable is empty

    ----------------------------------------
Command "c:python27python.exe -u -c "import setuptools, tokenize;__file__='c:\users\thomas~1\appdata\local\temp\pip-build-3oirm6\Twisted\setup.py';
f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('rn', 'n');f.close();exec(compile(code, __file__, 'exec'))"
 install --record c:usersthomas~1appdatalocaltemppip-xvbgd2-recordinstall-record.txt --single-version-externally-managed --compile" 
failed with error code 1 in c:usersthomas~1appdatalocaltemppip-build-3oirm6Twisted
Источник

I am trying to install Scrapy on Windows and have followed steps in the Scrapy installation guide to install all the dependencies. However, I got the following error message when I tried to use easy_install:

Download error on https://pypi.python.org/simple/Scrapy/: **[Errno 10061] 
No connection could be made because the target machine actively refused 
it -- Some packages may not be found!**
Couldn't find index page for 'Scrapy' (maybe misspelled?)

Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/
Download error on https://pypi.python.org/simple/: **[Errno 10061] 
No connection could be made because the target machine actively refused 
it -- Some packages may not be found!**
No local packages or download links found for Scrapy
error: Could not find suitable distribution for 
Requirement.parse('Scrapy')

I also tried to use pip but it doesn’t work either:

Downloading/unpacking Scrapy
 **Cannot fetch index base URL https://pypi.python.org/simple/
  Could not find any downloads that satisfy the requirement Scrapy**
Cleaning up...
No distributions at all found for Scrapy

I went to pip.log and it says

Could not fetch URL https://pypi.python.org/simple/Scrapy/: **connection
error: HTTPSConnectionPool(host='pypi.python.org', port=443): Max 
retries exceeded with url: /simple/Scrapy/ (Caused by 
<class 'socket.error'>: [Errno 10061] No connection could be made 
because the target machine actively refused it)**
  Will skip URL https://pypi.python.org/simple/Scrapy/ when looking 
for download links for Scrapy

I can access https://pypi.python.org/simple/Scrapy/ directly from internet explorer but not sure why pip or easy_install can’t access the link.

I’m using Anaconda python so easy_install and pip are already part of the package. I also had the following error at the last stage of installing pywin32:

close failed in file object destructor:

sys.excepthook is missing

lost sys.stderr

Is this the reason easy_install and pip failed? Could anyone help with these please?

Источник

December 16 2016, 22:44

Category:

  • IT
  • Cancel

При установке фреймворка для краулинга Scrapy на *win систему сталкиваешься с ошибками, которые связаны с отсутствием тех или иных компонентов. Все фиксы валидны только для Python 3.5 и *win.

Первое что должно быть при установке это Common Tools for Visual C++ 2015 или установленная Visual Studio 2015. Если студию устанавливать не хочется, да и не зачем, тогда скачиваем пакет отдельно.

Если студия уже установлена, но ошибки все рано возникают, при этом вывод рассказывает что-то о том, что не может отыскать vcvarsall.bat:
Открываем «Панель управленияВсе элементы панели управленияПрограммы и компоненты» ищем там установленную студию, нажимаем «изменить«. Далее выбираем «Modify» и отмечаем пункт «Programming Laguages > Visual C++ > Common Tools for Visual C++ 2015«. Инсталлятор, скорее всего, попросит установочный диск.

Если не собирается cryptography:
Скорее всего будет не найден OpenSSL:

fatal error C1083: Cannot open include file: 'openssl/opensslv.h': No such file or directory

пробуем установить командой

pip install --only-binary cryptography

если не получается, не отчаиваемся — скачиваем OpenSSL. (Для Python 3.5 будет 2015 версия, архитектура соответственно), распаковываем, прописываем пути:

pathtovcvarsall.bat x86_amd64
set LIB=C:OpenSSL-win64lib;%LIB%
set INCLUDE=C:OpenSSL-win64include;%INCLUDE%

Пробуем установить:

pip install cryptography

Если пути прописали верно, то все должно получиться.
Подробнее об установке cryptography

Если не хочет собираться lxml, наверняка, вывод будет следующим:

<...>
*********************************************************************************
Could not find function xmlCheckVersion in library libxml2. Is libxml2 installed?
*********************************************************************************
error: command '<...>Microsoft Visual Studio 14.0VCBINx86_amd64cl.exe' failed with exit status 2

Исправляется достаточно просто:

set STATICBUILD=true

Данная строчка «расскажет» инсталлятору о том, что если тому чего-то не хватает, то пускай подтягивает зависимости автоматически перед сборкой lxml.
И пробуем поставить:

pip install lxml

После всего этого, конечно же, устанавливаем scrapy:

pip install scrapy

Источник
pip install scrapy
Downloading/unpacking scrapy
  Downloading Scrapy-0.24.2-py2-none-any.whl (502kB): 502kB downloaded
Downloading/unpacking pyOpenSSL (from scrapy)
  Downloading pyOpenSSL-0.14.tar.gz (128kB): 128kB downloaded
  Running setup.py (path:/home/elie/.virtualenvs/stat/build/pyOpenSSL/setup.py) egg_info for package pyOpenSSL

    warning: no previously-included files matching '*.pyc' found anywhere in distribution
    no previously-included directories found matching 'doc/_build'
Requirement already satisfied (use --upgrade to upgrade): six>=1.5.2 in /home/elie/.virtualenvs/stat/lib/python2.7/site-packages (from scrapy)
Downloading/unpacking queuelib (from scrapy)
  Downloading queuelib-1.1.1.tar.gz
  Running setup.py (path:/home/elie/.virtualenvs/stat/build/queuelib/setup.py) egg_info for package queuelib

Requirement already satisfied (use --upgrade to upgrade): lxml in /home/elie/.virtualenvs/stat/lib/python2.7/site-packages (from scrapy)
Downloading/unpacking w3lib>=1.2 (from scrapy)
  Downloading w3lib-1.6.tar.gz
  Running setup.py (path:/home/elie/.virtualenvs/stat/build/w3lib/setup.py) egg_info for package w3lib

Requirement already satisfied (use --upgrade to upgrade): cssselect>=0.9 in /home/elie/.virtualenvs/stat/lib/python2.7/site-packages (from scrapy)
Downloading/unpacking Twisted>=10.0.0 (from scrapy)
  Downloading Twisted-14.0.0.tar.bz2 (4.3MB): 4.3MB downloaded
  Running setup.py (path:/home/elie/.virtualenvs/stat/build/Twisted/setup.py) egg_info for package Twisted

Downloading/unpacking cryptography>=0.2.1 (from pyOpenSSL->scrapy)
  Downloading cryptography-0.5.2.tar.gz (319kB): 319kB downloaded
  Running setup.py (path:/home/elie/.virtualenvs/stat/build/cryptography/setup.py) egg_info for package cryptography
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    c/_cffi_backend.c:13:17: fatal error: ffi.h: No such file or directory
     #include <ffi.h>
                     ^
    compilation terminated.
    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "/home/elie/.virtualenvs/stat/build/cryptography/setup.py", line 174, in <module>
        "test": PyTest,
      File "/usr/lib/python2.7/distutils/core.py", line 111, in setup
        _setup_distribution = dist = klass(attrs)
      File "/home/elie/.virtualenvs/stat/local/lib/python2.7/site-packages/setuptools/dist.py", line 239, in __init__
        self.fetch_build_eggs(attrs.pop('setup_requires'))
      File "/home/elie/.virtualenvs/stat/local/lib/python2.7/site-packages/setuptools/dist.py", line 263, in fetch_build_eggs
        parse_requirements(requires), installer=self.fetch_build_egg
      File "/home/elie/.virtualenvs/stat/local/lib/python2.7/site-packages/pkg_resources.py", line 564, in resolve
        dist = best[req.key] = env.best_match(req, self, installer)
      File "/home/elie/.virtualenvs/stat/local/lib/python2.7/site-packages/pkg_resources.py", line 802, in best_match
        return self.obtain(req, installer) # try and download/install
      File "/home/elie/.virtualenvs/stat/local/lib/python2.7/site-packages/pkg_resources.py", line 814, in obtain
        return installer(requirement)
      File "/home/elie/.virtualenvs/stat/local/lib/python2.7/site-packages/setuptools/dist.py", line 313, in fetch_build_egg
        return cmd.easy_install(req)
      File "/home/elie/.virtualenvs/stat/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 593, in easy_install
        return self.install_item(spec, dist.location, tmpdir, deps)
      File "/home/elie/.virtualenvs/stat/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 623, in install_item
        dists = self.install_eggs(spec, download, tmpdir)
      File "/home/elie/.virtualenvs/stat/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 809, in install_eggs
        return self.build_and_install(setup_script, setup_base)
      File "/home/elie/.virtualenvs/stat/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1015, in build_and_install
        self.run_setup(setup_script, setup_base, args)
      File "/home/elie/.virtualenvs/stat/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1003, in run_setup
        raise DistutilsError("Setup script exited with %s" % (v.args[0],))
    distutils.errors.DistutilsError: Setup script exited with error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
    Complete output from command python setup.py egg_info:
    Package libffi was not found in the pkg-config search path.

Perhaps you should add the directory containing `libffi.pc'

to the PKG_CONFIG_PATH environment variable

No package 'libffi' found

Package libffi was not found in the pkg-config search path.

Perhaps you should add the directory containing `libffi.pc'

to the PKG_CONFIG_PATH environment variable

No package 'libffi' found

Package libffi was not found in the pkg-config search path.

Perhaps you should add the directory containing `libffi.pc'

to the PKG_CONFIG_PATH environment variable

No package 'libffi' found

Package libffi was not found in the pkg-config search path.

Perhaps you should add the directory containing `libffi.pc'

to the PKG_CONFIG_PATH environment variable

No package 'libffi' found

Package libffi was not found in the pkg-config search path.

Perhaps you should add the directory containing `libffi.pc'

to the PKG_CONFIG_PATH environment variable

No package 'libffi' found

c/_cffi_backend.c:13:17: fatal error: ffi.h: No such file or directory

 #include <ffi.h>

                 ^

compilation terminated.

Traceback (most recent call last):

  File "<string>", line 17, in <module>

  File "/home/elie/.virtualenvs/stat/build/cryptography/setup.py", line 174, in <module>

    "test": PyTest,

  File "/usr/lib/python2.7/distutils/core.py", line 111, in setup

    _setup_distribution = dist = klass(attrs)

  File "/home/elie/.virtualenvs/stat/local/lib/python2.7/site-packages/setuptools/dist.py", line 239, in __init__

    self.fetch_build_eggs(attrs.pop('setup_requires'))

  File "/home/elie/.virtualenvs/stat/local/lib/python2.7/site-packages/setuptools/dist.py", line 263, in fetch_build_eggs

    parse_requirements(requires), installer=self.fetch_build_egg

  File "/home/elie/.virtualenvs/stat/local/lib/python2.7/site-packages/pkg_resources.py", line 564, in resolve

    dist = best[req.key] = env.best_match(req, self, installer)

  File "/home/elie/.virtualenvs/stat/local/lib/python2.7/site-packages/pkg_resources.py", line 802, in best_match

    return self.obtain(req, installer) # try and download/install

  File "/home/elie/.virtualenvs/stat/local/lib/python2.7/site-packages/pkg_resources.py", line 814, in obtain

    return installer(requirement)

  File "/home/elie/.virtualenvs/stat/local/lib/python2.7/site-packages/setuptools/dist.py", line 313, in fetch_build_egg

    return cmd.easy_install(req)

  File "/home/elie/.virtualenvs/stat/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 593, in easy_install

    return self.install_item(spec, dist.location, tmpdir, deps)

  File "/home/elie/.virtualenvs/stat/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 623, in install_item

    dists = self.install_eggs(spec, download, tmpdir)

  File "/home/elie/.virtualenvs/stat/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 809, in install_eggs

    return self.build_and_install(setup_script, setup_base)

  File "/home/elie/.virtualenvs/stat/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1015, in build_and_install

    self.run_setup(setup_script, setup_base, args)

  File "/home/elie/.virtualenvs/stat/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1003, in run_setup

    raise DistutilsError("Setup script exited with %s" % (v.args[0],))

distutils.errors.DistutilsError: Setup script exited with error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /home/elie/.virtualenvs/stat/build/cryptography
Storing debug log for failure in /home/elie/.pip/pip.log

I used to install scrapy before, never had issues, what’s wrong?

Источник

Понравилась статья? Поделить с друзьями:
  • Pip install r requirements txt ошибка
  • Pip install pytelegrambotapi ошибка
  • Pip install pyinstaller ошибка
  • Pip install pygame ошибка
  • Pip install pyautogui ошибка