I am trying to run a scraper to scrape images from Houzz.com and running in a lot of issues when trying to crawl a lot of product overviews. One of the issues is that I get a connection refused 111 error and the crawler just terminates... I am running my crawler on an Amazon EC2 instance.
Traceback (most recent call last):
File "/home/ubuntu/anaconda3/envs/python3/lib/python3.6/site-packages/scrapy/core/downloader/middleware.py", line 44, in process_request
defer.returnValue((yield download_func(request=request, spider=spider)))
twisted.internet.error.ConnectionRefusedError: Connection was refused by other side: 111: Connection refused.
Hopefully someone can point me in the right direction to fix this.