dirbot-mysql
dirbot-mysql copied to clipboard
rmax
Scrapy project based on dirbot to show how to use Twisted's adbapi to store the scraped data in MySQL.
scrapy list Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.5/bin/scrapy", line 11, in sys.exit(execute()) File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/scrapy/cmdline.py", line 108, in execute settings = get_project_settings() File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/scrapy/utils/project.py", line 60, in get_project_settings settings.setmodule(settings_module_path, priority='project')...
I am using your code snippet and getting next error: ``` --- --- File "/usr/lib/python2.7/dist-packages/twisted/enterprise/adbapi.py", line 452, in _runInteraction conn.rollback() File "/usr/lib/python2.7/dist-packages/twisted/enterprise/adbapi.py", line 49, in rollback self._connection.rollback() _mysql_exceptions.ProgrammingError: (2014, "Commands...
Not so much of an issue, but more a question about how to best begin getting data into the environment. I'm looking at getting a scraper / pipeline set up...
