ynam icon indicating copy to clipboard operation
ynam copied to clipboard

Docker quickstart works, importing of transactions errors out

Open nathanpaul opened this issue 2 years ago • 13 comments

2023-02-01 14:20:24 Traceback (most recent call last):
2023-02-01 14:20:24   File "/usr/local/lib/python3.8/dist-packages/ynam/main.py", line 24, in main
2023-02-01 14:20:24     mint_transactions = mint_api.get_transactions(start_date=arg('days'))
2023-02-01 14:20:24   File "/usr/local/lib/python3.8/dist-packages/ynam/mint_api.py", line 48, in get_transactions
2023-02-01 14:20:24     client.authorize(self.cookies(), self.key())
2023-02-01 14:20:24   File "/usr/local/lib/python3.8/dist-packages/ynam/mint_api.py", line 58, in cookies
2023-02-01 14:20:24     return ast.literal_eval(file.read())
2023-02-01 14:20:24   File "/usr/lib/python3.8/ast.py", line 59, in literal_eval
2023-02-01 14:20:24     node_or_string = parse(node_or_string, mode='eval')
2023-02-01 14:20:24   File "/usr/lib/python3.8/ast.py", line 47, in parse
2023-02-01 14:20:24     return compile(source, filename, mode, flags,
2023-02-01 14:20:24   File "<unknown>", line 0
2023-02-01 14:20:24     
2023-02-01 14:20:24     ^
2023-02-01 14:20:24 SyntaxError: unexpected EOF while parsing
2023-02-01 14:20:24 
2023-02-01 14:20:24 During handling of the above exception, another exception occurred:
2023-02-01 14:20:24 
2023-02-01 14:20:24 Traceback (most recent call last):
2023-02-01 14:20:24   File "/usr/local/bin/ynam", line 8, in <module>
2023-02-01 14:20:24     sys.exit(main())
2023-02-01 14:20:24   File "/usr/local/lib/python3.8/dist-packages/ynam/main.py", line 26, in main
2023-02-01 14:20:24     mint_api.updateAuth()
2023-02-01 14:20:24   File "/usr/local/lib/python3.8/dist-packages/ynam/mint_api.py", line 66, in updateAuth
2023-02-01 14:20:24     browser = self.browser(
2023-02-01 14:20:24   File "/usr/local/lib/python3.8/dist-packages/mintapi/browser.py", line 55, in __init__
2023-02-01 14:20:24     self.login_and_get_token(
2023-02-01 14:20:24   File "/usr/local/lib/python3.8/dist-packages/mintapi/browser.py", line 110, in login_and_get_token
2023-02-01 14:20:24     self.driver = driver or _create_web_driver_at_mint_com(
2023-02-01 14:20:24   File "/usr/local/lib/python3.8/dist-packages/mintapi/signIn.py", line 311, in _create_web_driver_at_mint_com
2023-02-01 14:20:24     driver = Chrome(options=chrome_options)
2023-02-01 14:20:24   File "/usr/local/lib/python3.8/dist-packages/seleniumrequests/request.py", line 148, in __init__
2023-02-01 14:20:24     super(RequestsSessionMixin, self).__init__(*args, **kwargs)
2023-02-01 14:20:24   File "/usr/local/lib/python3.8/dist-packages/selenium/webdriver/chrome/webdriver.py", line 80, in __init__
2023-02-01 14:20:24     super().__init__(
2023-02-01 14:20:24   File "/usr/local/lib/python3.8/dist-packages/selenium/webdriver/chromium/webdriver.py", line 104, in __init__
2023-02-01 14:20:24     super().__init__(
2023-02-01 14:20:24   File "/usr/local/lib/python3.8/dist-packages/selenium/webdriver/remote/webdriver.py", line 286, in __init__
2023-02-01 14:20:24     self.start_session(capabilities, browser_profile)
2023-02-01 14:20:24   File "/usr/local/lib/python3.8/dist-packages/selenium/webdriver/remote/webdriver.py", line 378, in start_session
2023-02-01 14:20:24     response = self.execute(Command.NEW_SESSION, parameters)
2023-02-01 14:20:24   File "/usr/local/lib/python3.8/dist-packages/selenium/webdriver/remote/webdriver.py", line 440, in execute
2023-02-01 14:20:24     self.error_handler.check_response(response)
2023-02-01 14:20:24   File "/usr/local/lib/python3.8/dist-packages/selenium/webdriver/remote/errorhandler.py", line 245, in check_response
2023-02-01 14:20:24     raise exception_class(message, screen, stacktrace)
2023-02-01 14:20:24 selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: crashed.
2023-02-01 14:20:24   (unknown error: DevToolsActivePort file doesn't exist)
2023-02-01 14:20:24   (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
2023-02-01 14:20:24 Stacktrace:
2023-02-01 14:20:24 #0 0x00400065d303 <unknown>
2023-02-01 14:20:24 #1 0x004000431d37 <unknown>
2023-02-01 14:20:24 #2 0x00400045a157 <unknown>
2023-02-01 14:20:24 #3 0x004000456330 <unknown>
2023-02-01 14:20:24 #4 0x0040004974a6 <unknown>
2023-02-01 14:20:24 #5 0x00400048e753 <unknown>
2023-02-01 14:20:24 #6 0x004000461a14 <unknown>
2023-02-01 14:20:24 #7 0x004000462b7e <unknown>
2023-02-01 14:20:24 #8 0x0040006ac32e <unknown>
2023-02-01 14:20:24 #9 0x0040006afc0e <unknown>
2023-02-01 14:20:24 #10 0x004000692610 <unknown>
2023-02-01 14:20:24 #11 0x0040006b0c23 <unknown>
2023-02-01 14:20:24 #12 0x004000684545 <unknown>
2023-02-01 14:20:24 #13 0x0040006d16a8 <unknown>
2023-02-01 14:20:24 #14 0x0040006d1836 <unknown>
2023-02-01 14:20:24 #15 0x0040006ecd13 <unknown>
2023-02-01 14:20:24 #16 0x004002600609 start_thread
2023-02-01 14:20:24 

Stacktrace above. Not sure what else I should be doing to get this to work. --quickstart is working as expected, which is promising as it pulled from YNAB as I thought it would. I'm not sure if there's something wrong with how it's pulling from mint?

nathanpaul avatar Feb 01 '23 19:02 nathanpaul

This error usually is because the expected files don't exist. Did you pass a volume mount to the docker image? More specifically, is that mounted directory persisted between runs?

snowskeleton avatar Feb 01 '23 20:02 snowskeleton

docker run -v "$HOME/.ynam_docker:/root/.ynam" -it -d snowskeleton/ynam

Yes it is. I checked before/after the --quickstart happened and it looks like secrets.json was populated as expected. After the first run, I see the selenium files, mint_api_key, and chromebrowser created in my directory as well.

nathanpaul avatar Feb 01 '23 20:02 nathanpaul

Could this possibly be because of a later version of the chromium browser breaking something? Am I able to pin the chromium browser version (by downloading it into my .ynam_docker directory) to one that you're using and that's working?

nathanpaul avatar Feb 01 '23 22:02 nathanpaul

if you're using the docker image, Selenium should take care of all the versioning. You only need to specify versions of you're running with just python.

It's possible that a corrupted cookies file could cause these issues. Try deleting just the cookies and key file to see if that works. edit: also delete the chromedriver. It shouldn't matter, but sometimes it can.

snowskeleton avatar Feb 01 '23 22:02 snowskeleton

to expound a bit more on version issues: if you want to play with pinning, change line 1 of Dockerfile to the desired value. I'm pretty sure the Selenium github has a list of which Selenium version maps to which Chrome(driver) version.

snowskeleton avatar Feb 01 '23 22:02 snowskeleton

Still happening after deleting the cookies, key file, and chromedriver. Curiously, the chromedriver file size is 0/empty, so I'm not sure how that's getting populated after they're all deleted.

nathanpaul avatar Feb 02 '23 17:02 nathanpaul

Some changes finally got pushed through upstream on mintapi/mintapi. If you haven't already found a solution to this, try a clean install and things should work. Changes include a more stable Selenium workflow that (hopefully) causes fewer versioning issues.

snowskeleton avatar Mar 01 '23 18:03 snowskeleton

have same issue macos

chernogorsky avatar Apr 14 '23 15:04 chernogorsky

With a clean install, it still errors out with an identical stack trace. Deleting the chromedriver, cookies, and key files seems to have no effect.

mbd-s avatar Jun 28 '23 06:06 mbd-s

This is most likely an issue with your chrome installation. Try running in Docker or reinstall Chrome/Chromium and see if the issue persists.

snowskeleton avatar Jul 03 '23 17:07 snowskeleton

I was already running in Docker 🥲

mbd-s avatar Jul 03 '23 19:07 mbd-s

Having the same issue in Docker, on MacOS. Let me know if there is anything I can provide to assist.

ryantrip avatar Aug 04 '23 01:08 ryantrip

Same with running in docker on Ubuntu server.

kabobbob avatar Sep 22 '23 13:09 kabobbob