ucfopen
Just started throwing InvalidAccessToken exception
My Python scripts that are based on canvasapi have stopped working in the last 10 days. (Definitely worked on 12/19/2022, was not working on 12/27/2022 and hasn't worked since.)
Just in case something changed locally, I uninstalled canvasapi-2.2.0 and installed canvasapi-3.0.0 ... removed some cruft from Python 3.9 and generally scoured my laptop for any other left over stuff.
Since the canvasapi package code had not changed, my best guess is that something canvasapi depends on changed but before I start sifting through the code, I thought I would ask if others have experienced the same problem.
Minimum (non)working example:
#!/usr/bin/python3
from canvasapi import Canvas
URL = 'https://uncc.instructure.com/'
KEY = '999999999999999999999999999999999999999999999999999999999999999999999'
canvas = Canvas(URL,KEY)
course = canvas.get_course(182727)
Results in an exception:
canvasapi.exceptions.InvalidAccessToken: [{'message': 'user authorization required'}]
However, I've checked and double checked access key ... it is the same one that has been working for 2+ years and it has not expired. Furthermore, I issued the same request from the command line:
curl https://uncc.instructure.com/api/v1/courses/182727 -X GET -H 'Authorization: Bearer 999999999999999999999999999999999999999999999999999999999999999999999'
and I get a reasonable JSON response:
{"id":182727,"name":"Logic System Design","account_id":108,"uuid":"7DbizAXeda7de7vOWXO8k1B9nfLKkSsDbmUoGoFO","start_at":null,"grading_standard_id":null,"is_public":false,"created_at":"2022-04-04T10:45:13Z","course_code":"202280-ECGR-2181-001:ECGR-2181-002-XLSEO202280_Combined","default_view":"wiki",...}
I'm hoping I am not doing something embarrassingly stupid ... but I'm stumped. Plus I'm traveling so I can't test on a clean system right now.
Ron
Additional info:
rick (master *)❱ pip show canvasapi Name: canvasapi Version: 3.0.0 Summary: API wrapper for the Canvas LMS Home-page: https://github.com/ucfopen/canvasapi Author: University of Central Florida - Center for Distributed Learning Author-email: [email protected] License: MIT License Location: /home/rsass/.local/lib/python3.10/site-packages Requires: arrow, pytz, requests Required-by:
rick (master *)❱ python3 --version Python 3.10.6
Hey @ron-sass, sorry you're running into issues. As you mentioned, the CanvasAPI code hasn't changed so it must be something upstream.
Canvas did release an API update on the 20th, which aligns with your timeline, but there's nothing in the description that would lead me to expect it to break.
I won't be able to dive deep into this until the new year, but in the meantime you can try a few things:
-
Ensure your URL is properly formatted. CanvasAPI does try to strip trailing slashes but maybe something weird happened, so just in case try
URL = "https://uncc.instructure.com" - Try with a new Access Token. I understand that the one you have is valid, but see if a freshly-created one performs differently.
- To validate the token, instead of grabbing a course, try to use the
get_current_userfunction. This will check that the token is valid and working with CanvasAPI without having to worry about permission issues with the token.user = canvas.get_current_user()
Unfortunately, that's all I can offer for now. If you're still experiencing the problem next week I'd be happy to take a closer look.
Thanks for the super-fast response! I tried your suggestions but no luck; details below:
- URL with or without the trailing slash fail with the same exception
- my first thought was that my token had expired but that was not the case, just to double-check, I did create a new token and got the same results
- to validate the token, I tried the get_current_user() method but that failed too (different exception, see below) ... I also tried from the command line and curl with the access token returned the correct JSON record with my id/information
I do appreciate the fast response and I appreciate your willingness to look into it next week. We are on break as well, so it is not a crisis this week. Have a good holiday!
Ron
Additional information .... when I used the get_current_user() method, it didn't throw the InvalidAccessToken exception. Rather it threw ResourceDoesNotExist("Not Found") exception. I verified with curl that I do exist :-) and I used the web interface to Canvas to double check as well.
@ron-sass canvasapi can be configured to output logs of the network calls it makes which might help troubleshoot this further. I have a feeling the request being prepared and sent through Python is different than the one from curl, despite the tokens being the same.
I just started using the lib, but I have the same problem. I just get
canvasapi.exceptions.InvalidAccessToken: [{'message': 'Invalid access token.'}]
Hey @MarkusvonStaden, sorry to hear you're having trouble with the library. Please test that your access key is valid via an external process, such as the Python requests library, curl, or Postman. Please also take the steps in our Troubleshooting Guide to check your url and access token.
@ron-sass @MarkusvonStaden If the issues persist, please configure logging and send us the result so we can further diagnose the issue.
This code snippet should do the trick, all you need to do is fill in your url and access token:
import logging
import sys
from canvasapi import Canvas
# logging
logger = logging.getLogger("canvasapi")
handler = logging.StreamHandler(sys.stdout)
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
handler.setLevel(logging.DEBUG)
handler.setFormatter(formatter)
logger.addHandler(handler)
logger.setLevel(logging.DEBUG)
API_URL = "" # CHANGE THIS VALUE
API_KEY = "" # CHANGE THIS VALUE
canvas = Canvas(API_URL, API_KEY)
print("BASE URL:")
print(canvas._Canvas__requester.base_url)
me = canvas.get_current_user()
print("Response:")
print(me)
Hi all,
I am not back in the office yet and I haven't looked at the log messages produced yet but just in case this turns out to be wider than just me and @MarkusvonStaden https://github.com/MarkusvonStaden I wanted to post the results ASAP.
I tested the API_KEY with a command-line curl command and it was successful.
curl -H "Authorization: Bearer *********" " https://canvas.instructure.com/api/v1/users/self" {"id":73010000000006290,"name":"Ron Sass","created_at":"2016-03-31T12:21:42-04:00","sortable_name":"Sass, Ron","short_name":"Ron Sass","avatar_url":" https://canvas.instructure.com/images/messages/avatar-50.png ","locale":null,"effective_locale":"en","permissions":{"can_update_name":false,"can_update_avatar":true,"limit_parent_app_web_access":false}}
(I obscured the key obviously.)
There was an early question. I tried both:
API_URL = 'https://uncc.instructure.com/'
and
API_URL = 'https://uncc.instructure.com'
too. Same result as below.
I am hoping to have a couple of hours tomorrow or on the weekend to maybe clone the repo and investigate myself.
Ron
The get_current_user() script generated this output...
BASE URL:
https://uncc.instructure.com/api/v1/
2023-01-05 14:14:16,819 - canvasapi.requester - INFO - Request: GET
https://uncc.instructure.com/api/v1/users/self
2023-01-05 14:14:16,819 - canvasapi.requester - DEBUG - Headers:
{'Authorization': '*'} #<-correct in script
2023-01-05 14:14:17,062 - canvasapi.requester - INFO - Response: GET
https://uncc.instructure.com/api/v1/users/self 404
2023-01-05 14:14:17,062 - canvasapi.requester - DEBUG - Headers: {'Date':
'Thu, 05 Jan 2023 19:14:17 GMT', 'Content-Type': 'application/json;
charset=utf-8', 'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive',
'Server': 'Apache', 'X-Request-Context-Id':
'fa42eaaa-2f35-4253-881b-d4953938e3b9', 'Vary': 'Accept-Encoding',
'Content-Encoding': 'gzip', 'X-Rate-Limit-Remaining': '700.0',
'X-Canvas-Meta':
'q=8644;a=1;g=rvJgYizQAsZ4qM2ktpJ6390RDRByFhnEGRr64JD6;s=7301;c=cluster23;z=us-east-1c;o=users;n=api_show;st=d9113137a2f9436da8f9cd5761ed29a1-f7347fa2e0f2d306-0;b=1671396;m=1671396;u=0.02;y=0.00;d=0.00;',
'Content-Security-Policy': "frame-ancestors 'self' uncc.instructure.com
uncc.staging.instructure.com uncc.beta.instructure.com
uncc.test.instructure.com;", 'X-Request-Cost': '0.018975302008811923',
'Cache-Control': 'no-cache', 'Strict-Transport-Security':
'max-age=31536000', 'Referrer-Policy': 'no-referrer-when-downgrade',
'X-Permitted-Cross-Domain-Policies': 'none', 'X-XSS-Protection': '1;
mode=block', 'X-Download-Options': 'noopen', 'X-Runtime': '0.029114',
'X-Content-Type-Options': 'nosniff', 'Set-Cookie':
'_csrf_token=WOnCLYGC0UCa145zzT48GkM%2Fb%2BvlQEgx8hp0tgLcR7A5k%2FJp9rW%2FOvKZ1kO7WHBUD3AYqpB0MAiFVCbOVpJ3%2Fg%3D%3D;
path=/; secure', 'X-Request-Processor': '0cc83bbaa007538bd', 'X-A11y-Ally':
'Dana Danger Grey', 'Status': '404 Not Found', 'P3P': 'CP="None, see
http://www.instructure.com/privacy-policy"'}
2023-01-05 14:14:17,062 - canvasapi.requester - DEBUG - Data:
'{"errors":[{"message":"The specified resource does not exist."}]}'
Traceback (most recent call last):
File "/home/rsass/work/courses/2022/bin/test-access-token/gen-log.py",
line 27, in
On Wed, Jan 4, 2023 at 10:55 AM Matthew Emond @.***> wrote:
Hey @MarkusvonStaden https://github.com/MarkusvonStaden, sorry to hear you're having trouble with the library. Please test that your access key is valid via an external process, such as the Python requests library, curl, or Postman. Please also take the steps in our Troubleshooting Guide https://canvasapi.readthedocs.io/en/stable/troubleshooting.html to check your url and access token.
@ron-sass https://github.com/ron-sass @MarkusvonStaden https://github.com/MarkusvonStaden If the issues persist, please configure logging https://canvasapi.readthedocs.io/en/stable/debugging.html and send us the result so we can further diagnose the issue.
This code snippet should do the trick, all you need to do is fill in your url and access token:
import loggingimport sys from canvasapi import Canvas
logginglogger = logging.getLogger("canvasapi")handler = logging.StreamHandler(sys.stdout)formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
handler.setLevel(logging.DEBUG)handler.setFormatter(formatter)logger.addHandler(handler)logger.setLevel(logging.DEBUG) API_URL = "" # CHANGE THIS VALUEAPI_KEY = "" # CHANGE THIS VALUE canvas = Canvas(API_URL, API_KEY) print("BASE URL:")print(canvas._Canvas__requester.base_url) me = canvas.get_current_user() print("Response:")print(me)
— Reply to this email directly, view it on GitHub https://github.com/ucfopen/canvasapi/issues/578#issuecomment-1371106325, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJNLM3ZLLCQV4WMRUIQZQJDWQWMQVANCNFSM6AAAAAATLRC7DM . You are receiving this because you were mentioned.Message ID: @.***>
One more piece of information ...
I took a peek at url just before it requests.session.get() is called and it looks correct. So at this point, it would be below canvasapi in the Python requests package.
I am running the latest PopOS (an Ubuntu-like distribution). I checked to make sure I wasn't using a custom version of requests. I'm not. But it is a little bit older than what is on PyPI right now ...
Mine:
rick (master *)❱ pip show requests Name: requests Version: 2.25.1 Summary: Python HTTP for Humans. Home-page: https://requests.readthedocs.io Author: Kenneth Reitz Author-email: [email protected] License: Apache 2.0 Location: /usr/lib/python3/dist-packages Requires: Required-by: canvasapi
From [https://pypi.org/project/requests/] it appears 2.28.1 is the latest and greatest.
When I'm back in my office I can test more easily on a different machine / different distribution.
Ron
I solved my problem, it was just a regular user error. Apperently I am not smart enough to read the dokumentation... I used a developer key instead of the API Key
I have never used sessions in the Python requests package but extracted the parameters that canvasapi are passing to request and distilled everything down to this simple script. Also, there are a couple of bug fixes in requests between requests 2.25 and 2.28.1 so I went and cloned the development version of request "just to be sure".
Here's my test:
#!/usr/bin/python3 import requests import os
url = 'https://uncc.instructure.com/api/v1/users/self' h = {'Authorization': 'Bearer *********'}
print('********** Using Python "requests" package') s = requests.Session() resp = s.get(url, headers=h) print(resp.text)
print('\n********** Using os.system to call curl') cmd = f'curl -H "Authorization: {h["Authorization"]}" "{url}"' rc = os.system(cmd) print() print()
And the output is the same:
rick (master *)❱ python3 simple-test.py ********** Using Python "requests" package {"errors":[{"message":"The specified resource does not exist."}]}
********** Using os.system to call curl {"id":6290,"name":"Ron Sass","created_at":"2016-03-31T12:21:42-04:00","sortable_name":"Sass, Ron","short_name":"Ron Sass","avatar_url":" https://uncc.instructure.com/images/messages/avatar-50.png ","locale":null,"effective_locale":"en","permissions":{"can_update_name":false,"can_update_avatar":true,"limit_parent_app_web_access":false}}
I have never worked with the requests packages so if my "resp = s.get(url, headers=h)" (modeled after what canvasapi does) is in fact correct, then it looks like the bug is in the requests package.
Before I go, one quick question: Is anyone else having this issue?
I'm using PopOS, but it is pretty generic, off-the-shelf install:
rick (master *)❱ lsb_release -a No LSB modules are available. Distributor ID:Pop Description: Pop!_OS 22.04 LTS Release: 22.04 Codename: jammy
Ron
On Thu, Jan 5, 2023 at 2:33 PM Ron Sass @.***> wrote:
Hi all,
I am not back in the office yet and I haven't looked at the log messages produced yet but just in case this turns out to be wider than just me and @MarkusvonStaden https://github.com/MarkusvonStaden I wanted to post the results ASAP.
I tested the API_KEY with a command-line curl command and it was successful.
curl -H "Authorization: Bearer *********" " https://canvas.instructure.com/api/v1/users/self" {"id":73010000000006290,"name":"Ron Sass","created_at":"2016-03-31T12:21:42-04:00","sortable_name":"Sass, Ron","short_name":"Ron Sass","avatar_url":" https://canvas.instructure.com/images/messages/avatar-50.png ","locale":null,"effective_locale":"en","permissions":{"can_update_name":false,"can_update_avatar":true,"limit_parent_app_web_access":false}}
(I obscured the key obviously.)
There was an early question. I tried both:
API_URL = 'https://uncc.instructure.com/'
and
API_URL = 'https://uncc.instructure.com'
too. Same result as below.
I am hoping to have a couple of hours tomorrow or on the weekend to maybe clone the repo and investigate myself.
Ron
The get_current_user() script generated this output...
BASE URL: https://uncc.instructure.com/api/v1/ 2023-01-05 14:14:16,819 - canvasapi.requester - INFO - Request: GET https://uncc.instructure.com/api/v1/users/self 2023-01-05 14:14:16,819 - canvasapi.requester - DEBUG - Headers: {'Authorization': '*'} #<-correct in script 2023-01-05 14:14:17,062 - canvasapi.requester - INFO - Response: GET https://uncc.instructure.com/api/v1/users/self 404 2023-01-05 14:14:17,062 - canvasapi.requester - DEBUG - Headers: {'Date': 'Thu, 05 Jan 2023 19:14:17 GMT', 'Content-Type': 'application/json; charset=utf-8', 'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', 'Server': 'Apache', 'X-Request-Context-Id': 'fa42eaaa-2f35-4253-881b-d4953938e3b9', 'Vary': 'Accept-Encoding', 'Content-Encoding': 'gzip', 'X-Rate-Limit-Remaining': '700.0', 'X-Canvas-Meta': 'q=8644;a=1;g=rvJgYizQAsZ4qM2ktpJ6390RDRByFhnEGRr64JD6;s=7301;c=cluster23;z=us-east-1c;o=users;n=api_show;st=d9113137a2f9436da8f9cd5761ed29a1-f7347fa2e0f2d306-0;b=1671396;m=1671396;u=0.02;y=0.00;d=0.00;', 'Content-Security-Policy': "frame-ancestors 'self' uncc.instructure.com uncc.staging.instructure.com uncc.beta.instructure.com uncc.test.instructure.com;", 'X-Request-Cost': '0.018975302008811923', 'Cache-Control': 'no-cache', 'Strict-Transport-Security': 'max-age=31536000', 'Referrer-Policy': 'no-referrer-when-downgrade', 'X-Permitted-Cross-Domain-Policies': 'none', 'X-XSS-Protection': '1; mode=block', 'X-Download-Options': 'noopen', 'X-Runtime': '0.029114', 'X-Content-Type-Options': 'nosniff', 'Set-Cookie': '_csrf_token=WOnCLYGC0UCa145zzT48GkM%2Fb%2BvlQEgx8hp0tgLcR7A5k%2FJp9rW%2FOvKZ1kO7WHBUD3AYqpB0MAiFVCbOVpJ3%2Fg%3D%3D; path=/; secure', 'X-Request-Processor': '0cc83bbaa007538bd', 'X-A11y-Ally': 'Dana Danger Grey', 'Status': '404 Not Found', 'P3P': 'CP="None, see http://www.instructure.com/privacy-policy"'} 2023-01-05 14:14:17,062 - canvasapi.requester - DEBUG - Data: '{"errors":[{"message":"The specified resource does not exist."}]}' Traceback (most recent call last): File "/home/rsass/work/courses/2022/bin/test-access-token/gen-log.py", line 27, in
me = canvas.get_current_user() File "/home/rsass/.local/lib/python3.10/site-packages/canvasapi/canvas.py", line 773, in get_current_user return CurrentUser(self.__requester) File "/home/rsass/.local/lib/python3.10/site-packages/canvasapi/current_user.py", line 14, in init response = self._requester.request("GET", "users/self") File "/home/rsass/.local/lib/python3.10/site-packages/canvasapi/requester.py", line 255, in request raise ResourceDoesNotExist("Not Found") canvasapi.exceptions.ResourceDoesNotExist: Not Found On Wed, Jan 4, 2023 at 10:55 AM Matthew Emond @.***> wrote:
Hey @MarkusvonStaden https://github.com/MarkusvonStaden, sorry to hear you're having trouble with the library. Please test that your access key is valid via an external process, such as the Python requests library, curl, or Postman. Please also take the steps in our Troubleshooting Guide https://canvasapi.readthedocs.io/en/stable/troubleshooting.html to check your url and access token.
@ron-sass https://github.com/ron-sass @MarkusvonStaden https://github.com/MarkusvonStaden If the issues persist, please configure logging https://canvasapi.readthedocs.io/en/stable/debugging.html and send us the result so we can further diagnose the issue.
This code snippet should do the trick, all you need to do is fill in your url and access token:
import loggingimport sys from canvasapi import Canvas
logginglogger = logging.getLogger("canvasapi")handler = logging.StreamHandler(sys.stdout)formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
handler.setLevel(logging.DEBUG)handler.setFormatter(formatter)logger.addHandler(handler)logger.setLevel(logging.DEBUG) API_URL = "" # CHANGE THIS VALUEAPI_KEY = "" # CHANGE THIS VALUE canvas = Canvas(API_URL, API_KEY) print("BASE URL:")print(canvas._Canvas__requester.base_url) me = canvas.get_current_user() print("Response:")print(me)
— Reply to this email directly, view it on GitHub https://github.com/ucfopen/canvasapi/issues/578#issuecomment-1371106325, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJNLM3ZLLCQV4WMRUIQZQJDWQWMQVANCNFSM6AAAAAATLRC7DM . You are receiving this because you were mentioned.Message ID: @.***>
I tracked down the problem! It was a local issue but for completeness, I'll describe what happened.
TL;DR : Python requests module always reads $(HOME)/.netrc file and it overrides authorization passed via a headers dictionary (which is what canvasapi passes to requests); simple solution is to avoid using/remove .netrc file
For those that are interested, I had to dig into the requests' session.py code and in the prepare_request method, it checks if auth was set when the session object is created. If it has not been set, then it tries the .netrc file and if finds anything then that overwrites the authorization key/value passed in the headers dictionary with the session.get method.
One solution might be for canvasapi to set the authorization (bearer access-token) when session is instantiated (and not pass it in the headers to the sessions.get call) . But, in my case, I only needed the .netrc for a while because I was developing an embedded system solution and was repeatedly ftp'ing an image. The .netrc file just saved me a lot of typing but I don't need it anymore. So just removing the .netrc solved my immediate problem, my canvas scripts are working, and I am ready to start classes on Monday!