AppDaemon
register_endpoint POST mixed MIME data broken v4.4.0+
What happened?
In this specific control task I've been using this amazing project to handle an incoming POST of mixed MIME data content JSON and Jpeg image faultlessly until the v4.4.0 update. See below for log error:
Version
4.4.2 through HASSIO Add-on 0.12.1 but also earlier 4.4.0
Installation type
Home Assistant add-on
Relevant log output
2023-04-21 12:29:07.333196 ERROR AppDaemon: ------------------------------------------------------------
2023-04-21 12:29:07.334374 ERROR AppDaemon: Unexpected error during API call
2023-04-21 12:29:07.335208 ERROR AppDaemon: ------------------------------------------------------------
2023-04-21 12:29:07.336049 ERROR AppDaemon: Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/appdaemon/http.py", line 869, in call_app_endpoint
ret, code = await self.dispatch_app_endpoint(endpoint, request)
File "/usr/lib/python3.10/site-packages/appdaemon/http.py", line 938, in dispatch_app_endpoint
args = await request.json()
File "/usr/lib/python3.10/site-packages/aiohttp/web_request.py", line 670, in json
body = await self.text()
File "/usr/lib/python3.10/site-packages/aiohttp/web_request.py", line 666, in text
return bytes_body.decode(encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 1450: invalid start byte
2023-04-21 12:29:07.336545 ERROR AppDaemon: ------------------------------------------------------------
2023-04-21 12:29:07.337319 WARNING AppDaemon: API Call to system: status: 500, An Error occured while processing request
Relevant code in the app or config file that caused the issue
def initialize(self):
self.log("Starting image POST endpoint")
self.last_call = None
# create route (endpoint)
self.register_endpoint(self.image_capture, "image_capture")
# async def image_capture(self, request, kwargs):
# def image_capture(self, args, cb_args):
async def image_capture(self, args, cb_args):
try:
now = datetime.now()
if self.last_call is not None:
if self.last_call > now - timedelta(seconds=15):
msg = 'Image capture skipped due to call in last 15s...'
self.log(msg)
response = {"message": msg}
return response, 400
self.last_call = now
isJsonOK = False
isImageOK = False
self.log("Get POST data...")
data = await request.post()
Anything else?
Only that is has been working fine. The POST contains MIME mixed message data, basically JSON and a JPEG image which is then extracted, with the jpg data written to a local file and the JSON used to send a external request. I've left out the other code (as I'm ashamed of it but has worked reliably for months) and the error is definitely occurring (I've checked by adding logging and the error message itself) at the request.post() function call.
I've reference the latest docs and example here and noted the "Breaking changes" in the log file here and hence the modified function definition and commented out part at the top of the code, to try and reflect these breaking changes.
No example is given of using "data = await request.post()", only "data = await request.json()". I'm also unsure where the request object is being referenced from, if it's now not passed in the arguments?
It's quite possibly my understanding of this, especially the async stuff!
If someone can shed any light on this and how it can be diagnosed and resolved, ANY pointers in fact, it would be much appreciated and hopefully for the benefit of all 😄
The problem here is that you were relying on undocumented/unintended functionality that got changed in 4.4.0 Apps were supposed to be for JSON data only, but because of the way @odianosen implemented async app endpoints, there was no JSON checking. I fixed that to make things more consistent in 4.4.0 and it broke your code because you are using JPEGs which are not JSON. I need to think about how we can restore the original behavior in a consistent way.
Hi @acockburn, thanks for your prompt response and I did think that might be the case - I've still a lot to learn! If I can be of any help testing, just let me know and in the meantime, I've rolled back to a pre-4.4.0 version. Look forward to an update, as and when you have time.