Cannot call service that returns a response

[jpearce73]

What happened?

Cannot call a service from AppDaemon when response data is wanted. Service call fails when either method is used. self.temp_forecast_entity = self.get_entity('weather.ksjc_daynight') response=self.temp_forecast_entity.call_service(service='get_forecasts', type='hourly', return_response=True) or response=self.call_service('weather/get_forecasts', entity_id = 'weather.forecast_home', type = 'hourly', return_response=True)

Full write up has been posted on the forum https://community.home-assistant.io/t/cant-call-service-that-returns-response/730859/7

Version

0.16.4

Installation type

Home Assistant add-on

Relevant log output

Appdaemon logs this
2024-05-20 16:37:10.625349 WARNING HASS: Code: 500, error: 500 Internal Server Error Server got itself in trouble
2024-05-20 16:37:10.622865 WARNING HASS: Error calling Home Assistant service default/weather/get_forecasts

homeassistant.log logs this
2024-05-20 16:37:10.605 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/aiohttp/web_protocol.py", line 452, in _handle_request
    resp = await request_handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/web_app.py", line 543, in _handle
    resp = await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/web_middlewares.py", line 114, in impl
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 92, in security_filter_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 83, in forwarded_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 26, in request_context_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 88, in ban_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp_session/__init__.py", line 199, in factory
    response = await handler(request)
               ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 295, in auth_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/headers.py", line 32, in headers_middleware
    response = await handler(request)
               ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/http.py", line 73, in handle
    result = await handler(request, **request.match_info)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/api/__init__.py", line 406, in post
    await shield(
  File "/usr/src/homeassistant/homeassistant/core.py", line 2692, in async_call
    raise ServiceValidationError(
homeassistant.exceptions.ServiceValidationError: The service call requires responses and must be called with return_response=True

Relevant code in the app or config file that caused the issue

Service call fails when either method is used.
self.temp_forecast_entity = self.get_entity('weather.ksjc_daynight') response=self.temp_forecast_entity.call_service(service='get_forecasts', type='hourly', return_response=True)
or
response=self.call_service('weather/get_forecasts', entity_id = 'weather.forecast_home', type = 'hourly', return_response=True)

Anything else?

I've tried two different service calls on different integrations I've also confirmed in the HA developer pane, the services work and return data. My suspicion is that the "return_response=True" portion of the service call is getting lost though the API.

[acockburn]

We now have this working in the dev branch

[dekiesel]

Is a release planned for the "near future" or should we use the dev branch in the meantime? Thank you for your work either way!

[acockburn]

It will be a few weeks yet but not a lot longer, use the dev branch if you really need it and are feeling brave! The changes in the current dev work well for returning results and the docs are all there but we are planning a rewrite of parts of it and in particular startup conditions are broken.

[Bjarne-Kinkel]

@acockburn Any news on when the new version will be available?

[acockburn]

Sorry for the delay - I got busy in real life - I hope to have some time to finalize this over the Holidays.

[LogicAnalysis]

I just ran into this issue myself. It's odd that the documentation explicitly mentions the ability to do this, and yet it does not actually work. Here's the example shown in the documentation: self.call_service("calendar/get_events", entity_id="calendar.home", start_date_time="2024-08-25 00:00:00", end_date_time="2024-08-27 00:00:00", return_result=True, hass_result=True, hass_timeout=10), and this is what I've tried:

print("ATTEMPT 1")
travel_events = self.call_service(
    "calendar/get_events",
    entity_id="calendar.travel",
    start_date_time=now.strftime("%Y-%m-%dT%H:%M:%S"),
    end_date_time=end_of_day.strftime("%Y-%m-%dT%H:%M:%S"),
    return_result=True,
    hass_result=True,
    hass_timeout=10
)
print(travel_events)

print("ATTEMPT 2")
calendar_entity = self.get_entity("calendar.travel")
travel_events = calendar_entity.call_service(
    service="get_events",
    start_date_time=now.strftime("%Y-%m-%dT%H:%M:%S"),
    end_date_time=end_of_day.strftime("%Y-%m-%dT%H:%M:%S"),
    return_result=True,
    hass_result=True,
    hass_timeout=10
)
print(travel_events)

Both of these result in the same error:

ATTEMPT 1
2024-12-22 09:17:42.933738 WARNING HASS: Error calling Home Assistant service default/calendar/get_events
2024-12-22 09:17:42.934223 WARNING HASS: Code: 400, error: {"message":"Service call requires responses but caller did not ask for responses. Add ?return_response to query parameters."}
None
ATTEMPT 2
2024-12-22 09:17:42.945895 WARNING HASS: Error calling Home Assistant service default/calendar/get_events
2024-12-22 09:17:42.946336 WARNING HASS: Code: 400, error: {"message":"Service call requires responses but caller did not ask for responses. Add ?return_response to query parameters."}
None

@acockburn I'm really excited to see this update in action. Thank you!

[acockburn]

AppDaemon's capability to return a result from a service was added before HASS had it, but it was internal to AD. That's why it's mentioned in the docs.

[gpbenton]

I tried the dev docker image, and it doesn't error out, but it also doesn't return anything, although its the first time I a have tried this, so I may not be using it correctly.

`

def initialize(self) -> None:
    self.log("__function__", level="DEBUG")
    try:
        self.list = self.get_entity("todo.chores")
    except KeyError as e:
        self.log(f" not defined in arguments: {e=}", level="ERROR")
        return

    if not self.list.exists():
        self.log(f"{self.location} does not exist", level="ERROR")
        return

    list_items = self.list.call_service("get_items", return_response=True) 

    self.log(f"{list_items=}", level="INFO")

`

yields

`

2024-12-31 12:51:28.815579 INFO AppDaemon: Calling initialize() for vacuum_lounge_floor 2024-12-31 12:51:28.820681 INFO vacuum_lounge_floor: list_items=None

`

I have tried the action in the developer tools, and it is returning the item in the list.

Edit:
I think I should have used return_result=True
But that still gives the error using the docker image

`

2024-12-31 13:00:24.835039 INFO AppDaemon: Calling initialize() for vacuum_lounge_floor 2024-12-31 13:00:24.840850 WARNING HASS: Error in HASS response: {'id': 14, 'type': 'result', 'success': False, 'error': {'code': 'service_validation_error', 'message': 'Validation error: The action requires responses and must be called with return_response=True', 'translation_key': 'service_lacks_response_request', 'translation_placeholders': {'return_response': 'return_response=True'}, 'translation_domain': 'homeassistant'}, 'ad_status': 'OK', 'ad_duration': 0.0008709430694580078} 2024-12-31 13:00:24.841868 INFO vacuum_lounge_floor: list_items={'id': 14, 'type': 'result', 'success': False, 'error': {'code': 'service_validation_error', 'message': 'Validation error: The action requires responses and must be called with return_response=True', 'translation_key': 'service_lacks_response_request', 'translation_placeholders': {'return_response': 'return_response=True'}, 'translation_domain': 'homeassistant'}, 'ad_status': 'OK', 'ad_duration': 0.0008709430694580078}

`

[acockburn]

You need to specify an extra flag, it’s in the dev docs (on my phone so can’T tell you more!)

Message ID: @.***>

[gpbenton]

Got it - hass_result=True

Works fine now.

Thanks.

[gpbenton]

I think the documentation for the entity.call_service api needs to be updated to match (or refer to the appdaemon api .

It all seems to work, but I think it was the reason I missed the new parameters yesterday.

[kbahey]

Any plans for a stable release soon? That would include this fix which is required for the current versions of Home Assistant.

[acockburn]

Yes, we are actively working on version 4.5 which will have this capability in it. It's a large release as we are also including a rewrite of the app loading architecture and pedantic support. I can't give an exact timeline as we all have day jobs, and this is a lot of work, but I would expect weeks rather than months.

[kbahey]

Thank you so much for the reply. And for appdaemon itself. It is just a brilliant piece of software ...

Weeks is perfectly fine ...

[jsl12]

Some weeks turned into some months, but this should all work now. The mechanics behind the Hass plugin have been extensively reworked to support returning values from calls to the websocket, including service calls.

The changes are now live in the dev branch, and @acockburn and I have each been running our personal systems on it for a while now.

The docs will be updated soon, and sometime soon after that, we'll get a release out