[FEATURE] setting the schedule for heat & water

[aashram]

Is your feature request related to a problem? Please describe. We change the 7 day schedule depending on a few criteria. Season/weather and sometimes the thermostat being tampered with too. We want to programme the schedule via the API

Describe the solution you'd like get and set methods for the schedule

Additional context Is there any documentation ONLINE for the HIVE REST API ?

[KJonline]

Unfortunately no there is no documentation online for hives api. We identify calls by running tracing in the browser.

[aashram]

from what I can trace (I am new to this-sorry) the payload when editing the schedule is sending the complete schedule back with some additional json added to the updated slots. in the below case I updated the monday 1170 event index 3

  {
    "value": {
      "target": 17
    },
    "start": 1170,
    "type": "HEATING",
    "dayIndex": 0,
    "eventIndex": 3
  }

complete payload

{"schedule":{"monday":[{"value":{"target":21},"start":300},{"value":{"target":17},"start":540},{"value":{"target":21},"start":1050},{"value":{"target":17},"start":1170,"type":"HEATING","dayIndex":0,"eventIndex":3},{"value":{"target":21},"start":1245},{"value":{"target":17},"start":1425}],"tuesday":[{"value":{"target":21},"start":300},{"value":{"target":17},"start":540},{"value":{"target":21},"start":1050},{"value":{"target":17},"start":1170},{"value":{"target":21},"start":1245},{"value":{"target":17},"start":1425}],"wednesday":[{"value":{"target":21},"start":300},{"value":{"target":17},"start":540},{"value":{"target":21},"start":1050},{"value":{"target":17},"start":1170},{"value":{"target":21},"start":1245},{"value":{"target":17},"start":1425}],"thursday":[{"value":{"target":21},"start":300},{"value":{"target":17},"start":540},{"value":{"target":21},"start":1050},{"value":{"target":17},"start":1170},{"value":{"target":21},"start":1245},{"value":{"target":17},"start":1425}],"friday":[{"value":{"target":21},"start":300},{"value":{"target":17},"start":540},{"value":{"target":21},"start":1050},{"value":{"target":17},"start":1170},{"value":{"target":21},"start":1245},{"value":{"target":17},"start":1425}],"saturday":[{"value":{"target":21},"start":300},{"value":{"target":17},"start":540},{"value":{"target":21},"start":1050},{"value":{"target":17},"start":1170},{"value":{"target":21},"start":1245},{"value":{"target":17},"start":1425}],"sunday":[{"value":{"target":21},"start":300},{"value":{"target":17},"start":540},{"value":{"target":21},"start":1050},{"value":{"target":17},"start":1170},{"value":{"target":21},"start":1245},{"value":{"target":17},"start":1425}]}}

[KJonline]

@aashram Do you use home assistant or are looking at using this library on it own? Did it send the whole data back in the post API call when it was updating the schedule?

[aashram]

I have a python application that uses the library directly. I have not tried anything with schedule as yet. I just did a trace on google chrome of what happens when doing schedule edits on the hive website

[KJonline]

How would you see this working you pass a day and time update the schedule dictionary and pass it all back to hive?

[aashram]

Maybe the most straightforward is iterate though dictionary (each event index and day index) and compare it to a pre-defined template. Anything thats different send a update for that dayIndex and eventindex position

[aashram]

@KJonline is there anyway to send out this json using the present API version to test that it updates the schedule ? Would I need to do the Request or setState method call ? I have a python app working which sets mode to schedule and monitors the boost settings. I wanted to add an additional feature to re-set the schedule if it was changed.

"{"schedule":{"monday":[{"value":{"target":19},"start":450},{"value":{"target":17},"start":555},{"value":{"target":17},"start":810},{"value":{"target":19},"start":1080},{"value":{"target":18.5},"start":1320,"type":"HEATING","dayIndex":0,"eventIndex":4},{"value":{"target":17},"start":1425}],"tuesday":[{"value":{"target":19},"start":450},{"value":{"target":17},"start":555},{"value":{"target":17},"start":810},{"value":{"target":19},"start":1080},{"value":{"target":18.5},"start":1320},{"value":{"target":17},"start":1425}],"wednesday":[{"value":{"target":19},"start":450},{"value":{"target":17},"start":555},{"value":{"target":17},"start":810},{"value":{"target":19},"start":1080},{"value":{"target":18.5},"start":1320},{"value":{"target":17},"start":1425}],"thursday":[{"value":{"target":19},"start":450},{"value":{"target":17},"start":555},{"value":{"target":17},"start":810},{"value":{"target":19},"start":1080},{"value":{"target":18.5},"start":1320},{"value":{"target":17},"start":1425}],"friday":[{"value":{"target":19},"start":450},{"value":{"target":17},"start":555},{"value":{"target":17},"start":810},{"value":{"target":19},"start":1080},{"value":{"target":18.5},"start":1320},{"value":{"target":17},"start":1425}],"saturday":[{"value":{"target":19},"start":450},{"value":{"target":17},"start":555},{"value":{"target":17},"start":810},{"value":{"target":19},"start":1080},{"value":{"target":18.5},"start":1320},{"value":{"target":17},"start":1425}],"sunday":[{"value":{"target":19},"start":450},{"value":{"target":17},"start":555},{"value":{"target":17},"start":810},{"value":{"target":19},"start":1080},{"value":{"target":18.5},"start":1320},{"value":{"target":17},"start":1425}]}}"

[martintoreilly]

@aashram I can confirm that posting a JSON payload with that structure to https://beekeeper-uk.hivehome.com/1.0/nodes/heating/<heating-id> results in my schedule updating. I think the update model is to pass the full schedule structure for the week each time a change is made. This is what I see in the request to the beekeeper API made by the Hive website when I update the schedule. I also see the full week's schedule sent over my home zigbee network when I update a single element via the Hive android app (one message per day, but always 7 messages for the full week).

[martintoreilly]

@KJonline I'm happy to make a PR to add a heating.setSchedule() function, wrapping a call to api.setState() following the model of heating.setTargetTemperature().

Edit: I can also add a heating.getSchedule() convenience function as requested by @aashram that either just grabs the schedule from the already retrieved heating product or refreshes the schedule by calling getProducts() first.

[aashram]

That would be excellent thank you @martintoreilly

[martintoreilly]

Note to self: If the payload isn't well-formed JSON, the beekeeper endpoint returns a "400 - Bad request" status and a JSON { "Could not process payload" } message. If the payload is well-formed JSON but not a valid property for the heating product then the beekeeper endpoint returns a "200 - OK" status and the body contains the same JSON as the request. Therefore, it would probably be a good idea to verify that the update has taken place by retrieving the schedule after the update and checking that it matches the schedule we just attempted to set, throwing an error if not.

[aashram]

@martintoreilly This is the error I was getting, glad you found out why that was happening. Could you please post an example message which gives a successful update ?

[martintoreilly]

@aashram The following works for me.

{"schedule":{"monday":[{"value":{"target":21},"start":420},{"value":{"target":7},"start":540},{"value":{"target":7},"start":720},{"value":{"target":7},"start":840},{"value":{"target":20},"start":1200},{"value":{"target":7},"start":1320}],"tuesday":[{"value":{"target":20},"start":420},{"value":{"target":7},"start":540},{"value":{"target":7},"start":720},{"value":{"target":7},"start":840},{"value":{"target":20},"start":1200},{"value":{"target":7},"start":1320}],"wednesday":[{"value":{"target":20},"start":420},{"value":{"target":7},"start":540},{"value":{"target":7},"start":720},{"value":{"target":7},"start":840},{"value":{"target":20},"start":1200},{"value":{"target":7},"start":1320}],"thursday":[{"value":{"target":20},"start":420},{"value":{"target":7},"start":540},{"value":{"target":7},"start":720},{"value":{"target":7},"start":840},{"value":{"target":20},"start":1200},{"value":{"target":7},"start":1320}],"friday":[{"value":{"target":20},"start":420},{"value":{"target":7},"start":540},{"value":{"target":7},"start":720},{"value":{"target":7},"start":840},{"value":{"target":20},"start":1200},{"value":{"target":7},"start":1320}],"saturday":[{"value":{"target":20},"start":540},{"value":{"target":7},"start":660},{"value":{"target":7},"start":720},{"value":{"target":7},"start":840},{"value":{"target":20},"start":1200},{"value":{"target":7},"start":1320}],"sunday":[{"value":{"target":20},"start":540},{"value":{"target":7},"start":660},{"value":{"target":7},"start":720},{"value":{"target":7},"start":840},{"value":{"target":20},"start":1200},{"value":{"target":7},"start":1320}]}}

[martintoreilly]

@aashram Actually, I've just tested with the JSON you posted and it works for me (after removing the outer double quotes).

[aashram]

@martintoreilly I have been trying to use session.api.setState to send the JSON to test if it works but not had much success.

schedule_payload_json = r'{"monday":[{"value":{"target":19},"start":450},{"value":{"target":17},"start":555},{"value":{"target":17},"start":810},{"value":{"target":19},"start":1080},{"value":{"target":18.5},"start":1320,"type":"HEATING","dayIndex":0,"eventIndex":4},{"value":{"target":17},"start":1425}],"tuesday":[{"value":{"target":19},"start":450},{"value":{"target":17},"start":555},{"value":{"target":17},"start":810},{"value":{"target":19},"start":1080},{"value":{"target":18.5},"start":1320},{"value":{"target":17},"start":1425}],"wednesday":[{"value":{"target":19},"start":450},{"value":{"target":17},"start":555},{"value":{"target":17},"start":810},{"value":{"target":19},"start":1080},{"value":{"target":18.5},"start":1320},{"value":{"target":17},"start":1425}],"thursday":[{"value":{"target":19},"start":450},{"value":{"target":17},"start":555},{"value":{"target":17},"start":810},{"value":{"target":19},"start":1080},{"value":{"target":18.5},"start":1320},{"value":{"target":17},"start":1425}],"friday":[{"value":{"target":19},"start":450},{"value":{"target":17},"start":555},{"value":{"target":17},"start":810},{"value":{"target":19},"start":1080},{"value":{"target":18.5},"start":1320},{"value":{"target":17},"start":1425}],"saturday":[{"value":{"target":19},"start":450},{"value":{"target":17},"start":555},{"value":{"target":17},"start":810},{"value":{"target":19},"start":1080},{"value":{"target":18.5},"start":1320},{"value":{"target":17},"start":1425}],"sunday":[{"value":{"target":19},"start":450},{"value":{"target":17},"start":555},{"value":{"target":17},"start":810},{"value":{"target":19},"start":1080},{"value":{"target":18.5},"start":1320},{"value":{"target":17},"start":1425}]}'

data = session.data.products[HeatingZone_1["hiveID"]] resp = session.api.setState( data["type"], HeatingZone_1["hiveID"], schedule=schedule_payload_json )

[martintoreilly]

~@aashram I'm having a bit of trouble figuring out the best way to select a product/device from the fields of the session object. How did you access / generate the HeatingZone_1 object?~

Edit: Nevermind. I'm getting the thermostat deviceList entry via thermostat = session.deviceList["climate"][0]

[martintoreilly]

@aashram Playing round with the code that constructs the JSON in api.setState(), your schedule_payload_json string is wrapped in double quotes within the JSON payload string constructed in api.setState().

I've tried passing the schedule in as a python dictionary and it escapes all the single quotes in the schedule dictionary keys when it constructs the full payload string.

@KJonline Do you have a trick to pass properties that are dictionaries into api.setState() and get valid JSON the beekeeper API will accept with the current jsc construction code? If not, I've played around a bit and I think that importing the json library and calling json.dumps(kwargs) should construct valid/acceptable JSON with both single valued and dictionary property values.

[martintoreilly]

@aashram I have schedule setting working using json.dumps(). I've started a work-in-progress branch for adding support for getting and setting schedules, with just this change in it so far. I've tested it with the following code (insert your own credentials). Your schedule is schedule_1 and mine is schedule_2.

from pyhiveapi import Hive, SMS_REQUIRED

session = Hive(username="<username>", password="<password>")
login = session.login()

if login.get("ChallengeName") == SMS_REQUIRED:
    code = input("Enter 2FA code: ")
    session.sms2FA(code, login)

session.startSession()
thermostat = session.deviceList["climate"][0]
schedule_1 = {"monday":[{"value":{"target":19},"start":450},{"value":{"target":17},"start":555},{"value":{"target":17},"start":810},{"value":{"target":19},"start":1080},{"value":{"target":18.5},"start":1320,"type":"HEATING","dayIndex":0,"eventIndex":4},{"value":{"target":17},"start":1425}],"tuesday":[{"value":{"target":19},"start":450},{"value":{"target":17},"start":555},{"value":{"target":17},"start":810},{"value":{"target":19},"start":1080},{"value":{"target":18.5},"start":1320},{"value":{"target":17},"start":1425}],"wednesday":[{"value":{"target":19},"start":450},{"value":{"target":17},"start":555},{"value":{"target":17},"start":810},{"value":{"target":19},"start":1080},{"value":{"target":18.5},"start":1320},{"value":{"target":17},"start":1425}],"thursday":[{"value":{"target":19},"start":450},{"value":{"target":17},"start":555},{"value":{"target":17},"start":810},{"value":{"target":19},"start":1080},{"value":{"target":18.5},"start":1320},{"value":{"target":17},"start":1425}],"friday":[{"value":{"target":19},"start":450},{"value":{"target":17},"start":555},{"value":{"target":17},"start":810},{"value":{"target":19},"start":1080},{"value":{"target":18.5},"start":1320},{"value":{"target":17},"start":1425}],"saturday":[{"value":{"target":19},"start":450},{"value":{"target":17},"start":555},{"value":{"target":17},"start":810},{"value":{"target":19},"start":1080},{"value":{"target":18.5},"start":1320},{"value":{"target":17},"start":1425}],"sunday":[{"value":{"target":19},"start":450},{"value":{"target":17},"start":555},{"value":{"target":17},"start":810},{"value":{"target":19},"start":1080},{"value":{"target":18.5},"start":1320},{"value":{"target":17},"start":1425}]}
schedule_2 = {"monday":[{"value":{"target":21},"start":420},{"value":{"target":7},"start":540},{"value":{"target":7},"start":720},{"value":{"target":7},"start":840},{"value":{"target":20},"start":1200},{"value":{"target":7},"start":1320}],"tuesday":[{"value":{"target":20},"start":420},{"value":{"target":7},"start":540},{"value":{"target":7},"start":720},{"value":{"target":7},"start":840},{"value":{"target":20},"start":1200},{"value":{"target":7},"start":1320}],"wednesday":[{"value":{"target":20},"start":420},{"value":{"target":7},"start":540},{"value":{"target":7},"start":720},{"value":{"target":7},"start":840},{"value":{"target":20},"start":1200},{"value":{"target":7},"start":1320}],"thursday":[{"value":{"target":20},"start":420},{"value":{"target":7},"start":540},{"value":{"target":7},"start":720},{"value":{"target":7},"start":840},{"value":{"target":20},"start":1200},{"value":{"target":7},"start":1320}],"friday":[{"value":{"target":20},"start":420},{"value":{"target":7},"start":540},{"value":{"target":7},"start":720},{"value":{"target":7},"start":840},{"value":{"target":20},"start":1200},{"value":{"target":7},"start":1320}],"saturday":[{"value":{"target":20},"start":540},{"value":{"target":7},"start":660},{"value":{"target":7},"start":720},{"value":{"target":7},"start":840},{"value":{"target":20},"start":1200},{"value":{"target":7},"start":1320}],"sunday":[{"value":{"target":20},"start":540},{"value":{"target":7},"start":660},{"value":{"target":7},"start":720},{"value":{"target":7},"start":840},{"value":{"target":20},"start":1200},{"value":{"target":7},"start":1320}]}

schedule_new = schedule_2
session.api.setState(thermostat["hiveType"],thermostat["hiveID"], schedule = schedule_new)

[martintoreilly]

@KJonline There are a few other places where you are constructing JSON strings by hand (see below). Are you ok with me converting them all to use json.dumps()? We might want to even consider replacing the data parameter in the calls to the underlying requests.request() with json and passing the dictionaries in directly without converting them to strings (supported since v2.4.2 of requests).

• refreshTokens(): hive_async_api.py | hive_api.py
• setAlarm(): hive_async_api.py (not present in hive_api.py)

Edit: Is the lack of setAlarm() in the synchronous hive_api.py an oversight or does that endpoint only work with aynschronous calls?

[martintoreilly]

@KJonline So far I've only made this change to the synchronous setState() function in hive_api.py [Edit: I've since made the corresponding edit to the asynchronous setState() function, but my following question still holds]. However, in setup.py it looks like you are using the unasync library to automatically generate the synchronous version of the pyhiveapi code from the asynchronous version of the code, so I'm a little confused why my changes to the synchronous hive_api.py setState() function didn't just get overwritten by the code autogenerated from the asynchronous version of setState() in hive_async_api.py.

Also, I've not quite got my head round the test suite yet. I can only see a list of test IDs in bandit.yaml looking at the bandit docs am I right in my understanding that none of these tests are specific to the pyhiveapi code? Am I missing some library-specific tests somewhere I should be running to check I've not broken anything and adding to when I add the getSchedule() and setSchedule() functions?

[martintoreilly]

Notes to self:

• When I accidentally nested the schedule dictionary as an element in a second parent schedule dictionary item (i.e. schedule = { "schedule": {"monday":...) and passed the json.dumps(schedule_dictionary) as the request data, I got a 500 - Internal server error with a response payload of { "error": "UNEXPECTED_ERROR" }
• If I pass the schedule to setState() as a dictionary when using the hand rolled JSON string construction code (i.e. schedule = {"monday:...), I get a 500 - Internal server errorwith a response payload of{ "error": "UNEXPECTED_ERROR" }. The JSON string has double quotes around the schedule value and escapes the single quotes round the keys (i.e. {"schedule": "{'monday': [{'value': ...`).
• If I pass the schedule to setState() as a string (i.e. schedule = '{"monday":...), I get a 400 - Bad request with a response payload of { "message": "Could not process payload" }. The JSON string has double quotes around the schedule value (i.e. {"schedule": "{"monday":[{"value":...).
• If I pass the schedule JSON as a string to setState() with json.dumps(schedule_json_string) as the request data, I get a 400 - Bad request with a response payload of { "message": "Could not process payload" }. The JSON produced by json.dumps() escapes all the quotes in the schedule_json_string by prefixing them with \\ (i.e. {"schedule": "{\\"monday\\":[{\\"value\\":).

[martintoreilly]

I'm also happy to extend the Hive Home Assistant Integration to expose this new functionality, though I'll need some time to get my head into the codebase.

[martintoreilly]

@aashram Do you have multiple heating zones and/or radiator TRVs? I only have one heating zone with a single central boiler module so can't query the beekeeper API for these.

[aashram]

@martintoreilly we only one heating zone and no TRV. We also have Hot Water too on schedule.

[martintoreilly]

@aashram I have hot water support on the central boiler module too, so will add hot water schedule support too. I may do this in PR #36 or may do it as a separate PR depending on the complexity. Not all installations have hot water support and I want to make sure this functionality behaves correctly in this case too.

@KJonline @Rendili Do either of you have any of the following in your Hive setups?

• Multiple heating zones?
• Individual radiator TRVs?
• Heating-only central boiler unit (i.e. no hot water control)?

[aashram]

@martintoreilly happy to help with any testing if required 👍

[martintoreilly]

@KJonline I'm going to add tests for this change. I plan to use pytest and ~mock-requests~requests-mock. Let me know if you'd prefer we use different modules.

[martintoreilly]

@KJonline I've tried running the current tests using pytest . from the root folder of the repo, but get the following error.

(.venv) hermes:Pyhiveapi martin$ pytest .
====================================== test session starts ======================================
platform darwin -- Python 3.9.9, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /Users/martin/Source/Personal/Pyhiveapi
plugins: requests-mock-1.9.3
collected 1 item                                                                                

tests/test_hub.py F                                                                       [100%]

=========================================== FAILURES ============================================
________________________________________ test_hub_smoke _________________________________________

    def test_hub_smoke():
        """Test for hub smoke."""
        result = None
    
>       assert result
E       assert None

tests/test_hub.py:8: AssertionError
==================================== short test summary info ====================================
FAILED tests/test_hub.py::test_hub_smoke - assert None
======================================= 1 failed in 0.24s =======================================
(.venv) hermes:Pyhiveapi martin$ 

Am I missing something here? Should I be running the tests in a different way or are the current non-bandit tests just stubs that need filled out?

[KJonline]

I re-wrote the library but never really got around to making the test work, so currently we have no tests what is committed is just playing around to see what we could get working. I have trv’s not multi zone and @Rendili has hot water

[KJonline]

@martintoreilly Happy for you to add tests in we do need to start building the test framework again