lightning icon indicating copy to clipboard operation
lightning copied to clipboard

Improve performance of the History page queries

Open stuartc opened this issue 2 years ago • 2 comments

Coming off #968.

We have several places where performance of the History page query is hampered, here is a list of things I noticed:

  1. We should not cast columns when searching, for example both log_lines and the dataclip tables have their columns cast into varchars - this is very expensive.
  2. Finding the last run by joining on finished_at is exponentially less efficient depending on how many attempts & runs there are.

Proposed solutions

  • [x] #1898
  • [ ] #1899
  • [ ] Add index to attempt_runs for attempt_id and/or run_id, currently the compound index doesn't appear to be used for these queries.

These solutions are distinct (and should be done regardless) from taking another approach which is creating aggregate tables

  1. Create aggregate (or columns) that get updated when an attempt/run is created or updated. Removing the need to join and filter Runs by exit code and finished_at.

References:

invocation.ex:417 - list_work_orders_for_project_query/2

stuartc avatar Jul 27 '23 08:07 stuartc

@stuartc, @taylordowns2000 back then when we were discussing about which partition to use for logs, I have talked about the benefits of easier records cleanup (drop expired partitions) and narrowing the query scope to a specific partition when user filters by date. For the incremental search on History Page approach, it might be an opportunity before the launch to switch the logs partitioning to by range.

jyeshe avatar Mar 18 '24 11:03 jyeshe

We'll need to make some synthetic benchmarks to know where our performance dropoff is unacceptable. Changing now will be quite a bit of work, and changing back (or any other flavour of partition) later will be even more difficult. We already have production data in the tables so we have to treat it as critical.

stuartc avatar Mar 18 '24 15:03 stuartc