azure-functions-sql-extension Set up perf test pipeline and monitoring

[x] Work was started for perf tests such that we have them and can run them, but right now it's a manual effort and not something that we actively run or monitor. We should set it up so it runs on a set schedule and keep an eye out for issues coming from it.
[ ] Set up monitoring/compare perf numbers to a baseline

https://github.com/Azure/azure-functions-sql-extension/blob/main/builds/azure-pipelines/performance.yml

Apr 05 '23 20:04 Charles-Gagnon

Closing this as addressed by #799

May 01 '23 19:05 MaddyDev

Looks like the perf tests were broken by my integration test refactor - will address that now.

May 01 '23 19:05 lucyzhang929

@MaddyDev @lucyzhang929 We had also discussed how we intend to use this pipeline, it's not very helpful right now as it is since we don't actually measure anything or compare against any sort of baseline. So we should either re-open this issue for tracking that additional work, or make a separate issue to follow up with getting that done.

May 01 '23 20:05 Charles-Gagnon

Modified the issue description to track perf monitoring.

May 01 '23 20:05 lucyzhang929

Monitoring perf numbers to a baseline plan:

For a crawl, we decided to manually add the current runtime numbers as baseline in a json file and as we make improvements, checking manually that we didn't deviate from the baseline numbers.
Plugin our telemetry and have a way to get these numbers and have a perf dashboard for us to monitor.
For run, have an automatic way to compare the runs against the baseline and fail if they deviate beyond a threshold.

Will be moving this to Post GA after the crawls is completed. (https://github.com/Azure/azure-functions-sql-extension/pull/941)

Oct 06 '23 17:10 MaddyDev