ENH: Introducing local sensitivity analysis

[Lucas-Prates]

Pull request type

• [x] Other (please describe): This PR is a draft to implement sensitivity analysis in RocketPy. It is a work in progress and needs validation.

Checklist

• [x] Lint (black rocketpy/ tests/) has passed locally

Current behavior

The Sensitivity Analysis notebook teaches the users how to perform the simulations, plot the distribution of some flight variables (e.g. apogee), and computes the prediction ellipses for the landing point.

New behavior

Our goal is to take sensitivity analysis even further. Briefly, we attempt to answer the following question: Which parameters would reduce the variability of the variable of interest (e.g. apogee) the most if we measured them with greater precision?

To that end, a bit of theory is developed, check the technical document. What was developed resembles the work of [1], a core reference in sensitivity analysis for engineering. His approach is a global sensitivity analysis with a full model containing interaction terms. Our first implementation considers a local sensitivity analysis using only first-order terms.

A quick and dirty test of the functionality of the SensitivityModel class is provided the "sensitivity_model_usage" notebook. This notebook is currently giving weird results! The linear approximations for the variables are, for some reason I still have to figure out, not good enough. This was not happening at previous experimentations that suggested that this approached worked. I have to look carefully at what is happening, but I did not want to delay the PR.

The concepts are discussed in-depth in the "sensitivity_analysis_parameter_importance" notebook (the notebook was not updated to the new SensitivityModel yet!)

Breaking change

• [x] No

Additional information

Technical Document

[1] Sobol, Ilya M. "Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates." Mathematics and computers in simulation

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 15.20000% with 212 lines in your changes missing coverage. Please review.

Project coverage is 74.40%. Comparing base (3a4c742) to head (522c8fe). Report is 38 commits behind head on develop.

Files with missing lines	Patch %	Lines
rocketpy/sensitivity/sensitivity_model.py	14.28%	102 Missing :warning:
rocketpy/tools.py	6.12%	46 Missing :warning:
rocketpy/prints/sensitivity_prints.py	16.66%	40 Missing :warning:
rocketpy/plots/sensitivity_plots.py	25.00%	24 Missing :warning:

Additional details and impacted files

@@             Coverage Diff             @@
##           develop     #575      +/-   ##
===========================================
- Coverage    75.87%   74.40%   -1.47%     
===========================================
  Files           85       89       +4     
  Lines        10085    10335     +250     
===========================================
+ Hits          7652     7690      +38     
- Misses        2433     2645     +212     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[Gui-FernandesBR]

Tests are not passing

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/runner/work/RocketPy/RocketPy/rocketpy/__init__.py", line 2, in <module>
    from .environment import Environment, EnvironmentAnalysis
  File "/home/runner/work/RocketPy/RocketPy/rocketpy/environment/__init__.py", line 1, in <module>
    from .environment import Environment
  File "/home/runner/work/RocketPy/RocketPy/rocketpy/environment/environment.py", line [16](https://github.com/RocketPy-Team/RocketPy/actions/runs/9096367936/job/25001616031?pr=575#step:6:17), in <module>
    from ..tools import exponential_backoff
  File "/home/runner/work/RocketPy/RocketPy/rocketpy/tools.py", line 556, in <module>
    parameters_list: list[str],
TypeError: 'type' object is not subscriptable
Error: Process completed with exit code 1.

Could you fix it before our review, please? That would help us. @Lucas-Prates

[Lucas-Prates]

Could you fix it before our review, please? That would help us. @Lucas-Prates

Sure, I will fix it briefly. This simplified type hinting started at python 3.9. I will make sure the tests pass this time. :P

[Gui-FernandesBR]

Please be aware of #444, we are not supporting type hinting or annotations yet.

[Lucas-Prates]

The PR is ready for review again. I fixed a bug on the computations of the sensitivity score and it explained why the model was failing overall. Now it works for several target variables and for more complex environments (e.g. forecasted atmosphere).

The sensitivity model usage and sensitivity simulation notebooks are the most important to review. There are still a couple of things left to do:

• Improve and extend the writing in the sensitivity model usage notebook;
• Understand why the approximation for x_impact and y_impact is failing.

[Lucas-Prates]

@Gui-FernandesBR and @MateusStano, this PR is ready for review again! I believe the review should be somewhat quick. The code has already been thoroughly review, so I recommend focusing on two files:

• docs/technical/sensitivity.rst
• docs/user/sensitivity.rst

In both cases, my recommendation is to read the text and check if it is clear. I wouldn't mind much about the mathematics, I took a second look at it and "looks fine." Moreover, I would like to know from you if the example I provided makes sense. It is very simplified, but I would hope it somehow creates the impression that sensitivity analysis can be useful.

@Gui-FernandesBR, I tried to address the "LAE is too large" by giving a pragmatic recommendation of how to use the tool. Giving an exact value of what is "too large" is, in my opinion, not easy/feasible right now.

[Gui-FernandesBR]

@Gui-FernandesBR and @MateusStano, this PR is ready for review again! I believe the review should be somewhat quick. The code has already been thoroughly review, so I recommend focusing on two files:

• docs/technical/sensitivity.rst
• docs/user/sensitivity.rst

In both cases, my recommendation is to read the text and check if it is clear. I wouldn't mind much about the mathematics, I took a second look at it and "looks fine." Moreover, I would like to know from you if the example I provided makes sense. It is very simplified, but I would hope it somehow creates the impression that sensitivity analysis can be useful.

@Gui-FernandesBR, I tried to address the "LAE is too large" by giving a pragmatic recommendation of how to use the tool. Giving an exact value of what is "too large" is, in my opinion, not easy/feasible right now.

@Lucas-Prates your proposed example is exactly what we needed when we created the issue. Problem solved, thank you so much!