GEOS icon indicating copy to clipboard operation
GEOS copied to clipboard

Log refactor - Output PVT

Open arng40 opened this issue 1 year ago • 3 comments

This PR follows the https://github.com/GEOS-DEV/GEOS/pull/2984 that aim to standardize logs. Currently PVT tables are only accessible through .csv files. In this PR, logs will indicate which csv files are generated along with their associated path

CSV Generated to inputFiles/compositionalMultiphaseWell/test.xml/fluid1_phaseModel1_PhillipsBrineDensity_table.csv :

---------------------------------------------------------------------------------------
|                    fluid1_phaseModel1_PhillipsBrineDensity_table                    |
---------------------------------------------------------------------------------------
|  The fluid1_phaseModel1_PhillipsBrineDensity_table PVT table exceeding 500 rows.    |
|  To visualize the tables, go to the generated csv                                   |
---------------------------------------------------------------------------------------

In addition to csv files, PVT Tables with dimensions n <= 2 can be written in log by meetings 2 conditions :

  • have sufficient logLevel
  • Rows in the table < 500 lines

CSV Generated to inputFiles/compositionalMultiphaseWell/test.xml/fluid1_phaseModel1_PhillipsBrineViscosity_table.csv

---------------------------------------------------------
|    fluid1_phaseModel1_PhillipsBrineViscosity_table    |
---------------------------------------------------------
|    temperature [C]     |       viscosity [Pa*s]       |
---------------------------------------------------------
|          0.01          |          0.0017914           |
|           10           |           0.001306           |
|           20           |          0.0010016           |
|           25           |           0.00089            |
|           30           |          0.0007972           |
---------------------------------------------------------

An example of 2D table :

----------------------------------------------------------------------------------------------------------
|                             fluid1_phaseModel2_SpanWagnerCO2Density_table                              |
----------------------------------------------------------------------------------------------------------
|           density [kg/m3]  |  temperature [C] = 12  |  temperature [C] = 62  |  temperature [C] = 112  |
----------------------------------------------------------------------------------------------------------
|    pressure [Pa] = 100000  |     1.867036801109489  |    1.5846667997020552  |      1.377165506424985  |
|   pressure [Pa] = 1100000  |    21.873072823677948  |     18.05377355524998  |     15.472646535860257  |
|   pressure [Pa] = 2100000  |     45.02058216836334  |     35.79619626115461  |     30.190059516194083  |
|   pressure [Pa] = 3100000  |      73.0547505549322  |     55.06117914599007  |     45.580574187479066  |
						[...]
|  pressure [Pa] = 67100000  |     1107.292557196501  |     983.4638012355585  |      863.9649406367214  |
|  pressure [Pa] = 68100000  |    1109.1855359680847  |     986.2233300634709  |       867.698289923355  |
|  pressure [Pa] = 69100000  |    1111.0569954467694  |     988.9398867966821  |      871.3611145780119  |
|  pressure [Pa] = 70100000  |    1112.9074848288449  |      991.614907000032  |      874.9561390383249  |
|  pressure [Pa] = 71100000  |    1114.7375318533834  |     994.2497540203499  |      878.4859333715468  |
|  pressure [Pa] = 72100000  |    1116.5476438596384  |     996.8457238250027  |      881.9529245875995  |
|  pressure [Pa] = 73100000  |    1118.3383088824808  |     999.4040493886989  |      885.3594069497734  |
|  pressure [Pa] = 74100000  |    1120.1099966304896  |    1001.9259047775354  |      888.7075513940321  |
----------------------------------------------------------------------------------------------------------

arng40 avatar May 29 '24 08:05 arng40

Codecov Report

Attention: Patch coverage is 97.72727% with 4 lines in your changes missing coverage. Please review.

Project coverage is 56.39%. Comparing base (c225416) to head (a537cdd). Report is 77 commits behind head on develop.

Files with missing lines Patch % Lines
src/coreComponents/functions/TableFunction.cpp 97.50% 2 Missing :warning:
...c/coreComponents/common/format/table/TableData.cpp 92.85% 1 Missing :warning:
...eComponents/common/format/table/TableFormatter.cpp 50.00% 1 Missing :warning:
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #3149      +/-   ##
===========================================
+ Coverage    56.33%   56.39%   +0.06%     
===========================================
  Files         1065     1065              
  Lines        90242    90326      +84     
===========================================
+ Hits         50836    50941     +105     
+ Misses       39406    39385      -21     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Jun 03 '24 15:06 codecov[bot]

@dkachuma I've updated the PR description, you can have a look of a 2D table representation

arng40 avatar Jun 19 '24 12:06 arng40

@arng40 This PR is not approved. I am moving it off of the queue.

rrsettgast avatar Jul 02 '24 19:07 rrsettgast

Hello @herve-gross , to get my PR into the merge queue, which code owners do I need to have approved? Because of merging with the develop, a lot of the code owners seem to be required.

arng40 avatar Jul 12 '24 15:07 arng40

Hello @herve-gross , to get my PR into the merge queue, which code owners do I need to have approved? Because of merging with the develop, a lot of the code owners seem to be required.

@rrsettgast is the right person for this.

herve-gross avatar Jul 12 '24 15:07 herve-gross

Hello @herve-gross , to get my PR into the merge queue, which code owners do I need to have approved? Because of merging with the develop, a lot of the code owners seem to be required.

it's based on the folders you touched. What's a bit annoying is that it's not very easy to know which are the strictly necessary ones. you only need one codeowner for each folder but github does not really tell you what is the minimum subset. You can kind of figure it out by hoovering on the files in the Files changed view.

CusiniM avatar Jul 12 '24 18:07 CusiniM

Some integrated tests are failing, @arng40 please verify what are the differences with the baseline. If these differences are only related to log output & are satisfying, add the "need rebaseline" tag and remove the "do not merge".

MelReyCG avatar Jul 23 '24 09:07 MelReyCG

Some integrated tests are failing, @arng40 please verify what are the differences with the baseline. If these differences are only related to log output & are satisfying, add the "need rebaseline" tag and remove the "do not merge".

I don't understand why I have these errors :

sherlock : sccache: error: Timed out waiting for server startup. Maybe the remote service is unreachable? and cuda side :

Error: error: RPC failed; curl 92 HTTP/2 stream 0 was not closed cleanly: CANCEL (err 8)
Error: error: 5081 bytes of body are still expected

Hello @rrsettgast, Who can I contact for these errors ?

arng40 avatar Jul 23 '24 13:07 arng40

Some integrated tests are failing, @arng40 please verify what are the differences with the baseline. If these differences are only related to log output & are satisfying, add the "need rebaseline" tag and remove the "do not merge".

I don't understand why I have these errors :

sherlock :

sccache: error: Timed out waiting for server startup. Maybe the remote service is unreachable?

and cuda side :


Error: error: RPC failed; curl 92 HTTP/2 stream 0 was not closed cleanly: CANCEL (err 8)

Error: error: 5081 bytes of body are still expected

Hello @rrsettgast, Who can I contact for these errors ?

Hello. Rerun the job. Sometimes sccache fails to access the Google account. This looks like that situation

rrsettgast avatar Jul 23 '24 13:07 rrsettgast

Hi @rrsettgast , @paveltomin, I only have this error in the integrated tests :

Rank 0 is comparing /tmp/tmp.LquJVkCo6y/GEOS_integratedTests_working/compositionalMultiphaseFlow/co2_hybrid_1d_02/0to10_restart_000000010/rank_0000000.hdf5 with /tmp/tmp.LquJVkCo6y/GEOS_integratedTests_baselines/compositionalMultiphaseFlow/co2_hybrid_1d_02/0to10_restart_000000010/rank_0000000.hdf5 
********************************************************************************
Error: /Problem/domain/Constitutive/fluid
	Group has a child 'writeCSV' in the file to compare but not the baseline file.
********************************************************************************

It seems normal to me because I added the writeCSV attribute in CO2BrineFluid file. I think we can safely rebaseline.

arng40 avatar Aug 30 '24 08:08 arng40

@arng40 I suggest you to go for the "option 2" I proposed.

It's good

arng40 avatar Sep 04 '24 15:09 arng40

A few more adjustments may need to be done.

MelReyCG avatar Sep 04 '24 15:09 MelReyCG

Hello @rrsettgast , All the comments has been addressed, I think we can safely rebase the PR and merge it

arng40 avatar Sep 09 '24 21:09 arng40

@arng40 @MelReyCG There are some documentation errors. This can be a little hard to navigate, and as we have just started enforcing this, there is no documentation on the process of finding errors from the readthedocs logs.

rrsettgast avatar Oct 01 '24 03:10 rrsettgast