plutus icon indicating copy to clipboard operation
plutus copied to clipboard

Decide how to deal with multiple cost models in the tests

Open kwxm opened this issue 1 year ago • 4 comments

PR #6087 introduces multiple cost models for Plutus Core. There are many budget tests which currently only use the latest cost model. We should decide what do do with these: for example, should we run the tests with every cost model and have a golden file for each?

kwxm avatar May 22 '24 20:05 kwxm

Something else to consider is that not all builtins are available in all Plutus Language versions. The on-chain deserialisation process checks for the use of unavailable builtiins and fails if any are found. However many (most?) of our tests are either read from textual UPLC or compiled from PlutusTx and no check for availability is performed, so you can run a program containing a builtin which is actually illegal for the PV/LL in use. We might want to serialise and then deserialise again before test execution in order to avoid this. The same could be done for uplc and so on. In fact, unavailable builtins are currently detected in these situations by giving them costing functions which are maximally expensive, but this isn't ideal and may change. Also this technique is slightly weaker than the deserialisation check because that will reject a script immediately if it contains a prohibited builtin whereas using an expensive costing function will only detect a prohibited builtin when it is called, which may never happen,

kwxm avatar May 23 '24 10:05 kwxm

We may also want to think about nofib-exe. For instance how do we evaluate terms? Do we want a flag saying which Plutus/PV/cost model to use?

kwxm avatar May 28 '24 18:05 kwxm

Also decide what to do with defaultCostModelParams.json.

kwxm avatar May 28 '24 20:05 kwxm

Maybe get rid of defaultCekParametersForTesting altogether: see this comment on PR #6087.

kwxm avatar May 28 '24 20:05 kwxm