Add cmdlet Test-IshSetting enabling a remote quality check

[ddemeyer]

Sharing an idea, hoping for a thumbs-up on the issue to show you are interested, perhaps even a community contribution.

New cmdlet Test-IshSetting is almost a test CMS settings - at least test the ones available over the web services API. So knowing the CMS software version, we can run a test suite for that (and older) CMS versions on how the system should behave.

• A similar idea is Compare-IshSettings - inspired by existing Compare-IshTypeFieldDefinition - that would require settings containers that can be reused and be made version aware. This could be very useful for DTAP instances - so is Acceptance running the same settings as Production.
• This cmdlet could check per product version if - this also means an ISHRemote per InfoShare release which is the case anyway.

Tests... For now focused on CMS fields/values/settings, not on configuration files (e.g. metadataconfig.xml, etc) or User/UserGroup/UserRoles

• Expected fields in IshTypeFieldDefinition - for example SRQ-16246 FISHPUBLANGUAGE is required since version x.y.z)… or should this be a Test-IshTypeFieldDefinition? No I think it can go here
• Expected CTCONFIGURATION fields with help, like "Run DBUT"
• Expected CTCONFIGURATION field values with help, like "Login as admin on real-url and correct the field bla"
• Expected plugin names (perhaps in a certain order with certain ishcondition, whatever changed from A->B) in the xml settings fields. Schema validation already happened server-side, this is about minimum business rules. With help, like "Login as admin on real-url and correct the field bla and make sure plugin X is configured in section Y with condition Z according to Administrator's Upgrade documentation" For example IshWritePlugin will get public "Check140003" which calls private "CheckISHEXTRACTLNGTITLETOMETADATA" functions (latter can be reused across versions)
  • ISHEXTRACTLNGTITLETOMETADATA is responsible to extract the titles from the new content to use it in the Outline view.
  • SETRESOLVEDONFIELD and SETRESOLVEDBYFIELD are responsible to fill in some values when you resolve an annotation, so these will fill in that resolvedTimestamp the system currently complains about.
  • added SETGLOSSTERM which seems to be new in 14SP3 as well

Parameter Sets...

• Parameter Set TestGroup
  • Optional parameter -IshVersion. Defaults to the detected product version over IshSession, you can explicitly specify the product version to run a certain test suite.
  • Output is a boolean
• Parameter Set ReportGroup
  • Mandatory boolean-switch -ShowReport parameter, so Test-* cmdlet no longer ends with a true/false. Similar to parameter -Quiet that forces a true/false answer like Test-Connection (ahum Ping) has.
  • Optional parameter -IshVersion. Defaults to the detected product version over IshSession, you can explicitly specify the product version to run a certain test suite.
  • (Optional parameter -IshSettingTestLevel, something that indicates the log level... for now show all)
  • Output are IshSettingTest objects that can be pretty-printed

Output IshSettingTest in detail... Pipeline objects that show

• ClientVersion, same as IshSession, in essence it states what the available tests are (the good, the bad and the bugs)
• ServerVersion, same as IshSession, showing the detected CMS version
• TestObject, something that identifies the thing getting tested. Could be CTCONFIGURATION, or a field name
• TestSinceVersion, indicates the IshVersion where the test originated for... could be 1.0.0.0 even
• TestLevel, probably an enum, so Error/Warning/Verbose that could influence the
• TestName, so
• TestResult, a boolean
• TestReason, the problem we are trying to detect and avoid
• TestAction, the corrective action

Implementation Input...

• Objects around Xml Settings, could be reused by Compare-IshSetting or even New-IshSession
• IshSettingTest with some nice short hand for rendering (format.xml)
• Where are the tests living, definitely not in the Test-IshSetting cmdlet, so inside the objects, ...some aggregation object which is loaded on the IshSession like IshTypeFieldDefinition