Cannot get Represt to work with project where Configuration Management in DNG is enabled

[denisbelanger101]

Hi,

When I use the below strings in a project that has Configuration Management enabled (Opt-in), it never works. Would someone know why it's like that? represt rm resources -J "https://server" -U username -P password -p "Project" -c Requirements -l "Project Initial Stream" -m "100000-Segment 1.1 VOI" -C "C:\TEMP##aatest\test1.csv"-V

Error message: (...) requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://... [PYI-37292:ERROR] Failed to execute script 'represt' due to unhandled exception!

Some help would be very appreciated.

Thank you, Denis

Screenshot to show what is working in my browser and when comparing it to the URL I get from the represt error message.

[barny]

Works for me :-)

From your screenshot it's the OSLC Query to find the module from its name "100000-Segment 1.1 VOI" that's failing, so it's not got as far as using the Reportable REST API.

Try with a different module?

What version of elmclient are you using?

What DOORS Next version and ifix are you using?

[denisbelanger101]

Thank you for your answer, I really appreciate.

The Versions are:

The Module name:

I tried it with four different Modules and I still have errors. I renamed the folder elmclient by elmclient-old and I freshly reinstalled it. Still doesn't work.

Is there a deeper debug mode I could enable or a log that I could provide?

Here is the complete error code...

---

----->NOTE: the editor here does not paste the text as it is... Below is the text and down below is the screenshot of the text...When you paste the text, it is ok, but when you click "Update comment" the text is different...

C:>represt rm resources -J "https://server.clm.ibmcloud.com" -U MyUsername -P password -p "ProjectName" -c Requirements -l "ProjectName Initial Stream" -m "402620-Technoparc" -C "C:\TEMP##aatest\test-new-module1.csv"-V elmclient.httpops: ERROR Exception on executing request. URL: https://server.clm.ibmcloud.com/rm/views?oslc.pageSize=200&oslc.paging=true&oslc.prefix=dcterms%3D%3Chttp%3A//purl.org/dc/terms/%3E%2Crdm_types%3D%3Chttp%3A//www.ibm.com/xmlns/rdm/types/%3E%2Cjazz_rm%3D%3Chttp%3A//jazz.net/ns/rm%23%3E&oslc.query=true&oslc.select=%2A&oslc.where=dcterms%3Atitle%3D%22402620-Technoparc%22%20and%20rdm_types%3AArtifactFormat%3Djazz_rm%3AModule&projectURL=https%3A//server.clm.ibmcloud.com/rm/process/project-areas/_uErTUHN5Eeq8dvNydv8COA, 400, <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:err="http://jazz.net/xmlns/prod/jazz/foundation/1.0/"> rdf:Description <err:detailedMessage rdf:datatype="http://www.w3.org/2001/XMLSchema#string" >GC app is unavailable so we should not be attempting to resolve LCs against GC Application URIs. Submitted GC URI was <https://server.clm.ibmcloud.com/rm/cm/stream/_upLHhnN5Eeq8dvNydv8COA>.</err:detailedMessage> <err:errorMessage rdf:datatype="http://www.w3.org/2001/XMLSchema#string" >Bad Request</err:errorMessage> <err:errorStatus rdf:datatype="http://www.w3.org/2001/XMLSchema#long" >400</err:errorStatus> </rdf:Description> </rdf:RDF>

Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in run_code File "c:\Users\MyUsername\AppData\Roaming\Python\Python313\Scripts\represt.exe_main.py", line 7, in sys.exit(main()) ~~~~^^ File "C:\Users\MyUsername\AppData\Roaming\Python\Python313\site-packages\elmclient\examples\represt.py", line 392, in main represt_main() ~~~~~~~~~~~~^^ File "C:\Users\MyUsername\AppData\Roaming\Python\Python313\site-packages\elmclient\examples\represt.py", line 223, in represt_main queryurl,queryparams,queryheaders = mainapp.process_represt_arguments( args, allapps ) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^ File "C:\Users\MyUsername\AppData\Roaming\Python\Python313\site-packages\elmclient_rm.py", line 1567, in process_represt_arguments modules = queryon.execute_oslc_query( qcbase, whereterms=[['dcterms:title','=',f'"{args.module}"'], ['rdm_types:ArtifactFormat','=','jazz_rm:Module']], select=['*'], prefixes={rdfxml.RDF_DEFAULT_PREFIX["dcterms"]:'dcterms',rdfxml.RDF_DEFAULT_PREFIX["rdm_types"]:'rdm_types',rdfxml.RDF_DEFAULT_PREFIX["jazz_rm"]:'jazz_rm'}) File "C:\Users\MyUsername\AppData\Roaming\Python\Python313\site-packages\elmclient\oslcqueryapi.py", line 442, in execute_oslc_query results = self._execute_vanilla_oslc_query(querycapabilityuri,query_params1, select=select, prefixes=prefixes, show_progress=show_progress, verbose=verbose, maxresults=maxresults, delaybetweenpages=delaybetweenpages, pagesize=pagesize, intent=intent, saverawresults=saverawresults, cacheable=cacheable) File "C:\Users\MyUsername\AppData\Roaming\Python\Python313\site-packages\elmclient\oslcqueryapi.py", line 618, in _execute_vanilla_oslc_query this_result_xml = self.execute_get_rdf_xml(query_url, params=params, headers=headers, cacheable=cacheable, intent=intent) File "C:\Users\MyUsername\AppData\Roaming\Python\Python313\site-packages\elmclient\httpops.py", line 171, in execute_get_rdf_xml response = request.execute( **kwargs ) File "C:\Users\MyUsername\AppData\Roaming\Python\Python313\site-packages\elmclient\httpops.py", line 408, in execute return self._execute_request( no_error_log=no_error_log, close=close, **kwargs ) ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MyUsername\AppData\Roaming\Python\Python313\site-packages\elmclient\httpops.py", line 417, in _execute_request result = self._execute_one_request_with_login( no_error_log=no_error_log, close=close, **kwargs) File "C:\Users\MyUsername\AppData\Roaming\Python\Python313\site-packages\elmclient\httpops.py", line 611, in _execute_one_request_with_login response.raise_for_status() ~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "C:\Users\MyUsername\AppData\Roaming\Python\Python313\site-packages\requests\models.py", line 1024, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://server.clm.ibmcloud.com/rm/views?oslc.pageSize=200&oslc.paging=true&oslc.prefix=dcterms%3D%3Chttp%3A//purl.org/dc/terms/%3E%2Crdm_types%3D%3Chttp%3A//www.ibm.com/xmlns/rdm/types/%3E%2Cjazz_rm%3D%3Chttp%3A//jazz.net/ns/rm%23%3E&oslc.query=true&oslc.select=%2A&oslc.where=dcterms%3Atitle%3D%22402620-Technoparc%22%20and%20rdm_types%3AArtifactFormat%3Djazz_rm%3AModule&projectURL=https%3A//server.clm.ibmcloud.com/rm/process/project-areas/_uErTUHN5Eeq8dvNydv8COA&oslc_config.context=https%3A%2F%2Fserver.clm.ibmcloud.com%2Frm%2Fcm%2Fstream%2F_upLHhnN5Eeq8dvNydv8COA

---

SCREENSHOT OF THE TEXT ABOVE....

[barny]

Add -L TRACE to the commandline - this will generate a log file in the logs folder which gives the request, header and body of all http requests and responses. I'm interested in what is probably the last operation in the log, the GET on /rm/views - the ones that's failing - what are the headers on the request? Obfuscate server names obviously, mostly interested in header Configuration-Context or vvc.configuration, are these present, what are their values?

Try adding -W to the command, this will clear the cache and force retreiving lastest data.

Also, are you able to use the oslcquery example, does something like this work (doing the same query for the module)? oslcquery -J "https://server" -U username -P password -p "Project" -C Requirements -F "Project Initial Stream" -q "dcterms:title=""100000-Segment 1.1 VOI""" -V -W

[barny]

I think this might be something I've fixed indirectly - just updated to version 0,29.0 please try that from pypi and hopefully it will work.

[denisbelanger101]

Ok, I tried 0,29.0. Result is the same. I added -W and -L TRACE , the log is attached here.

elmclient-20250401-144612 - obfuscated.log

For the oslcquery , here is the output screenshot:

Here is the output in an opt-out (CM disabled) project, it works.

I also found this on the web, An old case but that generated similar error messages involving config context... https://www.ibm.com/mysupport/s/defect/aCI3p000000Cwka/dt208104?language=pt_BR

[barny]

Did you try the oslcquery command?

[denisbelanger101]

Yes, as stated above. The big screenshot is the output of the error generated after the oslcquery line was sent.

I also tried it on my home PC. Fresh installation of Python and elmclient. The error message is the same and the header in the log is the same as in the above log.

[denisbelanger101]

Also, I will start a new comment for this before it's a new finding....

In DOORS NEXT, when I choose to see all My Projects, I see the two projects. the Demo project is the one that works and exports to the csv file. The other one, which has its name hidden with the yellow box, is the one that's not working. This one has a component...When I click on Show Component, I get the other window below that says that a component name called "Requirements" exists. I think there is something missing to go one level deeper inside that component to get the Modules. But how do we do this? In the URL of my open DNG there is this string: "&componentURI=https://server.com/rm/rm-projects/_uErTUHN5Eeq8dvNydv8COA/components/_upG2EHN5Eeq8dvNydv8COA" the project area is at the beginning as you can see, then there is a sub-level named "Component" would that be the issue? In the command I have a parameter "-c Requirements" but it's like the code does not see it...or I use it wrongly...

[denisbelanger101]

Another finding: In the get HTTP URL, the Module URI is not there, only the literal title: In the whole log file, the URI of the Module is not found...

The Module name is: 402620-Technoparc

The Module URI is: MD_C1L58Bo8Eeylw979BSgNIA

Shouldn't the module be called by its URI instead of the literal form?

[denisbelanger101]

I also redo the test for OSLCquery. The comand I used was this one:

oslcquery -J " https://Server.ibmcloud.com" -U Username -P Password -p "Project" -C "Requirements" -F "Project Initial Stream" -q "dcterms:title="402620-Technoparc"" -V -W -L DEBUG

It still didn't work but the data in the log is different.

The output window data is below and the debug log attached.

---

elmclient-20250403-144307 - obfuscated.log

[denisbelanger101]

Hi,

I tried everything I could think of, but still doesn't work. It works only with OPT-OUT project not when in a OPT-IN project. For the OPT-IN project, nothing works and all the logs are above. It is really unfortunate because it looks like there was a lot of energy put in this code project, and to see it not working is really sad. These functionalities would be so much useful.

Regards, Denis Belanger

[barny]

Hi Denis

I use oslcquery all the time on opt-in projects, so I don't think that's the issue. represt I don't use much, but AFAICT it works with opt-in.

I'll have to review what you provided, hopefully I can figure out the problem.

Does the project have archived configurations or components?

Regards Ian

[denisbelanger101]

Hi Ian,

There are no archived configurations or components. Tell me, are there any calls in this code that would require me to have an admin role in DNG?

Thank you, Denis

[denisbelanger101]

Good morning Ian,

Did you have time to look at what could be the issue?

Thank you! Denis

[denisbelanger101]

Hi Ian,

I know this is a collaborative project and that it is worked on only when people have time to do so. If you don't have time to look at the issue, would there be someone else that could look at it?

I have put more details on the issue to extract a Module from our DNG to csv. If someone can look at it , I would really really appreciate it!

Thank you, Denis

[denisbelanger101]

Hi,

Here are a couple of things more that we did. We created a new project, opt-in, in which I have fill admin access rights, and I tried to export a module using the same command string and it still doesn't work. From this point, we will stop working on this. We have thousands of users around the world that use DNG and they will not be able to enjoy all this work. That's so unfortunate.

Thanks, Denis

[barny]

Apologies for the delay getting back to you. Day job to do :-o

oslcquery -J " https://Server.ibmcloud.com" -U Username -P Password -p "Project" -C "Requirements" -F "Project Initial Stream" -q "dcterms:title="402620-Technoparc"" -V -W -L DEBUG

The problem with this is the quotes around/in the query. Assuming you're on Windows (because you're using double-quotes) the " in the query have to be doubled, probably to:

-q "dcterms:title=""402620-Technoparc"""

Try that, does oslcquery work better now?

[barny]

Shouldn't the module be called by its URI instead of the literal form?

One of the aims of oslcquery is to make query easier to use because URIs aren't very human-friendly, so the query uses the human-friendly name and the code finds the module from the name and uses its URI in the query :-)

[barny]

Think I found the problem - there were outdated references to rdm_types:artifactFormat which should have been replaced with rdf:type. Will publish a new version probably tomorrow. Please let me know if it works for you?

[barny]

Just published 0.32.0 - should work now!

[barny]

would there be someone else that could look at it?

I've had some very minor contributions, but basically there isn't anyone else I can call on: I'm it! It's open-source so anyone with some Python and ELM OSLC API knowledge can contribute, that would be really helpful.

Hope it works better for you now!

Regards Ian

[denisbelanger101]

Hi Ian!

Thanks for the update! Really appreciated! Attached is the CMD promp

CMD Prompt output.txt

t output when I try the query.

Thank you, Denis

[denisbelanger101]

(...) And here is the CMD prompt output when using the represt script.

CMD Prompt output - for represt.txt

[barny]

Ha that had me bamboozled for a couple of minutes, then I spotted that you used option -o (lowercase-oh) and I think you mean -O (capital-oh). -o is for an orderby property and "module1" can't be parsed as a tagged name so you get the rather ridiculously incomprehensible error message - sorry for that. Try with -O.

[denisbelanger101]

I change the o for a O. Here is the result...

CMP results output for OSLCquery.txt

[barny]

Looks like the represt problem is because GC isn't installed, never tested that before :-o Probably similar for oslcquery.

[denisbelanger101]

Ok, and GC stands for Garbage collection?

[barny]

Global Configurations application - I should have said GCM. Seems like the Reportable REST API doesn't like having the config specified using a specific parameter when GC not installed. Similar problem with OSLC Query. 0.33.0 published should work now.

[denisbelanger101]

Congrats! It works without errors! Thank you very much :) represt gave the entire module, and oslcquery gave something but it might be just setting the query correctly to have the expected output, but there was no errors either.

For the represt output csv file, do you know if it exists a code to convert the file to a "DNG Module" view with attribute columns and without the other data for the server source, module name, URI etc?

[denisbelanger101]

Hi Ian,

Something I noticed, the built-in attributes like Primary Text, Module, Artifact Type are not extracted. Only "Name" is, and Name is an attribute that typically displays the title or short description of an artifact. In other words, I get a truncated Primary Text which is not good. We also parse the Module attribute, which give the module number. We use this number to rename the file to the customer's name nomenclature.

Please look at the attached file for more details.

Thank you, Denis