loopback-datasource-juggler icon indicating copy to clipboard operation
loopback-datasource-juggler copied to clipboard

Js-code from lib/model.js gets included in SOAP-request causing SOAP-fault

Open jmmeijer opened this issue 8 years ago • 39 comments

Steps to reproduce

  • [ ] bug

I have auto discovered a SOAP 1.1 Webservice with lb soap and Loobback created models as expected. However when using the StrongLoop API explorer, the SOAP requests fail as they are not properly formatted.

Stack trace

"Error: faultcode: soap:Client faultstring: Unmarshalling Error: cvc-complex-type.2.4.a: Invalid content was found starting with element '__cachedRelations'. One of '{xxx, xxx}' is expected.
at XMLHandler.xmlToJson (testapp\node_modules\strong-soap\src\parser\xmlHandler.js:635:23)
at testapp\node_modules\strong-soap\src\client.js:279:28
at testapp\node_modules\loopback\node_modules\loopback-datasource-juggler\lib\observer.js:172:22
at doNotify (testapp\node_modules\loopback\node_modules\loopback-datasource-juggler\lib\observer.js:99:49)
at SOAPConnector.ObserverMixin._notifyBaseObservers (testapp\node_modules\loopback\node_modules\loopback-datasource-juggler\lib\observer.js:122:5)
at SOAPConnector.ObserverMixin.notifyObserversOf (testapp\node_modules\loopback\node_modules\loopback-datasource-juggler\lib\observer.js:97:8)
at cbForWork (testapp\node_modules\loopback\node_modules\loopback-datasource-juggler\lib\observer.js:162:14)
at Request._callback (testapp\node_modules\loopback-connector-soap\lib\http.js:98:9)
at Request.self.callback (testapp\node_modules\request\request.js:188:22)
at emitTwo (events.js:106:13)
at Request.emit (events.js:191:7)
at Request.<anonymous> (testapp\node_modules\request\request.js:1171:10)
at emitOne (events.js:96:13)
at Request.emit (events.js:188:7)
at IncomingMessage.<anonymous> (testapp\node_modules\request\request.js:1091:12)\
at IncomingMessage.g (events.js:292:16)"

When enabling the debug on loopback:persisted-model I can see the SOAP xml is cluttered by code originating from strongloop/loopback-datasource-juggler as is mentioned in issue 77 from loopback-connecter-soap by @liudonghua123. Since the code is in this repo I thought it would be best to open an issue here.

Expected result

I would expect the SOAP request to be properly formatted. And a SOAP response without an error.

Additional information

win32 x64 6.11.0

+-- [email protected] +-- [email protected] +-- [email protected] +-- [email protected] +-- [email protected]

jmmeijer avatar Jun 19 '17 19:06 jmmeijer

I can't help but think this has something to do with the XMLParser and the conversion from JS object to JSON to XML. But until now I cannot quite put my finger on the issue.

jmmeijer avatar Jun 21 '17 18:06 jmmeijer

@jmmeijer Can you post your WSDL with XSDs here?

rashmihunt avatar Jun 22 '17 22:06 rashmihunt

@jmmeijer As per the stack trace, I see that you are getting a response from your Web Service and it's a Fault response and the Fault response message from your web service is Invalid content was found starting with element '__cachedRelations'. One of '{xxx, xxx}' is expected. Can you check why your Web Service is returning this fault for your request? Check your SOAP request and see where it has '__cachedRelations' in SOAP body.. I am suspecting the JSON input for the request may be incorrect.

rashmihunt avatar Jun 22 '17 22:06 rashmihunt

@rashmihunt Thank you for your fast reply!

It seems there is no XSD defined for the WSDL. I was trying to post to method getDeelnemer, which has two parameters: apiSleutel and deelnemernummer.

This is the request format generated by SoapUI:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" 
xmlns:api="http://api.algemeen.webservices.eduarte.topicus.nl/">
   <soapenv:Header/>
   <soapenv:Body>
      <api:getDeelnemer>
         <!--Optional:-->
         <apiSleutel>xxx</apiSleutel>
         <deelnemernummer>xxx</deelnemernummer>
      </api:getDeelnemer>
   </soapenv:Body>
</soapenv:Envelope>

This returns valid response from the webservice.

This is the JSON that the Strongloop API Explorer proposes, which seems correspond with the SOAP XML.

{
  "apiSleutel": "string",
  "deelnemernummer": 0
}

Here are links to the WSDL:

jmmeijer avatar Jun 23 '17 07:06 jmmeijer

@rashmihunt The problem is similar to or the same as the issue i referred to. A complete SOAP request is posted over there. The problem is a part of the JS-code from /lib/model.js gets included in the SOAP request. Somehow the object of ModelBaseClass is converted to SOAP xml including some of the code to initiate its properties. This is how __cachedRelations (a protected propertyof the model) gets converted to <__cachedRelations/>.

jmmeijer avatar Jun 28 '17 20:06 jmmeijer

@rashmihunt Something goes wrong after the method jsonToXML gets called: xmlHandler.jsonToXml(soapBodyElement, nsContext, inputBodyDescriptor, args); I think the problem might be in the inputBodyDescriptor, when the form is unqualified, and because of that the object of ModelBaseClass is not properly converted to SOAP. Also the URI's referred in the WSDL's to are no longer available.

strong-soap:client client request. inputBodyDescriptor:

{
	"elements": [
		{
			"elements": [],
			"attributes": [],
			"qname": {
				"nsURI": "http://api.algemeen.webservices.eduarte.topicus.nl/",
				"name": "apiSleutel"
			},
			"type": {
				"nsURI": "http://www.w3.org/2001/XMLSchema",
				"name": "string"
			},
			"form": "unqualified",
			"isMany": false,
			"isSimple": true,
			"refOriginal": {
				"elements": [],
				"attributes": [],
				"qname": {
					"nsURI": "http://api.algemeen.webservices.eduarte.topicus.nl/",
					"name": "apiSleutel"
				},
				"type": {
					"nsURI": "http://www.w3.org/2001/XMLSchema",
					"name": "string"
				},
				"form": "unqualified",
				"isMany": false,
				"isSimple": true
			}
		},
		{
			"elements": [],
			"attributes": [],
			"qname": {
				"nsURI": "http://api.algemeen.webservices.eduarte.topicus.nl/",
				"name": "deelnemernummer"
			},
			"type": {
				"nsURI": "http://www.w3.org/2001/XMLSchema",
				"name": "int"
			},
			"form": "unqualified",
			"isMany": false,
			"isSimple": true,
			"refOriginal": {
				"elements": [],
				"attributes": [],
				"qname": {
					"nsURI": "http://api.algemeen.webservices.eduarte.topicus.nl/",
					"name": "deelnemernummer"
				},
				"type": {
					"nsURI": "http://www.w3.org/2001/XMLSchema",
					"name": "int"
				},
				"form": "unqualified",
				"isMany": false,
				"isSimple": true
			}
		}
	],
	"attributes": [],
	"qname": {
		"nsURI": "http://api.algemeen.webservices.eduarte.topicus.nl/",
		"name": "getDeelnemer"
	},
	"type": {
		"nsURI": "http://api.algemeen.webservices.eduarte.topicus.nl/",
		"name": "getDeelnemer"
	},
	"form": "qualified",
	"isMany": false,
	"isSimple": false,
	"typeDescriptor": {
		"elements": [
			{
				"elements": [],
				"attributes": [],
				"qname": {
					"nsURI": "http://api.algemeen.webservices.eduarte.topicus.nl/",
					"name": "apiSleutel"
				},
				"type": {
					"nsURI": "http://www.w3.org/2001/XMLSchema",
					"name": "string"
				},
				"form": "unqualified",
				"isMany": false,
				"isSimple": true,
				"refOriginal": {
					"elements": [],
					"attributes": [],
					"qname": {
						"nsURI": "http://api.algemeen.webservices.eduarte.topicus.nl/",
						"name": "apiSleutel"
					},
					"type": {
						"nsURI": "http://www.w3.org/2001/XMLSchema",
						"name": "string"
					},
					"form": "unqualified",
					"isMany": false,
					"isSimple": true
				}
			},
			{
				"elements": [],
				"attributes": [],
				"qname": {
					"nsURI": "http://api.algemeen.webservices.eduarte.topicus.nl/",
					"name": "deelnemernummer"
				},
				"type": {
					"nsURI": "http://www.w3.org/2001/XMLSchema",
					"name": "int"
				},
				"form": "unqualified",
				"isMany": false,
				"isSimple": true,
				"refOriginal": {
					"elements": [],
					"attributes": [],
					"qname": {
						"nsURI": "http://api.algemeen.webservices.eduarte.topicus.nl/",
						"name": "deelnemernummer"
					},
					"type": {
						"nsURI": "http://www.w3.org/2001/XMLSchema",
						"name": "int"
					},
					"form": "unqualified",
					"isMany": false,
					"isSimple": true
				}
			}
		],
		"attributes": [],
		"name": "getDeelnemer",
		"xmlns": "http://api.algemeen.webservices.eduarte.topicus.nl/",
		"isSimple": false
	}
}

strong-soap:client client request, calling jsonToXml. args:

{
	"apiSleutel": "xxx",
	"deelnemernummer": 000
}

jmmeijer avatar Jul 01 '17 12:07 jmmeijer

@rashmihunt Upon further investigation I found out, that args is not properly passed to xmlHandler.jsonToXml(). Somehow the code of the lib/model.js gets into the method. I've added the line debug('jsonToXml, node: %s', node); after src/parser/xmlHandler.js:26 and the code of the object of ModelBaseClass gets logged...

jmmeijer avatar Jul 01 '17 14:07 jmmeijer

Hi, I'm having the same problem in my application. Is there any way to fix this? Thanks

ghost avatar Jul 14 '17 15:07 ghost

@jmmeijer or @rashmihunt: Is there any workaround (e.g. by using an older version) or an expected duration until a fix will be available?

This only seems to affect models created using lb soap as creating the calls to the Soap Webservice yourself as described in e.g. in the strongloop blog seems to work.

I'm asking due to trying to evaluate whether it is worth it manually creating the soap calls until the fix will be available as lb soap would make this much easier.

ExTheSea avatar Jul 19 '17 09:07 ExTheSea

Fyi @william-santos-bwti @rashmihunt After a little bit of experimenting i have since found a way to work around the issue. While it's not that nice it seems to be enough to deep copy the input objects e.g. using JSON.parse(JSON.stringify(inputvar))

So the model methods inside server/models/.js would look like:

<ServiceBinding>.<someSoapMethod> = function(<inputObject>, callback) {
    <ServiceBinding>.<someSoapMethod>(JSON.parse(JSON.stringify(<inputObject>)), function(err, response) {
      var result = response;
      callback(err, result);
    });
  };

For one of the two soap services I tested though i had to additionally drill down to the first layer of the parsed object as it was otherwise added twice. So i did: JSON.parse(JSON.stringify(<inputObject>))["top_Element"]

It seems like the problem arises because the inputObject somehow either gets additional properties while the object is parsed to xml or the additional properties are ignored when doing JSON.stringify().

Hope this helps anyone and looking forward to updates.

ExTheSea avatar Jul 19 '17 11:07 ExTheSea

Hi, @ExTheSea I decide to use node module "soap" and implement the requisitions for the webservices inside a middleware loopback and store in mongodb with the help of the "adapter" pattern to turn the soap model into my model. My models are loopback default, use the mongodb datasource and are published as REST services. This solved for me while the updates do not come.

ghost avatar Jul 20 '17 22:07 ghost

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Sep 18 '17 22:09 stale[bot]

Bump to prevent this issue from being closed. Eventhough several workarounds have been proposed, this issue has not yet been resolved and persists in latest version.

jmmeijer avatar Sep 22 '17 07:09 jmmeijer

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Nov 27 '17 07:11 stale[bot]

Hello @rashmihunt, are you still involved? If this is not the case, could this ticket be reassigned to someone else?

jmmeijer avatar Nov 27 '17 08:11 jmmeijer

I updated the latest dependencies, but still the same error.

my dependencies part of package.json is below.

  "dependencies": {
    "compression": "^1.7.1",
    "cors": "^2.8.4",
    "helmet": "^3.9.0",
    "loopback": "^3.17.1",
    "loopback-boot": "^2.27.0",
    "loopback-component-explorer": "^5.2.0",
    "loopback-connector-soap": "^4.0.1",
    "serve-favicon": "^2.4.5",
    "strong-error-handler": "^2.3.0"
  }

When I invoke an endpoint on loopback-explorer like /api/APISoapBinding/authenticate. The following error is shown.

image

The error message of backend console is.

未处理请求 POST /api/APISoapBinding/authenticate 的错误:Error: faultcode: soap:Client faultstring: Unmarshalling Error: 意外的元素 (uri:"", local:"__cachedRelations")
。所需元素为<{}password>,<{}user_at_domain>
    at XMLHandler.xmlToJson (D:\code\loopback\ynu-mail-ws\node_modules\strong-soap\src\parser\xmlHandler.js:637:23)
    at D:\code\loopback\ynu-mail-ws\node_modules\strong-soap\src\client.js:279:28
    at D:\code\loopback\ynu-mail-ws\node_modules\loopback-datasource-juggler\lib\observer.js:250:22
    at doNotify (D:\code\loopback\ynu-mail-ws\node_modules\loopback-datasource-juggler\lib\observer.js:155:49)
    at SOAPConnector.ObserverMixin._notifyBaseObservers (D:\code\loopback\ynu-mail-ws\node_modules\loopback-datasource-juggler\lib\observer.js:178:5)
    at SOAPConnector.ObserverMixin.notifyObserversOf (D:\code\loopback\ynu-mail-ws\node_modules\loopback-datasource-juggler\lib\observer.js:153:8)
    at cbForWork (D:\code\loopback\ynu-mail-ws\node_modules\loopback-datasource-juggler\lib\observer.js:240:14)
    at Request._callback (D:\code\loopback\ynu-mail-ws\node_modules\loopback-connector-soap\lib\http.js:98:9)
    at Request.self.callback (D:\code\loopback\ynu-mail-ws\node_modules\request\request.js:186:22)
    at Request.emit (events.js:159:13)
    at Request.<anonymous> (D:\code\loopback\ynu-mail-ws\node_modules\request\request.js:1163:10)
    at Request.emit (events.js:159:13)
    at IncomingMessage.<anonymous> (D:\code\loopback\ynu-mail-ws\node_modules\request\request.js:1085:12)
    at Object.onceWrapper (events.js:254:19)
    at IncomingMessage.emit (events.js:164:20)
    at endReadableNT (_stream_readable.js:1062:12)

liudonghua123 avatar Dec 19 '17 13:12 liudonghua123

After reading up on ES6 I believe this issue might have to do with an object prototype which is iterated over with a for ... in loop for conversion to SOAP causing the methods of this object to be included. In this case using for...of loop would be preferable. Hope this helps! Read more: Difference between for...of and for...in

jmmeijer avatar Jan 20 '18 21:01 jmmeijer

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Mar 21 '18 21:03 stale[bot]

Another bump in order to prevent this issue from being closed by stale bot.

jmmeijer avatar Mar 21 '18 21:03 jmmeijer

@kjdelisle @jannyHou @loay @b-admike @ssh24 @virkt25 @dhmlau @zbarbuto @nitro404 Hi, could you help us with this issue?

liudonghua123 avatar Mar 22 '18 01:03 liudonghua123

@raymondfeng , could you please help?

dhmlau avatar Mar 22 '18 02:03 dhmlau

Object instances of LoopBack models need to be converted to plain object using toJSON() or toObject() before passing to json2xml.

raymondfeng avatar Mar 22 '18 03:03 raymondfeng

I am still facing the same issue. Getting created unwanted <__cachedRelations/>\n <__data> for every child node, I tried both toJSON and toObject but still the same. Spending like 3 days now to figure this out, Like tried almost everything suggested here, Could you please help me? If you need more details I can provide.

FYI I have generated the models using CLI, lb soap [WSDL local file]

AccountCreationAccountCreationSoap.AccountCreation = function(AccountCreation, callback) {

  //console.log('AccountCreation',AccountCreation);
  
  
  
  //iterate(soapDataSource.connector);
  
  //fs.writeFile('./GeneratedAccountCreationPayload.WSDL',soapDataSource.connector);
  
  var AccountCreationReq = AccountCreation.toObject();
  
  AccountCreationAccountCreationSoap.AccountCreation(AccountCreationReq, function (err, response) {
	
    console.log('GeneratedAccountCreationPayload',soapDataSource.connector);
	 
    var result = response;
    callback(err, result);
  });

}

Thanks in advance

jersonjohn avatar Apr 17 '18 12:04 jersonjohn

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Jun 16 '18 12:06 stale[bot]

Bump in order to prevent issue from being closed without solution...

jmmeijer avatar Jun 16 '18 12:06 jmmeijer

This error has been open for a year? Is it likely to be fixed ever? trying to test on a simple SOAP service http://www.dneonline.com/calculator.asmx . it's posting junk to the service.

martyjt avatar Jun 17 '18 07:06 martyjt

@jmmeijer Reading through some of the ideas here , I don't know if this fixes the overall bug , but changing

https://github.com/strongloop/loopback-soap/blob/master/lib/codegen/model.ejs , line 30 to +

'(' + op.operationId + '.toObject(), function (err, response) {';

I just added the .toObject() seems to fix it for me so I can generate objects that will work.

martyjt avatar Jun 17 '18 22:06 martyjt

@martyjt You can send a pr to fix this issue.

liudonghua123 avatar Jun 19 '18 00:06 liudonghua123

@liudonghua123 Sorry , I don't have time to learn about test coverage / create tests for this. It is really just to help out. I am not familar with github.

martyjt avatar Jun 19 '18 02:06 martyjt

@martyjt As far as I can tell strong-soap is being used instead of loopback-soap. However I did find similar code that is being used in the swagger API Explorer: https://github.com/strongloop/loopback-swagger/blob/master/lib/codegen/model.ejs

jmmeijer avatar Jun 19 '18 18:06 jmmeijer