PAT error with caching

[pablocarle]

Describe the bug

Instance of Zowe started in standalone mode, non-HA.

I could create the PAT without problem even without having components.gateway.apiml.security.personalAccessToken.enabled as true, when calling the validate I'm getting a HTTP 500 error with the following message:

{
    "messages": [
        {
            "messageType": "ERROR",
            "messageNumber": "ZWEAM701E",
            "messageContent": "The request to the URL '/gateway/auth/access-token/validate' has failed: NestedServletException: Request processing failed; nested exception is org.zowe.apiml.gateway.cache.CachingServiceClientException: Unable to read all key-value maps from cache list, caused by: 400 : "{"messages":[{"messageType":"ERROR","messageNumber":"ZWECS136E","messageContent":"The storage of list items is supported only on Infinispan.","messageAction":"Switch to Infinispan to be able to use this API.","messageReason":"This caching storage method doesn't support this API.","messageKey":"org.zowe.apiml.cache.incompatibleStorageMethod"}]}" caused by: HttpClientErrorException.BadRequest: 400 : "{"messages":[{"messageType":"ERROR","messageNumber":"ZWECS136E","messageContent":"The storage of list items is supported only on Infinispan.","messageAction":"Switch to Infinispan to be able to use this API.","messageReason":"This caching storage method doesn't support this API.","messageKey":"org.zowe.apiml.cache.incompatibleStorageMethod"}]}"",
            "messageAction": "Refer to specific exception details for troubleshooting. Create an issue with this message.",
            "messageReason": "The request failed because of an internal error.",
            "messageKey": "org.zowe.apiml.common.internalRequestError"
        }
    ]
}

Caching service is not stated as a requirement for PAT to work.

Steps to Reproduce

1. Instance without components.gateway.apiml.security.personalAccessToken.enabled specified in zowe.yaml
2. Start Zowe
3. Create PAT:

curl --location 'https://zowe:portgateway/api/v1/auth/access-token/generate' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic xxxxxx==' \
--data '{
    "validity": 90,
    "scopes": ["serviceId"]
}'

(Without PAT enabled should it create it?) 4. Validate:

curl --location 'https://zowe:port/gateway/api/v1/auth/access-token/validate' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic xxxxxxx==' \
--data '{
    "token": "<your PAT>",
    "serviceId": "serviceId"
}'

Expected behavior

• Probably PAT should not be possible to create without enabling the feature first.
• Specify if caching service is a prerequisite in documentation: https://docs.zowe.org/stable/user-guide/api-mediation/api-gateway-configuration#personal-access-token

Details

• Version and build number: 2.10
• Test environment: z/OS

[EvaJavornicka]

Catching service is required for personal access token feature.

[onlyati]

We are on mainframe, I setup cache method with VSAM, but the error above appears. Does it mean that using PAT is not possible with any other type of cache than Infinispan? Not even with VSAM? In the document I did not found any requirements regarding this, can anybody explain it a bit? Thank you in advance!

[pablocarle]

Hi @onlyati, I verified and you are correct, we need to update the documentation to reflect this requirement. PATs can only be used with infinispan set as the type of storage for the caching service.

[pablocarle]

Created a new issue in docs-site https://github.com/zowe/docs-site/issues/3707

[onlyati]

But what is the limitation to use it with VSAM cache? PAT sounds a useful and cool feature but lock it behind something non-mainframe, to be honest, sounds a bit of disappointment. Is there no plan to use it with VSAM cache? I mean what makes inifnispan so special that no other option (VSAM, redis) is not able to perform the cache for PAT?

[balhar-jakub]

@pablocarle @achmelo What would it take to use Caching Service in general, instead of depending on specific backed to caching service?

[pablocarle]

I believe the limitation is that there are a few methods used in PAT creation and validation that are only implemented by the infinispan storage mode, so for example VSAM storage will fail: https://github.com/zowe/api-layer/blob/f5ef28460cac6a9d5a1fc478cef9932789f7064d/caching-service/src/main/java/org/zowe/apiml/caching/service/vsam/VsamStorage.java#L99 I also checked the one for redis and it's the same.

So in order to ignore the backed storage we need to implement these methods in the other ones.

[onlyati]

@pablocarle Thank you for the quick check! Can you see any chance that it could be implemented or put this into somewhere on the roadmap? Currently PAT seems a very good and useful feature to us (because our MFA password is frequently changing). PAT would be useful if we could use it without involve any non-mainframe product.

[pablocarle]

Hi @onlyati, Could you clarify what do you mean by non-mainframe? The infinispan mode in caching service is a Java-based solution but it still runs on z/OS.

[onlyati]

Hi @pablocarle ,

I was snot aware of it. On its page it was not stated that it is tested against z/OS (and IBM Semeru Runtime) as well. Let me explain a bit detailed my previous comment. For now, I am on sandbox system with Zowe, try to work out some use case/scenario for this and I want to make a similar environment on sandbox as I would implement it on production.

For infinispan we need to install a software which has to maintain regularly. It may simple for a development environment, but for production it also need to setup some high availability feature. If it is installed on production we need some enterprise support for as well. I can see on infinipan website that if we need fully supported version, we should use Redhat Data Grid instead. And here we are again, red hat does not make product for mainframe, so at the end we would also need some Redhat server too that also has extra cost.

All of this hassle seems overkill and "not worthy to invest time" to have a key-value store that we already has in form of KSDS VSAM, that is working out-of-the-box. KSDS VSAM is there on every z/OS system (and Zowe at the end is a z/OS product) without any extra product/maintenance/cost.

Unless I misunderstand something (e.g.: infinispan is embedded into Zowe and no additional product needs to be setup), I would prefer the VSAM cache, it seems more universal on mainframe environments. If I misunderstand something, feel free to correct me and thank you in advance.

[balhar-jakub]

@onlyati Infinispan is embedded in Zowe Caching service and no additional product needs to be setup.

[onlyati]

Thanks @balhar-jakub! In this case I misunderstand the documentation: https://docs.zowe.org/stable/extend/extend-apiml/api-mediation-infinispan/#infinispan-configuration, I did believe that it is an external service outside of Zowe. Thank you for your answers and patient, tomorrow I will give it a shot! :-)

[balhar-jakub]

Thank you, let us know, if you run into any further questions.