apache
Ignite Thin client Index Query is not working with Ignite Server and client version : 2.16.0
When data available in cache fetching the data from cache using Index query throwing below error org.apache.ignite.client.ClientException: Ignite failed to process request [19]: com.cisco.cjp.cache.model.EntityValue (server status code [1])
Where there is no data exist in cache the response contain empty. not faced above mentioned issue.
Ignite Configuration
@Bean
Map<ResourceType, ClientCache<String, EntityValue>> **entityCaches**(IgniteClient client, MeterRegistry meterRegistry) {
Map<ResourceType, ClientCache<String, EntityValue>> result = new EnumMap<>(ResourceType.class);
Arrays.stream(ResourceType.values()).forEach(r -> {
ClientCacheConfiguration cacheCfg = new ClientCacheConfiguration()
.setName(r.resourceName)
.setCacheMode(CacheMode.PARTITIONED)
.setStatisticsEnabled(true)
.setBackups(numberOfBackupNodes)
.setPartitionLossPolicy(PartitionLossPolicy.IGNORE)
.setQueryEntities(createEntityValueQueryEntity());
ClientCache<String, EntityValue> cache = client.getOrCreateCache(cacheCfg);
result.put(r, cache);
Gauge.builder(
"cache_" + cache.getName() + "_size",
cache,
ClientCache::size
)
.register(meterRegistry);
});
return result;
}
private static QueryEntity **createEntityValueQueryEntity**() {
LinkedHashMap<String, String> fields = new LinkedHashMap<>();
fields.put("dtoJson", String.class.getName());
LinkedHashMap<String, String> alias = new LinkedHashMap<>();
fields.put("dtoJson", "dto_json");
return new QueryEntity(String.class,EntityValue.class)
.setAliases(alias)
.setValueFieldName("dtoJson")
.setFields(fields)
.addQueryField("dbId", String.class.getName(), null)
.setIndexes(Arrays.asList(new QueryIndex("dbId").setName("DB_ID_IDX")));
}
Entity Value
@Data
@RequiredArgsConstructor
public class EntityValue implements Serializable {
private static final long serialVersionUID = 2826796212018108592L;
private final String entityId;
@GridToStringInclude
private final String dtoJson;
private final String type;
private final String organizationId;
@GridToStringInclude
@QuerySqlField(orderedGroups = { @QuerySqlField.Group(name = "DB_ID_IDX", order = 0), })
private final String dbId;
private final String ciUserId;
private final String subType;
}
Index Query
IndexQuery<String, EntityValue> idxQry = new IndexQuery<String, EntityValue>(EntityValue.class);
List<Cache.Entry<String, EntityValue>> result = entityCache.query(idxQry).getAll();
System.out.println("Result size is ::"+result.size());
for (Entry<String, EntityValue> c : result) {
String dtoJson = c.getValue().getDtoJson();
System.out.println("Response is ::"+dtoJson);
}
Not finding a appropriate logs/reason why the index query not working with Thin client
Please assist if there are any configuration changes required.
Please provide full stack trace of the error. The text Ignite failed to process request seems to be coming from your code.
Thanks @ptupitsyn for quick response . Please find the stack trace .
o.a.i.i.client.thin.ClientServerError: Ignite failed to process request [19]: com.cisco.cjp.cache.model.EntityValue (server status code [1])
at o.a.i.i.client.thin.TcpClientChannel.processNextMessage(TcpClientChannel.java:565)
at o.a.i.i.client.thin.TcpClientChannel.onMessage(TcpClientChannel.java:251)
at o.a.i.i.c.t.i.g.GridNioClientConnection.onMessage(GridNioClientConnection.java:96)
at o.a.i.i.c.t.i.g.GridNioClientListener.onMessage(GridNioClientListener.java:56)\n\tat o.a.i.i.c.t.i.g.GridNioClientListener.onMessage(GridNioClientListener.java:30)\
at o.a.i.i.u.n.GridNioFilterChain$TailFilter.onMessageReceived(GridNioFilterChain.java:279)
at o.a.i.i.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)
at o.a.i.i.util.nio.GridNioCodecFilter.onMessageReceived(GridNioCodecFilter.java:116)
at o.a.i.i.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)
at o.a.i.i.u.nio.GridNioServer$HeadFilter.onMessageReceived(GridNioServer.java:3753)
at o.a.i.i.util.nio.GridNioFilterChain.onMessageReceived(GridNioFilterChain.java:175)
at o.a.i.i.u.n.GridNioServer$ByteBufferNioClientWorker.processRead(GridNioServer.java:1211)
at o.a.i.i.u.n.GridNioServer$AbstractNioClientWorker.processSelectedKeysOptimized(GridNioServer.java:2527)
at o.a.i.i.u.n.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2282)
at o.a.i.i.u.n.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1911)
at o.a.i.internal.util.worker.GridWorker.run(GridWorker.java:125)\n\t... 1 common frames omitted
Wrapped by: org.apache.ignite.client.ClientException: Ignite failed to process request [19]: com.cisco.cjp.cache.model.EntityValue (server status code [1])\n\tat o.a.i.i.client.thin.TcpClientChannel.convertException(TcpClientChannel.java:475)
at o.a.i.i.client.thin.TcpClientChannel.receive(TcpClientChannel.java:421)
at o.a.i.i.client.thin.TcpClientChannel.service(TcpClientChannel.java:311)
at o.a.i.i.client.thin.ReliableChannel.lambda$service$0(ReliableChannel.ja...
I cross checked the code the text failed to process request is coming as response from the ignite server.
Root cause is still not clear. We should either find the problem in the server logs, or enable sendServerExceptionStackTraceToClient in the server config to see the cause on the client.
https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/ThinClientConfiguration.html#sendServerExceptionStackTraceToClient--
I am using ClientConfiguration to initialise ignite, not find appropriate method to set sendServerExceptionStackTraceToClient method.
public static ClientConfiguration getServerConfiguration(String[] clusterNodes, int connectTimeout, int retryLimit) { var igniteCfg = new ClientConfiguration(); igniteCfg.setPartitionAwarenessEnabled(true); igniteCfg.setRetryLimit(retryLimit); igniteCfg.setTimeout(connectTimeout); igniteCfg.setTcpNoDelay(true); igniteCfg.setAddressesFinder(() -> clusterNodes); return igniteCfg; } Can you please provide an example to set the sendServerExceptionStackTraceToClient to ClientConfiguration?
@timoninmaxim @yurinaryshkin @ptupitsyn I am running ignite server in docker container not able to find any logs to see what is the issue. Is configuration for index query is valid ? Any suggestions to proceeded further I am blocked here. Please assist.
sendServerExceptionStackTraceToClient should be set on the server, I've linked the javadoc above. Let's figure out the root cause first.
thanks a lot @ptupitsyn , Please find the complete error logs from server
org.apache.ignite.client.ClientException: Ignite failed to process request [768]: com.cisco.cjp.cache.model.EntityValue class org.apache.ignite.binary.BinaryInvalidTypeException: com.cisco.cjp.cache.model.EntityValue at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:741) at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1772) at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1731) at org.apache.ignite.internal.binary.BinaryObjectImpl.deserializeValue(BinaryObjectImpl.java:866) at org.apache.ignite.internal.binary.BinaryObjectImpl.value(BinaryObjectImpl.java:198) at org.apache.ignite.internal.processors.cache.CacheObjectUtils.unwrapBinary(CacheObjectUtils.java:199) at org.apache.ignite.internal.processors.cache.CacheObjectUtils.unwrapBinaryIfNeeded(CacheObjectUtils.java:73) at org.apache.ignite.internal.processors.cache.CacheObjectContext.unwrapBinaryIfNeeded(CacheObjectContext.java:138) at org.apache.ignite.internal.processors.cache.GridCacheContext.unwrapBinaryIfNeeded(GridCacheContext.java:1767) at org.apache.ignite.internal.processors.cache.query.GridCacheQueryFutureAdapter.unwrapIfNeeded(GridCacheQueryFutureAdapter.java:353) at org.apache.ignite.internal.processors.cache.query.GridCacheQueryFutureAdapter.next(GridCacheQueryFutureAdapter.java:153) at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl$5.onHasNext(IgniteCacheProxyImpl.java:594) at org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:56) at org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45) at org.apache.ignite.internal.processors.cache.AutoClosableCursorIterator.hasNext(AutoClosableCursorIterator.java:49) at org.apache.ignite.internal.processors.platform.client.cache.ClientCacheQueryCursor.writePage(ClientCacheQueryCursor.java:76) at org.apache.ignite.internal.processors.platform.client.cache.ClientCacheQueryResponse.encode(ClientCacheQueryResponse.java:51) at org.apache.ignite.internal.processors.platform.client.ClientMessageParser.encode(ClientMessageParser.java:723) at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:222) at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:57) at org.apache.ignite.internal.util.nio.GridNioFilterChain$TailFilter.onMessageReceived(GridNioFilterChain.java:279) at org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109) at org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.body(GridNioAsyncNotifyFilter.java:97) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125) at org.apache.ignite.internal.util.worker.GridWorkerPool$1.run(GridWorkerPool.java:70) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: java.lang.ClassNotFoundException: com.cisco.cjp.cache.model.EntityValue at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:9373) at org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:9311) at org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:384) at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:717) ... 27 more (server status code [1])
Please let me know what I am missing in the configuration.
Looks like a bug in thin client index query - it tries to deserialize data on the server side, which is not only inefficient, but also can fail like this because client classes are not available.
We should always use rawCache() in ClientCacheIndexQueryRequest: https://github.com/apache/ignite/blob/a225705fc5f487c2170dc584913f769f1b80f34b/modules/core/src/main/java/org/apache/ignite/internal/processors/platform/client/cache/ClientCacheIndexQueryRequest.java#L158
@timoninmaxim @alex-plekhanov could you please have a look? See my previous comment for details.
Thanks @ptupitsyn @timoninmaxim , One question with the index query will it possible to fetch only required fields ? example from the EntityValue pojo I want to fetch only dtoJson, will it possible ? if possible please point to example.
@ptupitsyn , Do we need to define the index query entity before the cache creation ? what if the cache is already available will the index query take into effect(Tested looks like not taking into effect) do we need to stop and start the ignite server in order to take index query take into effect ? Is there any possibility to take index query entity take effort without restarting the ignite server ?
You don't have to restart the server, just make sure indexes are set up correctly for the cache. See IEP for more details https://cwiki.apache.org/confluence/display/IGNITE/IEP-71%3A+Public+API+for+secondary+index+search
@Manojred1236 sorry for the delay. This is a bug and some other users also reported it IGNITE-20943. I have prepared a patch that fixes this #11453. It will be part of release 2.17.
Thanks for reporting that.