Handle ABI V2 nested array case

[rach-id]

As discussed by @874341642 in https://github.com/web3j/web3j/issues/1321#issuecomment-1153054859, the following ABI:

{
  "outputs": [
    {
      "components": [
        {
          "internalType": "uint256[]",
          "name": "blocks",
          "type": "uint256[]"
        }
      ],
      "internalType": "struct Mining.PlayerWork",
      "name": "",
      "type": "tuple"
    }
  ]
}

Returns the following exception:

public PlayerWork(DynamicArray< Uint256 > blocks) {
super(blocks);
}
Error info
java.lang.UnsupportedOperationException: Array types must be wrapped in a TypeReference

[rach-id]

The first step to solve this would be to add this ABI definition to the AbiV2TestFixture.java file, such as: https://github.com/web3j/web3j/blob/a9cc039ad3da656049e8b7bc004656ba1113b40d/abi/src/test/java/org/web3j/abi/AbiV2TestFixture.java#L183-L228

This would add the above to the test cases we can work with. Then, add type tests under: web3j/abi/src/test/java/org/web3j/abi. You can be inspired by this PR: https://github.com/web3j/web3j/pull/1321.

After finishing with this, and having encoding decoding tests working. Move on to the codegen module. And, following the same above way, add the tests and see where the problem is happening.

This PR should be your reference for changes https://github.com/web3j/web3j/pull/1321 as it is doing almost the same thing to implement support for struct arrays support.

[874341642]

{
	"inputs": [{
		"internalType": "uint256",
		"name": "tokenId",
		"type": "uint256"
	}],
	"name": "getPlayerWork",
	"outputs": [{
		"components": [{
			"internalType": "address",
			"name": "workType",
			"type": "address"
		}, {
			"internalType": "address",
			"name": "owner",
			"type": "address"
		}, {
			"internalType": "uint256[]",
			"name": "blocks",
			"type": "uint256[]"
		}],
		"internalType": "struct Mining.PlayerWork",
		"name": "",
		"type": "tuple"
	}],
	"stateMutability": "view",
	"type": "function"
}

I still don't understand what you mean, how to write this output parameter?

[rach-id]

If I understand right, you will write your own Foo, call it something I don't know, that has the same ABI definition as the one above and test against it.

[874341642]

If I understand right, you will write your own Foo, call it something I don't know, that has the same ABI definition as the one above and test against it.

  public static class PlayerWork extends DynamicStruct {
        public String workType;

        public String careerAddr;

        public BigInteger startTime;

        public BigInteger endTime;

        public String owner;

        public List<BigInteger> blocks;

        public BigInteger beginTime;

        public PlayerWork(String workType, String careerAddr, BigInteger startTime, BigInteger endTime, String owner, List<BigInteger> blocks, BigInteger beginTime) {
            super(new org.web3j.abi.datatypes.Address(160, workType), 
                    new org.web3j.abi.datatypes.Address(160, careerAddr), 
                    new org.web3j.abi.datatypes.generated.Uint256(startTime), 
                    new org.web3j.abi.datatypes.generated.Uint256(endTime), 
                    new org.web3j.abi.datatypes.Address(160, owner), 
                    new org.web3j.abi.datatypes.DynamicArray<org.web3j.abi.datatypes.generated.Uint256>(
                            org.web3j.abi.datatypes.generated.Uint256.class,
                            org.web3j.abi.Utils.typeMap(blocks, org.web3j.abi.datatypes.generated.Uint256.class)), 
                    new org.web3j.abi.datatypes.generated.Uint256(beginTime));
            this.workType = workType;
            this.careerAddr = careerAddr;
            this.startTime = startTime;
            this.endTime = endTime;
            this.owner = owner;
            this.blocks = blocks;
            this.beginTime = beginTime;
        }

        public PlayerWork(Address workType, Address careerAddr, Uint256 startTime, Uint256 endTime, Address owner, DynamicArray<Uint256> blocks, Uint256 beginTime) {
            super(workType, careerAddr, startTime, endTime, owner, blocks, beginTime);
            this.workType = workType.getValue();
            this.careerAddr = careerAddr.getValue();
            this.startTime = startTime.getValue();
            this.endTime = endTime.getValue();
            this.owner = owner.getValue();
            this.blocks = blocks.getValue().stream().map(v -> v.getValue()).collect(Collectors.toList());
            this.beginTime = beginTime.getValue();
        }
    }

This will throw an exception

[rach-id]

How did you generate this?

[874341642]

How did you generate this?

 public static void generateClass(String abiFile,String generateFile){
	        String[] args = Arrays.asList(
	                "-a",abiFile,	             
	                "-p","cn.contract",
	                "-o",generateFile
	        ).toArray(new String[0]);
	        Stream.of(args).forEach(System.out::println);
	        SolidityFunctionWrapperGenerator.main(args);
	    }

[874341642]

How did you generate this?

org.web3j codegen 4.9.2

[rach-id]

And what do you run to get the exception?

[874341642]

[rach-id]

Thanks, but what are you running here?

[874341642]

[rach-id]

Can you provide a command or some code that throw this exception?

[rach-id]

Also, you can use permalink to share lines of code. And, code blocks for code blocks for a more readable discussion.

[874341642]

Also, you can use permalink to share lines of code. And, code blocks for code blocks for a more readable discussion.

As long as the tuple contains an array of output parameters, an exception will occur.

[874341642]

[{
	"inputs": [{
		"internalType": "uint256",
		"name": "tokenId",
		"type": "uint256"
	}],
	"name": "getPlayerWork",
	"outputs": [{
		"components": [{
			"internalType": "address",
			"name": "workType",
			"type": "address"
		}, {
			"internalType": "address",
			"name": "careerAddr",
			"type": "address"
		}, {
			"internalType": "uint256",
			"name": "startTime",
			"type": "uint256"
		}, {
			"internalType": "uint256",
			"name": "endTime",
			"type": "uint256"
		}, {
			"internalType": "address",
			"name": "owner",
			"type": "address"
		}, {
			"internalType": "uint256[]",
			"name": "blocks",
			"type": "uint256[]"
		}, {
			"internalType": "uint256",
			"name": "beginTime",
			"type": "uint256"
		}],
		"internalType": "struct Mining.PlayerWork",
		"name": "",
		"type": "tuple"
	}],
	"stateMutability": "view",
	"type": "function"
}]

This is the abi file and the generated java file is wrong.

[874341642]

package cn.contract;

import java.math.BigInteger;
import java.util.Arrays;
import java.util.List;
import java.util.stream.Collectors;
import org.web3j.abi.TypeReference;
import org.web3j.abi.datatypes.Address;
import org.web3j.abi.datatypes.DynamicArray;
import org.web3j.abi.datatypes.DynamicStruct;
import org.web3j.abi.datatypes.Function;
import org.web3j.abi.datatypes.Type;
import org.web3j.abi.datatypes.generated.Uint256;
import org.web3j.crypto.Credentials;
import org.web3j.protocol.Web3j;
import org.web3j.protocol.core.RemoteFunctionCall;
import org.web3j.tx.Contract;
import org.web3j.tx.TransactionManager;
import org.web3j.tx.gas.ContractGasProvider;

/**
 * <p>Auto generated code.
 * <p><strong>Do not modify!</strong>
 * <p>Please use the <a href="https://docs.web3j.io/command_line.html">web3j command line tools</a>,
 * or the org.web3j.codegen.SolidityFunctionWrapperGenerator in the 
 * <a href="https://github.com/web3j/web3j/tree/master/codegen">codegen module</a> to update.
 *
 * <p>Generated with web3j version 4.9.2.
 */
@SuppressWarnings("rawtypes")
public class Market extends Contract {
    public static final String BINARY = "Bin file was not provided";

    public static final String FUNC_GETPLAYERWORK = "getPlayerWork";

    @Deprecated
    protected Market(String contractAddress, Web3j web3j, Credentials credentials, BigInteger gasPrice, BigInteger gasLimit) {
        super(BINARY, contractAddress, web3j, credentials, gasPrice, gasLimit);
    }

    protected Market(String contractAddress, Web3j web3j, Credentials credentials, ContractGasProvider contractGasProvider) {
        super(BINARY, contractAddress, web3j, credentials, contractGasProvider);
    }

    @Deprecated
    protected Market(String contractAddress, Web3j web3j, TransactionManager transactionManager, BigInteger gasPrice, BigInteger gasLimit) {
        super(BINARY, contractAddress, web3j, transactionManager, gasPrice, gasLimit);
    }

    protected Market(String contractAddress, Web3j web3j, TransactionManager transactionManager, ContractGasProvider contractGasProvider) {
        super(BINARY, contractAddress, web3j, transactionManager, contractGasProvider);
    }

    public RemoteFunctionCall<PlayerWork> getPlayerWork(BigInteger tokenId) {
        final Function function = new Function(FUNC_GETPLAYERWORK, 
                Arrays.<Type>asList(new org.web3j.abi.datatypes.generated.Uint256(tokenId)), 
                Arrays.<TypeReference<?>>asList(new TypeReference<PlayerWork>() {}));
        return executeRemoteCallSingleValueReturn(function, PlayerWork.class);
    }

    @Deprecated
    public static Market load(String contractAddress, Web3j web3j, Credentials credentials, BigInteger gasPrice, BigInteger gasLimit) {
        return new Market(contractAddress, web3j, credentials, gasPrice, gasLimit);
    }

    @Deprecated
    public static Market load(String contractAddress, Web3j web3j, TransactionManager transactionManager, BigInteger gasPrice, BigInteger gasLimit) {
        return new Market(contractAddress, web3j, transactionManager, gasPrice, gasLimit);
    }

    public static Market load(String contractAddress, Web3j web3j, Credentials credentials, ContractGasProvider contractGasProvider) {
        return new Market(contractAddress, web3j, credentials, contractGasProvider);
    }

    public static Market load(String contractAddress, Web3j web3j, TransactionManager transactionManager, ContractGasProvider contractGasProvider) {
        return new Market(contractAddress, web3j, transactionManager, contractGasProvider);
    }

    public static class PlayerWork extends DynamicStruct {
        public String workType;

        public String careerAddr;

        public BigInteger startTime;

        public BigInteger endTime;

        public String owner;

        public List<BigInteger> blocks;

        public BigInteger beginTime;

        public PlayerWork(String workType, String careerAddr, BigInteger startTime, BigInteger endTime, String owner, List<BigInteger> blocks, BigInteger beginTime) {
            super(new org.web3j.abi.datatypes.Address(160, workType), 
                    new org.web3j.abi.datatypes.Address(160, careerAddr), 
                    new org.web3j.abi.datatypes.generated.Uint256(startTime), 
                    new org.web3j.abi.datatypes.generated.Uint256(endTime), 
                    new org.web3j.abi.datatypes.Address(160, owner), 
                    new org.web3j.abi.datatypes.DynamicArray<org.web3j.abi.datatypes.generated.Uint256>(
                            org.web3j.abi.datatypes.generated.Uint256.class,
                            org.web3j.abi.Utils.typeMap(blocks, org.web3j.abi.datatypes.generated.Uint256.class)), 
                    new org.web3j.abi.datatypes.generated.Uint256(beginTime));
            this.workType = workType;
            this.careerAddr = careerAddr;
            this.startTime = startTime;
            this.endTime = endTime;
            this.owner = owner;
            this.blocks = blocks;
            this.beginTime = beginTime;
        }

        public PlayerWork(Address workType, Address careerAddr, Uint256 startTime, Uint256 endTime, Address owner, DynamicArray<Uint256> blocks, Uint256 beginTime) {
            super(workType, careerAddr, startTime, endTime, owner, blocks, beginTime);
            this.workType = workType.getValue();
            this.careerAddr = careerAddr.getValue();
            this.startTime = startTime.getValue();
            this.endTime = endTime.getValue();
            this.owner = owner.getValue();
            this.blocks = blocks.getValue().stream().map(v -> v.getValue()).collect(Collectors.toList());
            this.beginTime = beginTime.getValue();
        }
    }
}

[rach-id]

I agree, but we need to find a way to reproduce this to add tests for it and know what we need to implement. Also, write a more refined issue following the templates.

Then, we can work on a fix and decide whether this is a bug or a missing feature.

[874341642]

I agree, but we need to find a way to reproduce this to add tests for it and know what we need to implement. Also, write a more refined issue following the templates.

Then, we can work on a fix and decide whether this is a bug or a missing feature.

Can you guys generate a java file based on the abi file I provided?

[rach-id]

Can you try generating it, please? And let us know the exact steps to see that exception?

[874341642]

public BigInteger getPlayerWork(BigInteger tokenId) { String functionName = "getPlayerWork"; List<Type> inputParameters = Arrays.asList(new Uint256(tokenId)); List<TypeReference<?>> outputParameters = Arrays.asList(new TypeReference< PlayerWork >() { }); Function function = new Function(functionName, inputParameters, outputParameters); String data = FunctionEncoder.encode(function); Transaction transaction = Transaction.createEthCallTransaction(BaseConst.myAddress, BaseConst.miningv3, data); try { EthCall ethCall = web3jUtil.getWeb3j().ethCall(transaction, DefaultBlockParameterName.LATEST).send(); List<Type> results = FunctionReturnDecoder.decode(ethCall.getValue(), function.getOutputParameters()); if (results != null && results.size() > 0) { List<Type> result = (List<Type>) results.get(0).getValue(); if (result != null && result.size() == 7) { return new BigInteger(result.get(6).getValue().toString()); } } } catch (Exception ex) { ex.printStackTrace(); } return null; }

[874341642]

public static class PlayerWork extends DynamicStruct {
    public String workType;

    public String careerAddr;

    public BigInteger startTime;

    public BigInteger endTime;

    public String owner;

    public List<BigInteger> blocks;

    public BigInteger beginTime;

    public PlayerWork(String workType, String careerAddr, BigInteger startTime, BigInteger endTime, String owner, List<BigInteger> blocks, BigInteger beginTime) {
        super(new org.web3j.abi.datatypes.Address(160, workType), 
                new org.web3j.abi.datatypes.Address(160, careerAddr), 
                new org.web3j.abi.datatypes.generated.Uint256(startTime), 
                new org.web3j.abi.datatypes.generated.Uint256(endTime), 
                new org.web3j.abi.datatypes.Address(160, owner), 
                new org.web3j.abi.datatypes.DynamicArray<org.web3j.abi.datatypes.generated.Uint256>(
                        org.web3j.abi.datatypes.generated.Uint256.class,
                        org.web3j.abi.Utils.typeMap(blocks, org.web3j.abi.datatypes.generated.Uint256.class)), 
                new org.web3j.abi.datatypes.generated.Uint256(beginTime));
        this.workType = workType;
        this.careerAddr = careerAddr;
        this.startTime = startTime;
        this.endTime = endTime;
        this.owner = owner;
        this.blocks = blocks;
        this.beginTime = beginTime;
    }

    public PlayerWork(Address workType, Address careerAddr, Uint256 startTime, Uint256 endTime, Address owner, DynamicArray<Uint256> blocks, Uint256 beginTime) {
        super(workType, careerAddr, startTime, endTime, owner, blocks, beginTime);
        this.workType = workType.getValue();
        this.careerAddr = careerAddr.getValue();
        this.startTime = startTime.getValue();
        this.endTime = endTime.getValue();
        this.owner = owner.getValue();
        this.blocks = blocks.getValue().stream().map(v -> v.getValue()).collect(Collectors.toList());
        this.beginTime = beginTime.getValue();
    }
}

}

[874341642]

Write these two classes to run.

[huangshun666]

使用 web3j1.4.1 生成java文件，支持Tuple类型

[YangXianqi]

When the bug can be fixed?

[metaglobal]

[
    {
        "inputs":[
            {
                "internalType":"address",
                "name":"account",
                "type":"address"
            }
        ],
        "name":"getStakingData",
        "outputs":[
            {
                "components":[
                    {
                        "internalType":"uint256",
                        "name":"totalStaked",
                        "type":"uint256"
                    },
                    {
                        "internalType":"uint256",
                        "name":"veTokenTotalSupply",
                        "type":"uint256"
                    },
                    {
                        "internalType":"uint256",
                        "name":"accountVeTokenBalance",
                        "type":"uint256"
                    },
                    {
                        "internalType":"uint256",
                        "name":"accountWithdrawableRewards",
                        "type":"uint256"
                    },
                    {
                        "internalType":"uint256",
                        "name":"accountWithdrawnRewards",
                        "type":"uint256"
                    },
                    {
                        "internalType":"uint256",
                        "name":"accountDepositTokenBalance",
                        "type":"uint256"
                    },
                    {
                        "internalType":"uint256",
                        "name":"accountDepositTokenAllowance",
                        "type":"uint256"
                    },
                    {
                        "components":[
                            {
                                "internalType":"uint256",
                                "name":"amount",
                                "type":"uint256"
                            },
                            {
                                "internalType":"uint32",
                                "name":"lockedAt",
                                "type":"uint32"
                            },
                            {
                                "internalType":"uint32",
                                "name":"lockDuration",
                                "type":"uint32"
                            }
                        ],
                        "internalType":"struct SharesTimeLock.Lock[]",
                        "name":"accountLocks",
                        "type":"tuple[]"
                    }
                ],
                "internalType":"struct SharesTimeLock.StakingData",
                "name":"data",
                "type":"tuple"
            }
        ],
        "stateMutability":"view",
        "type":"function"
    }
]

This complex structure also cannot be parsed

[wangweiai]

When the bug can be fixed?

[georgelombardi97]

Is there any fix for this? Hotfix or dirty way around it?

[NickSneo]

@874341642 @metaglobal Can you provide the Solidity code for the respective abi provided? We are looking into this issue

[NickSneo]

{
	"inputs": [{
		"internalType": "uint256",
		"name": "tokenId",
		"type": "uint256"
	}],
	"name": "getPlayerWork",
	"outputs": [{
		"components": [{
			"internalType": "address",
			"name": "workType",
			"type": "address"
		}, {
			"internalType": "address",
			"name": "owner",
			"type": "address"
		}, {
			"internalType": "uint256[]",
			"name": "blocks",
			"type": "uint256[]"
		}],
		"internalType": "struct Mining.PlayerWork",
		"name": "",
		"type": "tuple"
	}],
	"stateMutability": "view",
	"type": "function"
}

I still don't understand what you mean, how to write this output parameter?

I tried to create a solidity code in respect to your provided abi -

pragma solidity ^0.8.0;

contract Random {
    struct PlayerWork {
        address workType;
        address owner;
        uint256[] blocks;
    }

    function getPlayerWork(uint256 tokenId) public view returns (PlayerWork[] memory) {
        PlayerWork[] memory works = new PlayerWork[](1);
        
        works[0].workType = address(0x123);
        works[0].owner = address(0x456);
        works[0].blocks = new uint256[](3);
        works[0].blocks[0] = 1;
        works[0].blocks[1] = 2;
        works[0].blocks[2] = 3;

        return works;
    }
}

Then using web3j only I compiled and generated java wrappers.

The abi generated (look at the difference, instead of tuple this one has tuple[]) -

{
    "inputs": [
        {
            "internalType": "uint256",
            "name": "tokenId",
            "type": "uint256"
        }
    ],
    "name": "getPlayerWork",
    "outputs": [
        {
            "components": [
                {
                    "internalType": "address",
                    "name": "workType",
                    "type": "address"
                },
                {
                    "internalType": "address",
                    "name": "owner",
                    "type": "address"
                },
                {
                    "internalType": "uint256[]",
                    "name": "blocks",
                    "type": "uint256[]"
                }
            ],
            "internalType": "struct Random.PlayerWork[]",
            "name": "",
            "type": "tuple[]"
        }
    ],
    "stateMutability": "view",
    "type": "function"
}

Seems it is working fine for me, can you please check?

[NickSneo]

Closing this ticket as everything seems working fine with latest web3j 4.10.0, if someone faces any issue kindly reopen so we can discuss.