Unable to Datapreprocess CAL dataset

[ohsai]

I have executed DataPreprocess.exe for CAL dataset as you said to prepare CAL dataset for LoLa.

(Cryptonet root directory)\bin\x64\Release> .\DataPreprocess.exe CAL 

Then I yielded System.IndexOutOfRangeException.

Unhandled Exception: System.IndexOutOfRangeException: Index out of array range.
   위치: DataPreprocess.GetCAL.Run(String[] args) 파일 C:\Users\ohsai\Desktop\Workspace\CryptoNets\DataPreprocess\GetCAL.cs:줄 0
   위치: DataPreprocess.DataPreprocess.Main(String[] args) 파일 C:\Users\ohsai\Desktop\Workspace\CryptoNets\DataPreprocess\DataPreprocess.cs:줄 18

So I thought additional arguments for the program were necessary, so I looked at the GetCAL.cs to find out that second argument should be .tsv file and third argument should be the folder which contains all the images of Caltech-101 dataset.

So I have downloaded Caltech-101 image dataset and placed it in (Cryptonet root directory)\bin\x64\Release folder as a folder name of "CALObjectCategories",

(Cryptonet root directory)\bin\x64\Release> ls .\CALObjectCategories\

    디렉터리: 
(Cryptonet root directory)\bin\x64\Release\CALObjectCategories


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----     2019-08-07   오전 5:31                accordion
d-----     2019-08-07   오전 5:31                airplanes
.
.
.

and executed following command.

(Cryptonet root directory)\bin\x64\Release>.\DataPreprocess.exe CAL ".\cal_train.tsv" ".\CALObjectCategories\"

But It failed to process all the images, printing

failure on line ant/image_0013.jpg	4

faliure message for all the files.

So I tried to find out what caused the exception, and printed stacktrace for all try catch syntaxes on GetCAL.cs file.

Then only this exception stack trace was printed.

 location: Microsoft.ML.Transforms.OnnxTransformer..ctor(IHostEnvironment env, Options options, Byte[] modelBytes)
   location: Microsoft.ML.Transforms.AlexNetExtension.AlexNet(DnnImageModelSelector dnnModelContext, IHostEnvironment env, String outputColumnName, String inputColumnName, String modelDir)
   location: Microsoft.ML.Transforms.AlexNetExtension.AlexNet(DnnImageModelSelector dnnModelContext, IHostEnvironment env, String outputColumnName, String inputColumnName)
  location: Microsoft.ML.Transforms.DnnImageFeaturizerEstimator..ctor(IHostEnvironment env, String outputColumnName, Func`2 modelFactory, String inputColumnName)
   location: DataPreprocess.GetCAL._Run(String imagesCatalog, String imagesFolder) 파일 C:\Users\ohsai\Desktop\Workspace\CryptoNets\DataPreprocess\GetCAL.cs:line 93
   location: DataPreprocess.GetCAL._Run_failover(String line, String imagesFolder) 파일 C:\Users\ohsai\Desktop\Workspace\CryptoNets\DataPreprocess\GetCAL.cs:line58

So, my question is,

1. Is this the right way to fetch and preprocess CAL dataset? You said to fetch CAL dataset first by following the instruction after the first .\DataPreprocess.exe CAL, but it did not gave the instruction at all.
2. If it is, how should I deal with this error? From the exception stacktrace, I think the error comes from Microsoft.ML.Transforms module.

[rang-microsoft]

Sorry you encountered this problem and thanks for reporting. You are absolutely right that when you type .\DataPreprocess.exe CAL it should have responded with instructions of use. I will fix that shortly.

The exception you are seeing might be due to changes in the API of ML.NET library that we use to process the images. We are looking into it and I will notify once we pushed the solution.

[ranigb]

We are waiting for a bug fix from ML.NET that should solve this issue.

[HiKelly]

Hello, I have the same problems with you now. Are you solve the problems?