yzhao062
autoencoder StandardScaler and sigmoid as output layer / inconsistency in values size
Hello pyod community,
according to the standard autoencoder settings the output layer has the sigmoid activation function whose values are within 0 and 1, but the input data are scaled with StandardScaler whose values can be higher than 1 and smaller than 0.
Why do we have this inconsistency?
thanks!
autoencoder is more like a type of neural architecture with many design choices. It is not necessary to follow the exact setting. That being said, feel free to modify the file to use MinMaxScalaer or other things as you wish :)
with many design choices.
yes of course as many other neural models. But in your standard settings, what is the exact idea behind StandardScaler and sigmoid output layer, where input and output have different value ranges?
and do arise any problems because of these settings?
I think your concern makes sense. Maybe we should change the sigmoid to relu or leakyrelu. The reason to not use minmax scalaer for input is it is sensitive to outliers in the data.
You could initialize the model by setting output_activation='relu' for now I guess.
The reason to not use minmax scalaer for input is it is sensitive to outliers in the data.
that makes sense too
thank you for your response
Hi Lads,
Both standard and minmax scalers are sensitive to outliers. Except for the Robust Scaler which is based on the qunatiles.
Look here.
are there any papers to this topic?
Are you talking to me? If so, please mention that next time. Regarding your question, look at Scikit-learn docs which are also mentioned in the link provided.
@John-Almardeny
I meant in general whether some papers exist about this topic (how to scale data for autoencoders in case of anomalies etc)
@John-Almardeny
I meant in general whether some papers exist about this topic (how to scale data for autoencoders in case of anomalies etc)
There might be, didn't check. Most of these can be deducted intuitively, for example: since outliers are extreme values, statistical measures such as mean get skewed/affected by them heavily. Ironically, in my master's in AI and ML, I used Auto Encoder for outliers detection and best results were after applying minmax scaler. That's because autoencoder is an ANN and requires inputs to be small, but somehow didn't get affected by the downsides of minmax scaler!