TensorFlow.NET icon indicating copy to clipboard operation
TensorFlow.NET copied to clipboard

Keras.Tensorflow Dense KernelRegularizer & BiasRegularizer wrong Type and missing L1 regularizer

Open JeroenMBooij opened this issue 3 years ago • 1 comments

The Dense Layer accepts a DenseArgs object with a KernelRegularizer & BiasRegularizer property. However they are of type IInitializer, whereas keras.regularizers.l2 is of type IRegularizer. Furthermore keras.regularizers.l1 doesn't exist in .NET, but I do need it for my model.

JeroenMBooij avatar Oct 19 '22 05:10 JeroenMBooij

The error in DenseArgs object seems to have been patched. However the code creating the dense layer seems to not take in account anywhere this regularizer (see Tensorflow.Keras.Layers.Dense) :

 public override void build(KerasShapesWrapper input_shape)
      {
          _buildInputShape = input_shape;
          Debug.Assert(input_shape.Shapes.Length <= 1);
          Shape shape = input_shape.ToSingleShape();
          long num = shape.dims.Last();
          Dictionary<int, int> dictionary = new Dictionary<int, int>();
          dictionary[-1] = (int)num;
          int? min_ndim = 2;
          Dictionary<int, int> axes = dictionary;
          inputSpec = new InputSpec(TF_DataType.DtInvalid, null, min_ndim, null, axes);

          ///////////// Problematic line : KernelRegularizer is not set ! //////
          kernel = add_weight("kernel", new Shape(num, args.Units), initializer: args.KernelInitializer, dtype: base.DType); 
          ///////////////////////////////////////////////////////////////////////////////

          if (args.UseBias)
          {
              bias = add_weight("bias", new Shape(args.Units), initializer: args.BiasInitializer, dtype: base.DType);
          }
          built = true;
      }

Am I wrong ?

AdrienDeverin avatar Feb 28 '24 12:02 AdrienDeverin