How can I concat two embeddings in sparse feature with DeepFM

[llfzllfz]

[zanshuxun]

concat_func

[llfzllfz]

concat_func

Can you say more details how to use in the DeepFM. In the docs, I can't find how to use it. The source with DeepFM in the next paragraph, and where should I modify.

def DeepFM(linear_feature_columns, dnn_feature_columns, fm_group=[DEFAULT_GROUP_NAME], dnn_hidden_units=(128, 128), l2_reg_linear=0.00001, l2_reg_embedding=0.00001, l2_reg_dnn=0, seed=1024, dnn_dropout=0, dnn_activation='relu', dnn_use_bn=False, task='binary'): """Instantiates the DeepFM Network architecture.

:param linear_feature_columns: An iterable containing all the features used by linear part of the model.
:param dnn_feature_columns: An iterable containing all the features used by deep part of the model.
:param fm_group: list, group_name of features that will be used to do feature interactions.
:param dnn_hidden_units: list,list of positive integer or empty list, the layer number and units in each layer of DNN
:param l2_reg_linear: float. L2 regularizer strength applied to linear part
:param l2_reg_embedding: float. L2 regularizer strength applied to embedding vector
:param l2_reg_dnn: float. L2 regularizer strength applied to DNN
:param seed: integer ,to use as random seed.
:param dnn_dropout: float in [0,1), the probability we will drop out a given DNN coordinate.
:param dnn_activation: Activation function to use in DNN
:param dnn_use_bn: bool. Whether use BatchNormalization before activation or not in DNN
:param task: str, ``"binary"`` for  binary logloss or  ``"regression"`` for regression loss
:return: A Keras model instance.
"""

features = build_input_features(
    linear_feature_columns + dnn_feature_columns)

inputs_list = list(features.values())

linear_logit = get_linear_logit(features, linear_feature_columns, seed=seed, prefix='linear',
                                l2_reg=l2_reg_linear)

group_embedding_dict, dense_value_list = input_from_feature_columns(features, dnn_feature_columns, l2_reg_embedding,
                                                                    seed, support_group=True)

fm_logit = add_func([FM()(concat_func(v, axis=1))
                     for k, v in group_embedding_dict.items() if k in fm_group])

dnn_input = combined_dnn_input(list(chain.from_iterable(
    group_embedding_dict.values())), dense_value_list)
dnn_output = DNN(dnn_hidden_units, dnn_activation, l2_reg_dnn, dnn_dropout, dnn_use_bn, seed=seed)(dnn_input)
dnn_logit = tf.keras.layers.Dense(
    1, use_bias=False, kernel_initializer=tf.keras.initializers.glorot_normal(seed=seed))(dnn_output)

final_logit = add_func([linear_logit, fm_logit, dnn_logit])

output = PredictionLayer(task)(final_logit)
model = tf.keras.models.Model(inputs=inputs_list, outputs=output)
return model

[zanshuxun]

For example, you can concatenate two embeddings in this way:

    group_embedding_dict, dense_value_list = input_from_feature_columns(features, dnn_feature_columns, l2_reg_embedding,
                                                                        seed, support_group=True)
    a = group_embedding_dict['default_group'][0]  # (?, 1, 4)
    b = group_embedding_dict['default_group'][1]  # (?, 1, 4)
    c = concat_func((a, b))  # (?, 1, 8)

And what will you do after concatenate these two embeddings?

[llfzllfz]

For example, you can concatenate two embeddings in this way:

    group_embedding_dict, dense_value_list = input_from_feature_columns(features, dnn_feature_columns, l2_reg_embedding,
                                                                        seed, support_group=True)
    a = group_embedding_dict['default_group'][0]  # (?, 1, 4)
    b = group_embedding_dict['default_group'][1]  # (?, 1, 4)
    c = concat_func((a, b))  # (?, 1, 8)

And what will you do after concatenate these two embeddings?

For example, I get a and b, now I concat them as c, I want to use c to replace a and b to join the train in the DeepFM, how I can do it. Such as:

group_embedding_dict['default_group'][0] = c

But the shape become(?, 1, 8) from (?, 1, 4), will it work fine in the train.

[zanshuxun]

There is no need to put c into group_embedding_dict. Concat all the ( ?, 1, 8) tensors and feed them into FM(), it will work.