Implementation of adversarial domain adaptation

[davidshumway]

This article mentions both adversarial learning and domain adaptation.

Chen, D., Zhou, R., Pan, Y., & Liu, F. (2022). A Simple Baseline for Adversarial Domain Adaptation-based Unsupervised Flood Forecasting. arXiv preprint arXiv:2206.08105.

There are two models, both 1-d CNN with three layers. The first model is three layers, 36 filters x kernel size 2. This is the encoder model. The second model is three layers, the first two being 36 filters x kernel size 2 and the final layer being 1 layer x kernel size 3. This is the "prediction head" model.

I'm not sure how these two models fit together. It makes sense to use sklearn and Sequential with Conv1D. But how does the prediction head model attach to the embedding model?

Is it possible to use the adapt library to implement these models?

[antoinedemathelin]

Hi @davidshumway,

Yes, if I understand well the paper, you can make something close using adapt, but not totally equivalent. The paper use a two step training as ADDA but a Wasserstein loss as WDGRL

Here is one exemple to use these algorithms with the architecture you want:

import numpy as np
import tensorflow as tf
from adapt.feature_based import WDGRL

# Make Fake data
Xs = np.random.randn(100, 6, 7)
ys = Xs.mean(-1)

Xt = np.random.randn(100, 6, 7)
yt = Xt.mean(-1)

# Encoder
encoder = tf.keras.Sequential()
encoder.add(tf.keras.layers.Conv1D(36, 2, activation="relu", padding="same",
                                   input_shape=(6, 7)))
encoder.add(tf.keras.layers.Conv1D(36, 2, activation="relu", padding="same"))
encoder.add(tf.keras.layers.Conv1D(36, 2, activation="relu", padding="same"))

# Prediction head
prediction_head = tf.keras.Sequential()
prediction_head.add(tf.keras.layers.Conv1D(36, 2, activation="relu", padding="same",
                                           input_shape=(6, 36)))
prediction_head.add(tf.keras.layers.Conv1D(36, 2, activation="relu", padding="same"))
prediction_head.add(tf.keras.layers.Conv1D(1, 3, padding="same"))
prediction_head.add(tf.keras.layers.Flatten())

# Discriminator WDGRL
disc_wdgrl = tf.keras.Sequential()
disc_wdgrl.add(tf.keras.layers.Conv1D(36, 2, activation="relu", padding="same",
                                           input_shape=(6, 36)))
disc_wdgrl.add(tf.keras.layers.Conv1D(36, 2, activation="relu", padding="same"))
disc_wdgrl.add(tf.keras.layers.Conv1D(1, 3, padding="same"))
disc_wdgrl.add(tf.keras.layers.Flatten())

# WDGRL
model = WDGRL(encoder=encoder,
              task=prediction_head,
              discriminator=disc_wdgrl,
              loss="mse",
              optimizer=tf.keras.optimizers.Adam(0.001),
              gamma=10,
              lambda_=1.,
              copy=True)

model.fit(Xs, ys, Xt, epochs=5, batch_size=32)

Here is the code for ADDA, you have to change a little bit the discriminator to make it like a classifier:

from adapt.feature_based import ADDA

# Discriminator ADDA
disc_adda = tf.keras.Sequential()
disc_adda.add(tf.keras.layers.Conv1D(36, 2, activation="relu", padding="same",
                                           input_shape=(6, 36)))
disc_adda.add(tf.keras.layers.Conv1D(36, 2, activation="relu", padding="same"))
disc_adda.add(tf.keras.layers.Conv1D(1, 3, padding="same"))
disc_adda.add(tf.keras.layers.Flatten())
disc_adda.add(tf.keras.layers.Dense(1, activation="sigmoid"))

# ADDA
model = ADDA(encoder=encoder,
             task=prediction_head,
              discriminator=disc_adda,
              loss="mse",
              optimizer=tf.keras.optimizers.Adam(0.001),
              pretrain=True,
              copy=True)

model.fit(Xs, ys, Xt, epochs=5, batch_size=32)