multivariate data on Tulog.ipynb

[richardcai88]

• Orion version: 0.3.2
• Python version:3.7
• Operating System: linux

Description

Hi I tried using a multivariate data ('multivariate/S-1') which has 25 signals on Tulog.ipynb

The Orion API at part 2 is able to fit the data. However at the Pipeline Training and Detection Section using primitives,

from model import hyperparameters from orion.primitives.tadgan import TadGAN

hyperparameters["epochs"] = 5 hyperparameters["input_shape"] = (100, 25) # based on the window size hyperparameters["optimizer"] = "keras.optimizers.Adam" hyperparameters["learning_rate"] = 0.0005 hyperparameters["latent_dim"] = 20 hyperparameters["batch_size"] = 64

tgan = TadGAN(**hyperparameters) tgan.fit(X)

1. I encountered the following error during fitting : ValueError: Error when checking input: expected input_6 to have shape (100, 1) but got array with shape (100, 25) Seems to the input shape at the critic_x

2. I noticed on the Colab version of Tulog.ipynb , it seems to use tgan.fit(X,X) instead of tgan.fit(X) from github

Thanks Richard

Paste the command(s) you ran and the output.
If there was a crash, please include the traceback here.

---

ValueError Traceback (most recent call last) in 12 13 tgan = TadGAN(**hyperparameters) ---> 14 tgan.fit(X,X)

/app/orion/primitives/tadgan.py in fit(self, X, y, **kwargs) 292 self._build_tadgan(**kwargs) 293 --> 294 self._fit(X, y) 295 self._fitted = True 296

/app/orion/primitives/tadgan.py in _fit(self, X, target) 262 z = np.random.normal(size=(self.batch_size, self.latent_dim, 1)) 263 epoch_cx_loss.append( --> 264 self.critic_x_model.train_on_batch([y, z], [valid, fake, delta])) 265 epoch_cz_loss.append( 266 self.critic_z_model.train_on_batch([x, z], [valid, fake, delta]))

/usr/local/lib/python3.6/dist-packages/keras/engine/training.py in train_on_batch(self, x, y, sample_weight, class_weight, reset_metrics) 1506 x, y, 1507 sample_weight=sample_weight, -> 1508 class_weight=class_weight) 1509 if self._uses_dynamic_learning_phase(): 1510 ins = x + y + sample_weights + [1]

/usr/local/lib/python3.6/dist-packages/keras/engine/training.py in _standardize_user_data(self, x, y, sample_weight, class_weight, check_array_lengths, batch_size) 577 feed_input_shapes, 578 check_batch_axis=False, # Don't enforce the batch size. --> 579 exception_prefix='input') 580 581 if y is not None:

/usr/local/lib/python3.6/dist-packages/keras/engine/training_utils.py in standardize_input_data(data, names, shapes, check_batch_axis, exception_prefix) 143 ': expected ' + names[i] + ' to have shape ' + 144 str(shape) + ' but got array with shape ' + --> 145 str(data_shape)) 146 return data 147

ValueError: Error when checking input: expected input_6 to have shape (100, 1) but got array with shape (100, 25)

[sarahmish]

Hi @richardcai88! Thanks for using Orion!

There is one extra step for multivariate TadGAN if you are running the tulog.ipynb, which is that you need to create a y variable that refers to the channel you want to reconstruct.

channel = 0 # index of channel to reconstruct
y = X[:, :, channel:channel+1] # slice it out of X

tgan.fit(X, y)

Let me know if you have any further questions!

[prithuls]

Hi, I am facing similar problem with a multivariate dataset of window size 100 and dimensions 25.

Here's my code:

channel = 0
Y = X[:, :, channel:channel+1]
## X.shape - (1225, 100, 25) 
## Y.shape - (1225, 100, 1)


from model import hyperparameters
from orion.primitives.tadgan import TadGAN

hyperparameters["epochs"] = 1
hyperparameters["input_shape"] = (100, 1) # based on the window size
hyperparameters["optimizer"] = "keras.optimizers.Adam"
hyperparameters["learning_rate"] = 0.0005
hyperparameters["latent_dim"] = 20
hyperparameters["batch_size"] = 64

tgan = TadGAN(**hyperparameters)
tgan.fit(X, Y)

I get the following error:

ValueError: Exception encountered when calling layer 'critic_x' (type Sequential).

Input 0 of layer "conv1d_24" is incompatible with the layer: expected axis -1 of input shape to have value 25, but received input with shape (None, 100, 1)

Call arguments received by layer 'critic_x' (type Sequential):
  • inputs=tf.Tensor(shape=(None, 100, 1), dtype=float64)
  • training=None
  • mask=None```

Any suggestions?

UPDATE: The following seems to work:

channel = 0
Y = X[:, :, channel:channel+1]
## X.shape - (1225, 100, 25) 
## Y.shape - (1225, 100, 1)

from model import hyperparameters
from orion.primitives.tadgan import TadGAN

hyperparameters["epochs"] = 1
hyperparameters["input_shape"] = (100, 25) # based on the window size
hyperparameters["optimizer"] = "keras.optimizers.Adam"
hyperparameters["learning_rate"] = 0.0005
hyperparameters["latent_dim"] = 20
hyperparameters["target_shape"] = None
hyperparameters["batch_size"] = 64

tgan = TadGAN(**hyperparameters)
tgan.fit(X,Y)