How to avoid overfitting

[umas]

Dear contributors,

Thanks a ton for putting this library together.

I based my classification model on the sentiment.py example. I find that my (Embedding + GatedRecurrent / LstmRecurrent + Dense) combination of layers overfits the data. Training error goes down steadily, but at the same time, prediction error on some completely unseen data goes up steadily.

Do you have any suggestions? p_drop is 0.75, updater=Adadelta(lr=0.5). I have already tried removing last dense layer, reducing the size of each layer and increasing the size of training data.

Thanks in advance! Uma