Cannot inherit from base config with hydra

[henrycharlesworth]

What is your question?

Let's say I have a base config called "roberta_test.yaml":

common:
  fp16: true
  log_format: json
  log_interval: 200

checkpoint:
  no_epoch_checkpoints: true #only keep best and last checkpoints.

task:
  _name: masked_lm
  data: ../../../experimenting/data/wikitext-103
  sample_break_mode: complete
  tokens_per_sample: 512

criterion: masked_lm

dataset:
  batch_size: 16
  ignore_unused_valid_subsets: true

optimizer:
  _name: adam
  weight_decay: 0.01
  adam_betas: (0.9, 0.98)
  adam_eps: 1e-06

lr_scheduler:
  _name: polynomial_decay
  warmup_updates: 10000

optimization:
  clip_norm: 0
  lr: [0.0005]
  max_update: 125000
  update_freq: [16]

model:
  _name: roberta
  max_positions: 512
  dropout: 0.1

which is working as expected with hydra train, and then let's say I want to inherit from this in a new yaml file ("roberta_inherit"):

defaults:
  - roberta_test
  - _self_

model:
  encoder_layers: 10

task:
  tokens_per_sample: 700

but this isn't working properly. It only seems to inherit the model and task entries of "roberta_test" but none of the others. When I test this independently with hydra it does seem to work, but there seems to be something a bit odd going on here.

Any idea what's going on/ does anyone have a way of inheriting from a base config like this that will work properly here?

Note: I upgraded to hydra-core==1.1.0 (for other features), which I know isn't officially supported by fairseq, but does generally seem to work.