defining update before initializing parameters gives counter-intuitive behavior

[marcoct]

@gen function foo()
    @param x::Float64
    return x
end

update = ParamUpdate(FixedStepGradientDescent(0.01, foo))
init_param!(foo, :x, 0.0)
trace = simulate(foo, ())
accumulate_param_grad!(trace, 1.0) # set nonzero gradient wrt x
apply!(update) # this is a no-op

The issue does not happen if you list the parameters explicity:

update = ParamUpdate(FixedStepGradientDescent(0.01, foo => :x)

The issue is due a call to get_params inside the 1-argument constructor for ParamUpdate. This call only finds parameters that are already initialized with init_param!.

[ztangent]

I wonder if we could automatically initialize all parameters to default values (e.g. to 0 of the appropriate type, nothing if no type is specified) so that this doesn't occur? I think it would be fairly straightforward to walk the entire AST during macro-expansion and find @param expressions for the dynamic language. And for static models we have the static IR, which gives us all the parameters we need to initialize.

[SamWitty]

Adding to @ztangent's post above, would it be possible to allow a user to initialize the parameters within the body of the model definition? Something like the following:

@gen function foo()
    @param mu::0.0
    @param var::1.0
    @trace(normal(mu, var), :x)
end

[marcoct]

Adding to @ztangent's post above, would it be possible to allow a user to initialize the parameters within the body of the model definition?

Yes, except that I think the syntax would be:

@gen function foo()
    @param mu::Float64 = 0.0
    @param var::Float64 = 1.0
    @trace(normal(mu, var), :x)
end
``