Lack of tuning of scale factor for variance of normal proposal

[caesoma]

A pretty common feature of Metropolis-Hastings samplers seems to be missing for parameter proposals/jumps, which is the tuning of the variance of a normal distribution according to the acceptance rate. For one-at-a-time parameter proposals, the size of the jump would be scaled according to the acceptance rate to obtain an "optimal" jump size.

In a nutshell, acceptance rates around approx 0.23 wouldn't require tuning, but rates of acceptance that are too low or too high would trigger scaling the size down or up. Given that acceptance rates are tracked, it should be straightforward to implement it by checking them for each parameter at a regular interval (possibly during warm-up only, or throughout) and using a function like what they had in PyMC2. That would provide a "standard" proposal method that didn't require hand tuning proposal distributions, or relied on the scaling of priors, whose variance need not match the size of jumps.

function tune(currentScale, acceptanceRate):
    """Tunes the scaling parameter for the proposal
    distribution according to the acceptance rate over the
    last tune_interval:

    Rate    Variance adaptation
    ----    -------------------
    <0.001        x 0.1
    <0.05         x 0.5
    <0.2          x 0.9
    >0.5          x 1.1
    >0.75         x 2
    >0.95         x 10
    """
    if (acceptanceRate < 0.001)
        # reduce by 90 percent
        newScale = currentScale * 0.1
    elseif (acceptanceRate < 0.05)
        # reduce by 50 percent
        newScale = currentScale * 0.5
    elseif (acceptanceRate < 0.2)
        # reduce by 10 percent
        newScale = currentScale*0.90
    elseif (acceptanceRate > 0.5)
        # increase by ten percent
        newScale = currentScale * 1.1
    elseif (acceptanceRate > 0.75)
        # increase by one hundred percent
        newScale = currentScale * 2
    elseif (acceptanceRate > 0.95):
        # increase by one thousand percent
        newScale = currentScale * 10
    else
        newScale = currentScale
    end

    return newScale
end