Potential bug in rdp analysis

[cgastaud]

Hello ! I'm getting suspicious results when computing epsilon with small alphas. For example with the code:

from tensorflow_privacy.privacy.analysis.rdp_accountant import compute_rdp
from tensorflow_privacy.privacy.analysis.rdp_accountant import get_privacy_spent

noise_multiplier = 1
batch_size = 105
steps = 1
delta = 1e-3

orders = [1 + 1e-8] + [1 + x / 10. for x in range(1, 100)] + list(range(12, 64))
sampling_probability = batch_size / 100000
rdp = compute_rdp(
    q=sampling_probability,
    noise_multiplier=noise_multiplier,
    steps=steps,
    orders=orders
)
print(get_privacy_spent(orders, rdp, target_delta=delta))  # returns (0, 0.001, 1.00000001)

The privacy spent should be small but not zero. When I remove (1 + 1e-8) of orders, I get the correct answer. I tracked down this behavior to this line, which puts epsilon to zero for sufficiently small rdp, but I'm not sure why...