error in batch_filter when using data with missing values

[crystal-butler]

I have created a second order Kalman filter for 3-dimensional data that I can successfully apply using batch_filter() if I fill missing values in the input first. But if I try running the filter by assigning None to the missing values with df.replace(np.nan, None) (before turning my measurements into a list), I get the error:

TypeError: unsupported operand type(s) for -: 'NoneType' and 'float'

The call stack tracing the error is:

Xs, Ps, Xs_prior, Ps_prior = tracker.batch_filter(body8_joint_measurements)
self.update(z, R=R, H=H)
self.y = z - dot(H, self.x)

I found the related issue and fix for missing values using masked arrays in linear_case_filterpy.ipynb example, but should batch_filter() be working as-is when missing values are filled with None?

[jamieas]

It looks like batch_filter actually wants a list of measurements with None wherever there are missing measurements, as opposed to an array with None as values in the array. However, if you insert None, you get

ValueError: setting an array element with a sequence.

due to the line n = np.size(zs, 0) here:

https://github.com/rlabbe/filterpy/blob/master/filterpy/kalman/kalman_filter.py#L1744

@rlabbe does it make sense to change this to n = len(zs)?

[Jonny-air]

That didn't fix the issue for me. However a quick fix would be to check if any element of z is nan in the update fuction (and all other occurences)

https://github.com/rlabbe/filterpy/blob/3b51149ebcff0401ff1e10bf08ffca7b6bbc4a33/filterpy/kalman/kalman_filter.py#L515

replace if z is None: with if z is None or np.isnan(z).any():

That would achieve the expected behavior. Was there a particular motivation for not doing that?

[AlexAtCCRI]

fixing this would be nice. i dont think the current behavior is functional .

a hack-around

class FilterpyHackList(list):
    @property 
    def shape(self):
        # returns something that yeilds correct value for np.size(zs,axis=0)
        return [len(self),1]