NaN coordinates cause segfault

[sfreilich]

If the input coordinates are NaN, the call to tessTesselate segfaults in pqInit, since the code falsely assumes a total ordering. Possibly tessAddCountour should check that the values are not NaN?

Specifically, it defines its comparisons as:

#define LT(x,y)     (! LEQ(y,x))
#define GT(x,y)     (! LEQ(x,y))

The function LEQ eventually expands to does comparisons with <= and <, which are always false for NaN. Thus LT(x, y) is always true, and the code here:

do {
  do { ++i; } while( GT( **i, *piv ));
  do { --j; } while( LT( **j, *piv ));
  Swap( i, j );
} while( i < j );

increments i until that pointer escapes the array bounds.