Stack Exchange
Python: np.nanpercentile, which datatype does my dataframe need to have?
I have a panda dataframe of type object.
df.dtypes

Out:
data        object
stimulus    object
trial       object
dtype: object

df.head()

Out:
    data    stimulus    trial
0   2      -2           1
1   2      -2           2
2   2      -2           3
3   2      -2           4
4   2      -2           5

I want to get a specific percentile of my dataset. When I use this code, I get NaN in my output, probably because I have NaN in my dataset itself, which python interprets as infinity, so it will get problems when calculating higher percentiles.
df.groupby('stimulus').data.apply(lambda x: np.percentile(x, q=66))

Out:
stimulus
-2.00     2.0
-1.75     2.9
-1.00     1.0
-0.75     1.0
-0.50     0.0
 0.50     7.8
 1.00     9.9
 1.25    11.9
 1.75    13.9
 2.50     NaN

I already found out that I would need to use np.nanpercentile() instead, but when I use np.nanpercentile() instead then I get this error. I read somewhere else that np.nanpercentile() checks the data format of the input array and complains if it doesn't fit. Do you know how and to which format I need to change my data?
df.groupby('stimulus').data.apply(lambda x: np.nanpercentile(x, q=66))

Out:
TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''


Can you just convert the data to a numpy array and then apply np.nanpercentile to it? Like Arr = df.as_matrix(column_number) and then np.nanpercentile(Arr,66)

This did the job for me in the end:
df = df.astype(float)   

