How To Get Mean Of Each List Inside The List Avoiding Certain Value?

How to calculate mean of values of each list separately inside the list avoiding a special value (-999)? A = [[4,5,7,8,-999],[3,8,5,7,-999]] M = [sum(x)/len(x) for x in [y for y in

Solution 1:

As you mentioned numpy:

>>> import numpy as np
>>> 
>>> A = [[4,5,7,8,-999],[3,8,5,7,-999]]
>>> M = [np.mean([x for x in L if x > -999]) for L in A]
>>> print M
[6.0, 5.75]

EDIT

As you mentioned speed as an important requirement, you can do this:

>>> B = np.array(A)
>>> np.average(B, axis=1, weights=B!=-999)
array([ 6.  ,  5.75])

Everything happens in numpy (i.e. C) space, which should get it pretty fast.

To explain a bit what is happening:

np.mean(A, axis=1) and the equivalent np.average(A, axis=1) compute the average over the columns of your array, i.e. the vector of means of each row, which is what you want. average allows the use of weights: we use B!=-999, which is a boolean array, evaluated as 1s and 0s when used as weights, i.e. ignoring the values evaluated as False.

Solution 2:

A = [[4,5,7,8,-999],[3,8,5,7,-999]]
M = [sum(z)/float(len(z)) for z in [[x for x in y if x != -999] for y in A]]
print M

Output

[6.0, 5.75]

Solution 3:

Using a generator function, and no need to create a new list in memory:

A = [[4,5,7,8,-999],[3,8,5,7,-999]]
def solve(lis, val):
      for item in lis:
          c = item.count(val)                 #count number of -999 in list
          total = sum(item) - (val*c)         #subtract it from total sum
          yield float(total)/ (len(item) - c) #return mean

print list(solve(A, -999))
#[6.0, 5.75]