Stack Exchange
How to include two lambda operations in transform function?
I have a dataframe like as given below
df = pd.DataFrame({
'date' :['2173-04-03 12:35:00','2173-04-03 17:00:00','2173-04-03 20:00:00','2173-04-04 11:00:00','2173-04-04 12:00:00','2173-04-04 11:30:00','2173-04-04 16:00:00','2173-04-04 22:00:00','2173-04-05 04:00:00'],
'subject_id':[1,1,1,1,1,1,1,1,1],
'val' :[5,5,5,10,10,5,5,8,8]
 })

I would like to apply couple of logic (logic_1 on val column and logic_2 on date column) to the code. Please find below the logic
logic_1 = lambda x: (x.shift(2).ge(x.shift(1))) & (x.ge(x.shift(2).add(3))) & (x.eq(x.shift(-1)))
logic_2 = lambda y: (y.shift(1).ge(1)) & (y.shift(2).ge(2)) & (y.shift(-1).ge(1)) 

credit to SO users for helping me with logic
This is what I tried below
   df['label'] = ''
   df['date'] = pd.to_datetime(df['date'])
   df['tdiff'] = df['date'].shift(-1) - df['date']
   df['tdiff'] = df['tdiff'].dt.total_seconds()/3600
   df['lo_1'] = df.groupby('subject_id')['val'].transform(logic_1).map({True:'1',False:''})
   df['lo_2'] = df.groupby('subject_id')['tdiff'].transform(logic_2).map({True:'1',False:''}) 

How can I make both the logic_1 and logic_2 be part of one logic statement? Is it even possible? I might have more than 2 logics as well. Instead of writing one line for each logic, is it possible to couple all logics together in one logic statement.
I expect my output to be with label column being filled with 1 when both logic_1 and logic_2 are satisfied


You can do this:
df['lo_1'] = df.groupby('subject_id')['val'].transform(lambda x: function(x)).map({True:'1',False:''})
and the you have a def function(x): with both logics

But, we have val column for logic_1 and tdiff column for logic_2. In your answer, you are using val column

You mean,  df.groupby('subject_id')['val'].transform(logic_1) & df.groupby('subject_id')['tdiff'].transform(logic_2)?

Yes, you are right

That's the answer..

You men, the one from other post? If you can put it here, I can mark it as answer

Will try your suggestion

Don't assign lambda expressions to variables; use a def statement.

Can you write as an answer @chepner

@MarkWang - You mean to say that we can't get away with groupby statements. Since we have two columns, 2 groupby is necessary?

I'm saying if you need to apply condition based on two columns, you may need groupby.apply instead of transform.

You have a few things to fix
First, in logic_2, you have lambda x but use y, so, you got to change that as below
logic_2 = lambda y: (y.shift(1).ge(1)) & (y.shift(2).ge(2)) & (y.shift(-1).ge(1))

Then you can use the logic's together as below'
No need to create a blank column label. You can create the '`label' column directly as below.
df['label'] = ((df.groupby('subject_id')['val'].transform(logic_1))
           & (df.groupby('subject_id')['tdiff'].transform(logic_2))).map({True:'0',False:'1'})

Note: You logic produces all False values. So, you will get 1's if False is mapped to 1, not True

If I have 10 logics for 10 columns in my real data, do I have to use 10 groupby statements?

Unfortunately, i think yes. But if you put the question, we can see if we can accomplish it with just one def

Sorry, I didn't get you.what question?

for the question where you asked "If I have 10 logics for 10 columns in my real data, do I have to use 10 groupby statements?" , my answer is "Unfortunately, i think yes. But if you put a new question with the result & the logic to get that result, we can see if we can accomplish it with just one"

okay, for logic_1 refer this post  https://stackoverflow.com/questions/57676327/how-to-vectorize-code-with-nested-if-and-loops-in-python

for logic_2, it just that tdiff should be >= 1. If you refer my output screenshot example, you will understand

logic 2 is nothing but diff between n-1 and n should be >=1 and diff between n-2 and n should be >= 2 and difference between n and n+1 should be >= 1. Hope this helps

I did not get the logic. Instead of explaining with numbers, please explain what you are trying to do. For example, you can say, within each day, if the value doubles , then the whole day should get a value of 1 in column label or something like that (i just made this up now). Since it takes quite a bit of explanation, i would say a different question is better.

You can push several columns to do the lambda function, and then, inside the function, separate the columns to perform the individual logic column by column like:
df.groupby('subject_id')['val', 'tdiff', 'colum3', 'column4'.., 'colum10'].transform(logic_1)

Can you giys help me with this post? https://stackoverflow.com/questions/57703423/how-to-get-minimum-of-each-group-for-each-day-based-on-hour-criteria. It is related post
