I was using the ConversationID of the mails in the Inbox and the mails in the Sent Items, assuming that the IDs will be same for the mails that we replied to, and it is same but the problem begins with Email Chains (or the Re: mails aka Conversations).
Since the ID remains the same, and people keep replying even after days, performing a simple
datetime - datetime (with same ConversationIDs) results in outputs such as -1Day 20:05:01, -9Days etc.
I just want to find the Response time of first reply to any mail, that enters the inbox.
(Sorry, cannot share Code).
To get the Response Time from a Chain of Mails which all have exactly the same ConversationID (generated automatically by Outlook), I did the following:
I already had two Pandas DFs (let's say 'A' and 'B'), having the two columns each, namely:
ConversationID and Time (for received and sent mails).
Now,

Created a Pandas DataFrame 'C' with an inner join of 'A' and 'B' on `ConversationID'.
(This gave me all the mails that I had replied to).

Sort the both the Time columns as "Ascending".
(This brings the First Mail received and the First Mail replied to the top).

Now filter the new DataFrame 'C' on both Time columns with the condition,
Replied Mail Time > Received Mail Time
(Removes the -1 day outputs I was getting)

Groupby on Received Mail Time and use .first()
(Finally giving a genuine response time, after calculating Replied Mail Time - Received Mail Time)
Python Outlook (MAPI) Get Response Time of Mails Replied