Thanks @valhalla for your response.
You mean like this ? Is that where 100 goes ?
# inside group_texts function
marker =  [i for i in enumerate(examples.input_ids) if i==985][1] # 985 is id of ']', we take second occurence.
result["labels"] = result["input_ids"].copy()
result["labels"][:marker+1] = [-100]*(marker+1)

I’m a bit confused about labels here. In the the script I see result["labels"] = result["input_ids"].copy which just means we are taking everything as label and then perform CLM on both sentences. Also what about attention_mask and token_type_ids. I’m not clear on where to specify the context, and where to specify label (i.e sentence2).
Can you please clarify how to go about it ?