I think, more than anything, it was the fact that he was proven wrong. Think about it: His wife abandoned him, he had become a mentally ill recluse, and everyone, including his own parents, valued his brother more than him. The one thing that kept him going, and what drove his rivalry with <PERSON>, was the idea that, no matter how much everyone was against him, he'd always be proven to have the moral high ground in the eyes of the law. In the finale, he lost this edge: He was prepared to sue HM on the grounds that they were wrong to push him into retirement, and the law would prove that he deserved to stay. When <PERSON> bought him out of the firm, it was a huge blow to his ego, as he legally proved that he had the firm's interests in mind when <PERSON> didn't. Notice the surprise on his face when <PERSON> tells him his claim that he was betrayed is bs: <PERSON> was the one person who seemed to remain on his side after everything, and now even he was telling <PERSON> he was wrong.
But the final nail in the coffin was his last meeting with <PERSON>. 86
<PERSON> came to make amends, and <PERSON> rejected him. Now, his words about <PERSON> having never mattered to him were obviously a lie, seeing as most of <PERSON>'s actions since well before the start of the series revolved around <PERSON>. But the message was clear: whether it was actually <PERSON>'s fault or not, nothing good had come from having him in his life. <PERSON> actually seemed to be doing fine until <PERSON> showed up - at the very least, he was still able to keep his electricity on - but the fact that <PERSON> was willing to reach his hand out to <PERSON>, and <PERSON> rejected him, was something <PERSON> couldn't cope with. Unlike his other confrontations with <PERSON> (like after the Mesa Verde incident), he didn't have his legal career to fall back on. <PERSON> was truly left with nothing, no brother, no friends, and no moral superiority over anyone, and that was the end of the road for him. 86