Why Do We Say Salam at the End of Prayer?
Question

When we say “As-salamu `Alaykum wa rahmat-Allah” at the end of the prayer, to whom are we saying that? Are we saying it to the angels who are present on our right and on our left, or are we saying it to the people on our right and on our left?
Praise be to Allah.Taslim is one of the pillars of prayer
The Taslim or Salam at the end of the prayer is one of its pillars or essential parts, and one cannot end or exit from the prayer without saying it. 
Abu Dawud (61) and At-Tirmidhi (3) narrated from `Ali (may Allah be pleased with him) who said: The Messenger of Allah (blessings and peace of Allah be upon him) said: “The key to prayer is purification, its start is the Takbir (saying Allahu Akbar) and its exit is the Taslim (saying the Salam).” (Classed as authentic by Al-Albani in Sunan Abu Dawud) 
This is the view of a number of scholars, including the Imams Malik, Ash-Shafi`i and Ahmad. 
An-Nawawi (may Allah have mercy on him) said: 
“Our view is that it is obligatory, one of the pillars (essential parts) of prayer without which it is not valid. This is the view of the majority of scholars among the Companions, Successors and those who came after them. (End quote from Al-Majmu`, 3/462) 
Why do we say Salam at the end of prayer?
When saying the Salam, the worshipper should intend to exit the prayer and say Salam upon the Imam and those to his right and his left, and to the recording angels. 
An-Nawawi (may Allah have mercy on him) said: 
“When saying the Salam, the Imam should intend the first Salam as an exit from the prayer and a greeting to those who are on his right and upon the recording angels, and he should intend the second Salam as a greeting to those on his left and the recording angels. The one who is praying behind him should intend the first Salam as an exit from the prayer and as a greeting to the Imam, the recording angels and the worshippers in his vicinity, in his row and behind him and in front of him, and he should intend the second Salam as a greeting to the worshippers in his vicinity, and if the Imam is in front of him, he should intend his two Salams to be greetings to him too. 
The one who is praying alone should intend his first Salam as an exit from the prayer and a greeting to the recording angels, and the second Salam as a greeting to the recording angels. The basic principle in his case is that which was narrated by Samurah ibn Jundub (may Allah be pleased with him) who said: The Messenger of Allah (blessings and peace of Allah be upon him) instructed us to send Salams upon ourselves and to greet one another with Salam… 
If he intends to exit the prayer and does not intend anything else, that is permissible because saying Salam to those who are present is Sunnah.” (End quote from Al-Majmu`, 3/456)
Muslim (431) narrated that Jabir ibn Samurah (may Allah be pleased with him) said: When we prayed with the Messenger of Allah (blessings and peace of Allah be upon him) we used to say, As-salamu `Alaykum wa rahmat-Allah, As-salamu `alaykum wa rahmat-Allah (Peace be upon you and the mercy of Allah), and he gestured with his hand to either side. The Messenger of Allah (blessings and peace of Allah be upon him) said: “Why do you gesture with your hands as if they were the tails of restive horses? Rather it is sufficient for one of you to put his hand on his thigh then say Salams to his brothers to his right and left.” 
An-Nawawi (may Allah have mercy on him) said: 
“The words of the Prophet (blessings and peace of Allah be upon him), “then say Salams to his brothers to his right and left” refer to his fellow worshippers who are present on his right on his left. (End quote) 
Ibn Qudamah (may Allah have mercy on him) said:
“He should intend when saying the Salam, to exit the prayer; if he also intends alongside that to respond to the two angels and to those who are behind him, if he is the Imam, or to the Imam and the other people with him, if he is a member of the congregation, there is nothing wrong with that. This was stated by Ahmad who said: He may say the Salam at the end of the prayer, intending thereby to respond to the Imam.” End quote from Al-Mughni, 1/326-327)
Shaykh Ibn `Uthaymin (may Allah have mercy on him) said: 
“If it is asked: upon whom are Salams being sent? 
The answer is that they said: If there is a group of others with him, then the Salam is upon them; if there is no group with him, then the Salam is upon the angels who are on his right and his left, so he says: As-salamu `alaykum wa rahmat-Allah. (End quote from Ash-Sharh Al-Mumti`, 3/208) 
To sum up, when saying the Salam at the end of the prayer, the worshipper intends three things: 

To exit the prayer
To send Salams upon the recording angels
To send Salams upon his fellow-worshippers.

If he is praying alone, then he intends when saying the Salam to exit the prayer and send Salams upon the recording angels. 
For more details, please see the following answers: 105297 , 69819 , 125897 , and 138666 .
And Allah knows best.