Reasons for Sujud As-Sahw
Question

When is it prescribed for a worshipper to do the prostration of forgetfulness?
Praise be to Allah.By the mercy of Allah to His slaves, as one of the beauties of this perfect religion, Allah has prescribed that His slaves may make up for shortcomings and mistakes that they make in their worship and cannot avoid completely, by performing supererogatory (nafl) acts of worship, praying for forgiveness and so on. 
One of the things that Allah has prescribed for His slaves to make up for shortcomings that may occur in their prayer is the prostration of forgetfulness, but it is only prescribed to make up for certain things; it does not make up for everything nor is it prescribed for everything. 
Reasons for Sujud As-Sahw
Shaykh Ibn ‘Uthaymin (may Allah have mercy on him) was asked about the reasons for doing the prostration of forgetfulness , and he replied as follows: 
The prostration of forgetfulness in prayer is generally prescribed for three reasons: 

Doing something extra
Omitting something
Being uncertain

Doing something extra: for example, doing an extra bow (ruku`), prostration (sujud), standing or sitting. 
Omitting something: such as omitting an essential part of the prayer, or omitting one of the obligatory parts of the prayer. 
Being uncertain: such as when a person is not sure how many rak`ahs he has prayed , whether it was three or four, for example. 
How to offer Sujud as-Sahw when doing something extra in prayer
In the case of doing something extra, if a person adds something to his prayer – bowing, prostrating, standing or sitting – deliberately, then his prayer becomes invalid, because when he added it, that means that he did not do the prayer in the manner ordained by Allah and His Messenger (peace and blessings of Allah be upon him). The Prophet (peace and blessings of Allah be upon him) said: “Whoever does an action that is not part of this matter of ours (i.e., Islam), will have it rejected.” (Narrated by Muslim, 1718) 
But if he did that extra thing by mistake, then his prayer is not invalidated, but he should do the prostration of forgetfulness after saying the salam. The evidence for that is the hadith of Abu Hurayrah (may Allah be pleased with him) who said that when the Prophet (peace and blessings of Allah be upon him) said the salam after two rak`ahs in one of the two afternoon prayers, either Zuhr or ‘Asr, and they told him about that, he (peace and blessings of Allah be upon him) did the rest of the prayer, then he said the salam, then he prostrated twice (the prostration of forgetfulness) after saying the salam. Narrated by al-Bukhari, 482; Muslim, 573. 
And Ibn Mas’ud (may Allah be pleased with him) narrated that the Prophet (peace and blessings of Allah be upon him) led them in praying Zuhr and he prayed five rak`ahs. When he had finished it was said to him: “Has something been added to the prayer?” 
He said: “Why are you asking that?” 
They said: “You prayed five (rak`ahs).” 
So he turned towards the qiblah and prostrated twice. Narrated by al-Bukhari, 4040; Muslim, 572. 
How to offer Sujud as-Sahw when omitting something in prayer
In the case of omitting something, if a person omits one of the essential parts of the prayer, one of the following two scenarios must apply: 
Either he remembers it before he reaches the same point in the following rak`ah, so he has to go back and do that essential part of the prayer and whatever came after it, 
Or he does not remember it until he reaches the same point in the following rak`ah, in which case the following rak`ah takes the place of the one in which he omitted that essential part, and he should make up the invalid rak`ah. 
In either of these two cases, he should do the prostration of forgetfulness after saying the salam. 
For example: a man stood up after doing the first prostration of the first rak`ah and did not sit or do the second prostration. When he started to recite Quran he remembers that he had not done the second prostration or sat between the two prostrations. In that case he should go back and sit as between the two prostrations, then prostrate, then stand up and do whatever is left of his prayer, and do the prostration of forgetfulness after saying the salam. 
An example of one who did not remember until after he reached the same point in the following rak`ah is a man who stood up following the first prostration in the first rak`ah and did not do the second prostration or sit between the two prostrations, but he did not remember that until he sat between the two prostrations in the second rak`ah. In this case he should make the second rak`ah the first rak`ah, and add another rak`ah to his prayer, then say the salam then do the prostration of forgetfulness. 
Omitting an obligatory part of the prayer – such as if he omits an obligatory part and moves on to the next part of the prayer. For example, he forgets to say Subhan Rabbiy al-A’la (Glory be to my Lord Most High) and he does not remember until he has got up from prostrating. In this case he has omitted an obligatory part of the prayer by mistake, so he should carry on with his prayer then do the prostration of forgetfulness before saying the salam, because when the Prophet (peace and blessings of Allah be upon him) omitted the first tashahhud he carried on with his prayer and did not go back and repeat anything, then he did the prostration of forgetfulness before saying the salam. 
How to offer Sujud as-Sahw when you are uncertain
In the case of being uncertain, such as if a person is not sure if he has done something extra or omitted something, e.g., he is not sure whether he has prayed three rak`ahs or four, one of the following two scenarios must apply: 
Either he thinks that one or the other is more likely to be case, whether it is doing more or omitting something, in which case he should proceed on the basis of what he thinks is more likely to be the case, complete his prayer accordingly, then do the prostration of forgetfulness after saying the salam; 
Or neither appears more likely to be the case, so he should proceed on the basis of what is certain, namely the lesser amount, and complete his prayer accordingly, then do the prostration of forgetfulness before saying the salam. 
For example: a man prays Zuhr then he is not sure whether he is praying the third or fourth rak`ah, but he thinks it more likely that it is the third. So he should pray another rak`ah, then say the salam, then do the prostration of forgetfulness. 
An example of when neither seems to him more likely to be the case is when a man is praying Zuhr and is not sure whether he is in the third rak`ah or the fourth, and neither seems to him more likely to be the case. In that case he should proceed on the basis of what is certain, which is the lesser amount. So he should regard it as the third rak`ah, then do another rak`ah and do the prostration of forgetfulness before saying the salam.
Thus it becomes clear that sujud as-sawh should be done before the salam if you have omitted one of the obligatory parts of the prayer or if you are not sure how many rak`ahs you have done, and neither of the two choices seems more likely to be the case. And it should be done after the salam if you have added something extra to the prayer or you are not certain but one of the two choices seems more likely to be the case. (See Majmu’ Fatawa al-Shaykh, 14/14-16) 
For more, please see this category: Prostration of Forgetfulness
And Allah knows best.