He wants more money from him than they agreed upon
Question

                Last year I went for ‘Umrah, praise be to Allah. Before I paid the costs for travel, I asked the agent between me and the travel agency to let me stay during my trip to the holy places in a double occupancy room, me and my wife, and he agreed on condition that I pay an extra cost of 40,000. I agreed to that and I paid all the costs agreed upon. When we reached al-Madinah al-Munawwarah, the agent gave me the key to a double occupancy room, but in Makkah al-Mukarramah, the agent said to me: You did not have the right to a double occupancy room; the extra amount you paid only covered the room in Madinah. If you want a double occupancy room you have to pay the same extra amount, which was 40,000. I only agreed to give him the extra amount when we returned to our country, and he agreed to that and give me a double occupancy room. I had a friend with me who did not pay a single extra dinar, but he was given a double occupancy room in Madinah and also in Makkah, and they did not ask him for anything.
I think that the agent was taking advantage of me to make me pay again. Until now, and after my return to my country, I have not paid what they asked me for. Please note that 40,000 is equivalent to 26% of the cost of ‘umrah.
Praise be to Allah.If there was an agreement between you and the agent that you would pay the extra cost of 40,000 in return for staying in double occupancy rooms for the duration of your trip to the holy places, then the agent is obliged to uphold that agreement, because Allah, may He be exalted, says (interpretation of the meaning): “O you who believe! Fulfill (your) obligations [or contracts]” [al-Maa’idah 5:1]. And the Prophet (blessings and peace of Allah be upon him) said: “The Muslims are bound by their conditions.” Narrated by Abu Dawood (3594); classed as saheeh by al-Albaani in Saheeh Abi Dawood. 
It is not permissible for him to demand extra payments or to claim that that money was only for accommodation in Madinah, so long as the agreement covered the entire trip. 
What matters in this case is the wording of the agreement between you and him, not whether he took money from someone else or let him off. Each person should act according to his contract and agreement, and the agent or travel agency may let some pilgrims off and not others. 
If you are certain that the agreement covered the duration of the entire trip, i.e., it covered your stay in both Madinah and Makkah, then you do not have to pay anything more than what you already paid, and it does not matter that you agreed to pay the additional amount and promised to do that. 
And Allah knows best.