Dzsenifer Marozsan put Germany ahead at the Maracana with a curling finish, and Sweden's Linda Sembrant gifted them a second by scoring an own goal after Marozsan's free-kick hit a post.
Stina Blackstenius pulled a goal back from Kosovare Asslani's low cross.
Germany face Brazil in the men's final on Saturday at 21:30 BST.
Canada secured women's bronze by beating Brazil 2-1 in Sao Paulo.
The women's football event has been held at only six Olympics, and this was Germany's fourth medal after they won bronze in 2000, 2004 and 2008.
Sweden reached the final after penalty shootout victories over the United States and Brazil in the quarter-finals and semi-finals respectively.
Described as a "bunch of cowards" by USA goalkeeper Hope Solo, Sweden's silver was their first Olympic women's football medal.
Success for Germany comes a year after they finished fourth in the 2015 World Cup, when they lost 1-0 to England in the third-place play-off.
England head coach Mark Sampson, whose side also beat Canada, this week said Great Britain could have won Olympic gold if they had been able to compete in Rio.
They earned a qualification place as England finished third - the highest-ranked European team - in the World Cup.
However, the English, Scottish, Welsh and Northern Irish football associations could not agree on sending a combined British team, with Sweden winning a qualifying tournament to replace them.
Great Britain reached the quarter-finals at London 2012.
Germany won women's Olympic football gold for the first time with a 2-1 victory over Sweden in the final in Rio.