However, Olšauskas due to his pro-German and anti-Polish attitudes was not an acceptable candidate to the Holy See. The pope appointed Jurgis Matulaitis-Matulevičius as the new bishop in October 1918. After the failure to become bishop, Olšauskas eyed the position of the Minister of Education in the newly formed government of Augustinas Voldemaras. Instead, he was sent as the Lithuanian delegate to the Paris Peace Conference. His primary goal was to obtain diplomatic recognition as well as financial and military support for Lithuania. Olšauskas together with representatives of minorities in Lithuania – Simon Yakovlevich Rosenbaum (Jewish), Dominik Semashko (Belarusian), and Jurgis Daujotas (Polish) – arrived to Paris in early January 1919 and already found Lithuanians working there – priest Adomas Vilimavičius, engineer Ernestas Galvanauskas, and poet Oscar Milosz. They were soon joined by Juozas Gabrys. Thus, there were three Lithuanian groups that sometimes competed with each other. On 6 January, "Le Temps" published an interview with Rosenbaum who expressed opinions contrary to Lithuanian positions. Due to these misunderstandings, Olšauskas was quickly replaced by Augustinas Voldemaras and returned to Lithuania at the end of March. During the short time, Olšauskas was received by French Prime Minister Georges Clemenceau and introduced Lithuanian issues to other European diplomats. Olšauskas returned to Lithuania in spring 1919. He became a prelate and chaplain of Kaunas Cathedral. He returned to the Saulė Society as chairman and helped finding funding for the various Lithuanian schools – now that government permission was not an issue, the society organized gymnasiums in Kaunas and Telšiai and 11 pro-gymnasiums in Biržai, Utena, Ukmergė, etc. To address the shortage of qualified teachers, the society organized two-year pedagogical courses in Jurbarkas, Utena, Žagarė, Vilkija. After the coup d'état of 1926, authoritarian regime of President Antanas Smetona began limiting and nationalizing private Catholic schools in an effort to weaken the opposition, including the Lithuanian Christian Democratic Party. Olšauskas campaigned against such government actions and harshly criticized Smetona. Together with others, in 1913, he purchased a manor in (a total of ).