In 2018, La Presse Canadienne (the French-language counterpart of The Canadian Press) developed Ultrad, a machine translation (MT) system. Ultrad is one of the few MT systems used daily by journalists in Canada. All Ultrad English to French translations are reviewed before being published in the target language. Our article focuses on the evaluation of the translations provided by the tool and post-editing by the news agency’s journalists. To conduct this work, we examined a sample of 148 English-language articles with both machine-translated and post-edited versions in French. The evaluation of the translations, on the one hand, and the comparison of the machine-translated and post-edited texts, on the other, relied on different methods of analysis: comparison between word frequencies, paragraph alignment and their sub-segmentation into graphic sentences for each of the two groups of bitexts (English with the raw translation and English with the revised post-edited translation) and the information volumes of the pairs of segments; the three-dimensional Euclidean distance (ED) between the source and the target segments; and the linguistic correction provided by the Antidote linguistic correction software on the entirety of the post-edited articles. Our study is one of the few that has examined the use of MT by journalists. It highlights the shortcomings of MT and the importance of human interventions in post-editing, even if these interventions can be error-prone.