Package holidays, for instance, were up 0.6% between April and May, compared with a fall of 0.4% a year ago.
The fall in the pound since last year's Brexit vote is making itself felt.
Imported goods, particularly, have become more expensive. The cost of games - especially computer games - toys and hobbies shot up by 2.7%.
Prices are also on the rise for clothing, up 0.6% compared with a fall of 0.3% a year ago, with children's clothing seeing the biggest increase.
The ONS also reported higher prices for furniture and household goods.
Shoppers have been used to falling prices in the supermarket aisles in recent years but this trend is now at an end.
Sugar, jams, chocolate, syrups and confectionary all contributed to the latest rise in this category.
With inflation now at its highest rate in nearly four years, the impact of past falls in sterling continue to feed through to households, according to Richard Lim, chief executive of Retail Economics: "Food inflation rose by 2.5% according to the latest data, the fastest since 2013. Given the least affluent households spend around 17% of their expenditure on food and drink, rising prices will have a significant impact on their discretionary spending power going forward.
"What's more, real earnings are already shrinking. With inflation expected to accelerate further in the coming months, the backdrop for retailers looks set to become even more challenging."
Electricity bills jumped by 4% and gas bills were also slightly up, reflecting price hikes from some of the big six energy suppliers.
But travel costs were down. Air fares usually go down after Easter, one of the peak holiday periods. Because Easter came later this year, air fares fell back in May, which was a big contributor to the downward effect.
And the prices at the pump also dropped between April and May this year, bringing some relief, at least, to consumers.
The rising cost of foreign holidays, toys and computer games are the main drivers behind this month's jump in inflation.