Importance of tire width How much of an impact does tire width have on the performance of a bicycle?

I am in the process of buying my first real bicycle and am having trouble deciding between a more traditional road bike with 25mm tires versus an "adventure" bike with 32mm tires.  The manufacturers description for the second specifically mentions paths and trails with variable terrain while the road bike only mentions different weather conditions.

Most of my local riding would be on paved trails where I wouldn't expect any problems with the road bike.  However, there is a possibility that I would also like to do some packed dirt canal towpaths that are a little further away.  Ideally one bike would be good in both locations.

Is it a terrible idea to try and use a road bike anywhere but pavement?  How much of a difference does a wider tire make while riding on pavement?
25 mm would be at major disadvantage on dirt canal towpaths.   

32 mm with touring or road type tread is only at minor disadvantage on the road.  Slightly more rolling and air resistance but you pick up softer ride, more traction, and less pinch flats.

You can put smaller tires on the adventure bike but I doubt you could put 32 mm on the road bike.