Question: My friend and I are visiting for 5 days in early Jan and are uncertain as to whether we should base ourselves in Reykjavik and do multiple single day tours. Or go on an actual multi day tour like this one. 
<https://www.adventures.is/iceland/multiday-tours/multiday-adventures/golden-circle-south-coast-ice-cave/>

Can you someone give me an idea of the pros and cons of both options?

The main priorities are seeing the northern lights, glaciers and ice caving.

Answer: 
With enough time, you can chose and pick the best day, the best weather for each tour when doing single day tours.

I managed to have beautiful weather for each of the tours I did, spending the rainy days in the city on the days between.

Multiple day tours mostly pack in more for the time you are out, covering a bigger area of the country, but you are stuck with the weather on the day you are there.

Which one is better depends on the weather the time you are there, how much time you have to avoid bad weather and the actual tours involved.

If you have only 5 days for 5 day of tours, a 5 day tour might be your best bet. But if you are willing to pick and chose, and do only three days of tours, day tours might be better.
A multi-day tour will have access to further locations. There are still plenty of things to do within reach of Reykjavik but if you want to see Jokulsarlon, which I would say is definitely one of the beautiful spots in Iceland, a day is too short to reach it and back.

The advantage of day tours is that you have your own choice of lodging and you do not have to move your stuff between places. It's a different pace but also a you have a different reach. You also have more flexibility for meals.

Note that most of the day is dark in Iceland in January, so even with several full days of tours, there will not be much more time for site-seeing than with a single-day tour. Now, if this were June, you could be site-seeing in the middle of the night. Actually, the light was so beautiful that two nights I decided not to sleep at all in order to take more in!