Why was the Sharingan's genjutsu ability not used much in the 4th Ninja War? Until the 4th ninja war, a Mangekyou Sharingan user had been showcased as a formidable enemy, with Genjutsu being one of their many powerful techniques. Itachi was adept with Tsukuyomi. Sasuke suppressed the Nine Tails' power with his Sharingan when Naruto faced him in Orochimaru's hideout. 

It would be fair to consider that both Tobi and Madara are equally adept, if not better, with the Genjutsu abilities of the Mangekyou Sharingan. However, in the 4th Ninja War, they do not use Genjutsu much. 

It could be argued that a Sharingan user may be able to cancel out Sharingan's Genjutsu, but Genjutsu could still be effective against a non-Sharingan user, such as Naruto or Bee. 

Tobi could easily put them under Genjutsu and take over their tailed beasts, but he has not done it. He did not use it on Guy either when they fought.

Is there a reason why the Sharigan users did not use Genjutsu in the 4th Ninja War?
Using genjutsu on an enemy is futile, if they have allies nearby, who can infuse chakra to take them out of it. Genjutsu would not work on Bee, since his bijuu Gyuki would easily bring him out of it, as he did when Bee was momentarily trapped in Sasuke's genjutsu. 

If Tobi were to use it on Naruto, Bee would easily bring him out of it. Moreover, during the Kazekage Rescue arc, Naruto already demonstrated that he knows how to cancel genjutsu (although it didn't work). Now he was much stronger, having learned Sage Mode and Bijuu Mode. In short, the chances of a Genjutsu on Naruto succeeding are very small. 

Moreover, Tobi now had a Rinnegan, which he considered much more powerful, and wanted to try it out in battle, which he did with the so-called "Jinchuriki of Six Paths" technique. 

As for Madara, his reincarnation did not go according to his plan. He wasn't meant to capture the tailed beasts, but even if he did, he did not have the Gedo Mazo to seal them in. 

Further, ever since he had been reincarnated, Madara has spent more time showing off his powers, and complaining that the current generation of ninja is so weak compared to Hashirama and him than actually fighting. Casting a Genjutsu on "weak" ninja is probably not as satisfying for his pride as dropping meteors, using Mokuton techniques, Susanoo, and the like. :)