I'd say no, personally. It's not that much to keep track of, since most skills only affect your own character, except for stuff like Charisma and Daunt. Plus, they're what put us above the mooks. Not to mention it's a mechanical bonus (and penalty to a lesser extent) for fluff, which is always nice.