But do you?  You really only need to restart the processes using those packages.  Technically, a kernel update (specifically security update, bug fixes may not be important) would only require a reboot.