Question: I am looking at uClinux system that builds the kernel with arm-linux-xxx, but builds the user apps with arm-elf-xxx.

If the apps are intended to run on linux, wouldn't it be better to build everything with arm-linux-xxx ?

Where does one set that option in the overall uClinux build config?

Answer: 
You can run `arm-linux-gcc -v` and `arm-elf-gcc -v` to see some differences between these two versions of gcc (such as target architectures/languages).
The difference is that the linked output from the arm-elf-xxx toolchain is a linux loadable binary, while arm-linux-xxx outputs a kernel object (ko), which is a blob with its own linkage format (for modules), or the main kernel itself, which doesn't need a packaging format. I.e., for user space you want the final dynamically linked object to be an ELF executable, library, or shared object. For kernel space, you want the final object (kernel or module) to be in the linux-arm-kernel linkage format.

You'll notice that the intermediate objects (.o files) of both targets are probably all ELF, since that is what the toolchain is expecting.