Question: One of my senior collegues optimizes a function (he is implementing image filtering) by writing in-line assembly. Is that really necessary? Wouldn't modern compiler do that for us? Typically, how much gain do we have by converting C code into assembly? If assembly code really brings lots of benefits, when should we convert C/C++ code into assembly and when should we leave the code as it is, since assembly code is hard to read and maintain.

Answer: 
If you are smarter than the compiler, you *may* be able to make your code faster on one specific platform by writing it by hand in assembly.

However, most big C/C++ compilers are *extremely* good optimizers; you are unlikely to be smarter than them.
I'd imagine that for image filtering you might benefit from e.g. the availability of [SIMD](http://en.wikipedia.org/wiki/Streaming_SIMD_Extensions) instructions, but not all compilers can automatically compile your code to use them, and not all the time. So in-line assembly or [intrinsics](http://en.wikipedia.org/wiki/Intrinsic_function) can help with that.