Author of LuaJIT explains why compilers can't beat hand-coded assembly
A couple of years ago I wrote a direct-threaded .NET interpreter. This article made me go and have a look at the assembly that is being generated for the simple 'add' instruction case.The .NET IL is stack-based which the interpreter runs directly, so it will never be as efficient as LuaJits register-based approach, but I was somewhat disappointed to see this as the output of the Windows VS2008 C compiler in release mode (enables all optimizations):  JIT_ADD_I32I32_start:
  	BINARY_OP(I32, I32, I32, +);
  0041164D  mov         eax,dword ptr [pCurEvalStack] 
  00411650  sub         eax,4 
  00411653  mov         dword ptr [pCurEvalStack],eax 
  00411656  mov         ecx,dword ptr [pCurEvalStack] 
  00411659  mov         edx,dword ptr [ecx-4] 
  0041165C  mov         eax,dword ptr [pCurEvalStack] 
  0041165F  add         edx,dword ptr [eax] 
  00411661  mov         ecx,dword ptr [pCurEvalStack] 
  00411664  mov         dword ptr [ecx-4],edx 
  JIT_ADD_I32I32_end:
  	GO_NEXT();
  00411667  mov         edi,dword ptr [pCurOp] 
  0041166A  add         edi,4 
  0041166D  mov         dword ptr [pCurOp],edi 
  00411670  jmp         dword ptr [edi-4]

The assembly after 'BINARY_OP(...)' is the integer add code. pCurEvalStack points to the top of the current evaluation stack. Binary ops are performed on the top two items on the stack, and pushing the result back onto the stack.The assembly after 'GO_NEXT()' is the standard epilogue, which just dispatches the next instruction (see http://en.wikipedia.org/wiki/Threaded_code#Direct_threading for description of direct-threading).There are 3 seperate loads of the same memory address, which surely can be done better.I'll probably have a go at altering the definition of the BINARY_OP(...) macro to see if I can persuade it to generate better code. I'm not keen to hand-write the assembly for this as it can compile to multiple processors.The macro is currently defined as:  #define BINARY_OP(returnType, type1, type2, op) \
  	pCurEvalStack -= sizeof(type1) + sizeof(type2) - sizeof(returnType); \
  	*(returnType*)(pCurEvalStack - sizeof(returnType)) = \
  	*(type1*)(pCurEvalStack - sizeof(returnType)) op \
  	*(type2*)(pCurEvalStack - sizeof(returnType) + sizeof(type1))

...which can probably be improved, or at least specialised to give better results for binary ops where the operand and result types are all the same - e.g. integer addition.