Q:

.o vs .out in C

I compile C code using gcc in a debian system. Normally, I would use gcc file.c -o file.out. But I mistakenly typed gcc file.c -o file.o .
When, running ./file.o it still worked! 
What is this .o file, is it same as .out?

A:

By convention, the .o suffix is the object code.  GCC and other compilers actually run through several steps when compiling.  At a high level, it looks like this**:

pre-processor.  This resolves your #define and #include and other #... macros.  This step is rarely output to a file - it's almost universally passed directly into the next step.
Object compilation.  The lines of code are transformed into machine code and symbol tables.  The symbol tables are not yet linked into an executable.  For larger programs, this is an intermediate output when you are compiling multiple files which may have dependencies on one another.
Linking. The symbols in the object code are now resolved to the actual memory locations or call points to provide a fully fledged executable.

** yes, this is a simplified view that leaves out optimizations, debugging symbols, stripping, library linkages and a few other steps, but those could also be considered sub steps in the compilation process.
By convention:

C files are named .c and .h 
C++: .cpp and .hpp or .c++ and .h++
Object files are named .o.  
Fortran is .f.  
Assembly is .as.
Static libraries are .a on UNIX and .lib on Windows
Dynamic libraries are .so on UNIX and .dll on Windows
Executables generally have no extension on UNIX, and .exe on Windows, though there is a default of a.out for linked C and C++ programs that haven't specified an output name.

There is no reason or requirement for the above, except that's what has been done since the '70s and that's what programmers expect.

A:

.o files are the object files compiled from a source code by a compiler. This are raw binary files and are not yet linked with any platform dependent library. You can't just run a .o file on your system.
gcc -c program.c

(Notice the -c flag is telling the compiler only to compile.)
will produce a binary file program.o which you cannot be run on your platform immediately.You need to link it with crt objects, and the dynamic libraries for the platform.
For my linux system.
ld -dynamic-linker /lib64/ld-linux-x86-64.so.2 /usr/lib/x86_64-linux-gnu/crt1.o /usr/lib/x86_64-linux-gnu/crti.o program.o -lc /usr/lib/x86_64-linux-gnu/crtn.o

Now this will create an executable file a.out which you could execute right from the console.
however when you run
gcc program.c

The compiler performs the linking part for you and spits out a a.out executable file.
Conclusion: Compilers generates .o object files by compiling the source code which then needs to be linked with platform's object files and libraries to generate the executable .out object file for that platform, a process called linking.
Some references:
[1] https://en.wikipedia.org/wiki/A.out 
[2] https://en.wikipedia.org/wiki/Crt0 
[3] https://en.wikipedia.org/wiki/Object_file 
[4] http://man7.org/linux/man-pages/man8/ld.so.8.html 
Edit:
the -o flag is used place the output of compiler in the specified file.
This is the excerpt from the manual.
-o file

Place output in file file.  This applies to whatever sort of
             output is being produced, whether it be an executable file, an
             object file, an assembler file or preprocessed C code.
       If -o is not specified, the default is to put an executable file
       in a.out, the object file for source.suffix in source.o, its
       assembler file in source.s, a precompiled header file in
       source.suffix.gch, and all preprocessed C source on standard
       output.

The extension does not really matter here.

Filesystems for UNIX-like operating systems do not separate the
  extension metadata from the rest of the file name. The dot character
  is just another character in the main filename.
  It is mostly intended for the human user. It is more common, especially 
    in binary files, for the file itself to contain internal metadata
    describing its contents.

Both the .o and .out or any extension files would contain same binary output if they are coming from the same compile operation.

