Q:

How does Linux upgrade packages that are in use?

I am using Arch Linux and I can do a system upgrade while surfing the net for example. How can the browser package be upgraded if the program itself is in use? Or the Kernel for example? Doesn't the executable need to be stopped in order to be replaced with a new one? Or does it happen at the next reboot?

A:

The browser files on disc just get replaced. The running program (if not completely in memory) keeps the old executable files open until the program closes (but until then those are no longer the executables files you get via the directory entries). On the next restart of the browser you get the version.
No reboot necessary except for the program that gets loaded at reboot (i.e. the kernel) There are programs that patch the kernel in place, not even requiring a reboot for that, but that is not the same thing.

A:

As you suspected, Unix-like systems prevent most files being executed from being overwritten. Here's what the standard says about the open system call:

The open() function may fail if:
[ETXTBSY] The file is a pure procedure (shared text) file that is being executed and oflag is O_WRONLY or O_RDWR.

But a file being executed can still be unlinked, and that's how the package managers do their work.
Here's what dpkg on Debian and Ubuntu does when installing the new version of /bin/cpio that just came out:
open("/bin/cpio.dpkg-new", O_WRONLY|O_CREAT|O_EXCL, 0) = 10
// lots of reads and writes omitted from this listing.
// It's copying the new version into dpkg-new
fchown(10, 0, 0)                  = 0 
fchmod(10, 0755)                  = 0 
close(10)                         = 0 
rename("/bin/cpio.dpkg-new", "/bin/cpio") = 0 

In detail, this:

copies the new version to cpio.dpkg-new in the same directory as cpio.
sets the owner and file permissions to whatever the package says they should be
renames cpio.dpkg-new to cpio

The way to safely install a new version of a file without ever being in the position of the file not existing, even for an instant, is to use the rename system call. This requires both files to be in the same filesystem, and that's why dpkg created the new version of cpio in the same directory as the old version. The standard says:

int rename(const char *old, const char *new);
The rename() function shall change the name of a file. The old argument points to the pathname of the file to be renamed. The new argument points to the new pathname of the file.
If the link named by the new argument exists, it shall be removed and old renamed to new. In this case, a link named new shall remain visible to other processes throughout the renaming operation and refer either to the file referred to by new or old before the operation began.

The terminology may be a bit confusing because the old and new files given to rename here are the new version of cpio and the old version of cpio, respectively.
Finally, here is the key to answering your question:

If the link named by the new argument exists and the file's link count becomes 0 when it is removed and no process has the file open, the space occupied by the file shall be freed and the file shall no longer be accessible. If one or more processes have the file open when the last link is removed, the link shall be removed before rename() returns, but the removal of the file contents shall be postponed until all references to the file are closed.

The last sentence means that if any processes have cpio open (or are running it) when the rename completes, they will continue to see the previous contents of the file until they close the file (or exit).
Arch's pacman appears to do roughly the same thing:
snprintf(checkfile, len, "%s.paccheck", filename);
...
if(perform_extraction(handle, archive, entry, checkfile, entryname_orig)) {
    errors++;
    goto needbackup_cleanup;
}
...
if(try_rename(handle, checkfile, filename)) {
    errors++;
}

Notwithstanding the efforts by the package managers to install new files safely, there can be some problems when a program is updated while in use. The firefox package, for instance, has more than a dozen executables and shared objects. Someone running an old version of firefox during an update may find that an extension they call after the update has completed is incompatible with the old version of firefox. I recently updated firefox on Ubuntu, and apt-get printed out:

Please restart all running instances of firefox, or you will experience problems.

If you're a system administrator of a multi-user system, it's a good idea to announce pending updates to the user community.

