In physics, the gauge covariant derivative is a means of expressing how fields vary from place to place, in a way that respects how the coordinate systems used to describe a physical phenomenon can themselves change from place to place. The gauge covariant derivative is used in many areas of physics, including quantum field theory and fluid dynamics and in a very special way general relativity.   

If a physical theory is independent of the choice of local frames, the group of local frame changes, the gauge transformations, act on the fields in the theory while leaving unchanged the physical content of the theory. Ordinary differentiation of field components is not invariant under such gauge transformations, because they depend on the local frame. However, when gauge transformations act on fields and the gauge covariant derivative simultaneously, they preserve properties of theories that do not depend on frame choice and hence are valid descriptions of physics.  Like the covariant derivative used in general relativity (which is special case), the gauge covariant derivative is an expression for a connection in local coordinates after choosing a frame for the fields involved, often in the form of index notation.

Overview
There are many ways to understand the gauge covariant derivative. The approach taken in this article is based on the historically traditional notation used in many physics textbooks.  Another approach is to understand the gauge covariant derivative as a kind of connection, and more specifically, an affine connection. The affine connection is interesting because it does not require any concept of a metric tensor to be defined; the curvature of an affine connection can be understood as the field strength of the gauge potential.  When a metric is available, then one can go in a different direction, and define a connection on a frame bundle. This path leads directly to general relativity; however, it requires a metric, which particle physics gauge theories do not have.

Rather than being generalizations of one-another, affine and metric geometry go off in different directions: the gauge group of (pseudo-)Riemannian geometry must be the indefinite orthogonal group O(s,r) in general, or the Lorentz group O(3,1) for space-time. This is because the fibers of the frame bundle must necessarily, by definition, connect the tangent and cotangent spaces of space-time. In contrast, the gauge groups employed in particle physics could in principle be any Lie group at all, although in practice the Standard Model only uses U(1), SU(2) and SU(3).  Note that Lie groups do not come equipped with a metric.

A yet more complicated, yet more accurate and geometrically enlightening, approach is to understand that the gauge covariant derivative is (exactly) the same thing as the exterior covariant derivative on a section of an associated bundle for the principal fiber bundle of the gauge theory; and, for the case of spinors, the associated bundle would be a spin bundle of the spin structure. Although conceptually the same, this approach uses a very different set of notation, and requires a far more advanced background in multiple areas of differential geometry.

The final step in the geometrization of gauge invariance is to recognize that, in quantum theory, one needs only to compare neighboring fibers of the principal fiber bundle, and that the fibers themselves provide a superfluous extra description.  This leads to the idea of modding out the gauge group to obtain the gauge groupoid as the closest description of the gauge connection in quantum field theory.

For ordinary Lie algebras, the gauge covariant derivative on the space symmetries (those of the pseudo-Riemannian manifold and general relativity) cannot be intertwined with the internal gauge symmetries; that is, metric geometry and affine geometry are necessarily distinct mathematical subjects: this is the content of the Coleman–Mandula theorem. However, a premise of this theorem is violated by the Lie superalgebras (which are not Lie algebras!) thus offering hope that a single unified symmetry can describe both spatial and internal symmetries: this is the foundation of supersymmetry.

The more mathematical approach uses an index-free notation, emphasizing the geometric and algebraic structure of the gauge theory and its relationship to Lie algebras and Riemannian manifolds; for example, treating gauge covariance as equivariance on fibers of a fiber bundle. The index notation used in physics makes it far more convenient for practical calculations, although it makes the overall geometric structure of the theory more opaque.  The physics approach also has a pedagogical advantage: the general structure of a gauge theory can be exposed after a minimal background in multivariate calculus, whereas the geometric approach requires a large investment of time in the general theory of differential geometry, Riemannian manifolds, Lie algebras, representations of Lie algebras and principle bundles before a general understanding can be developed. In more advanced discussions, both notations are commonly intermixed.

This article attempts to follow more closely to the notation and language commonly employed in physics curriculum, touching only briefly on the more abstract connections.

Motivation of the covariant derivative through gauge covariance requirement

Consider a generic (possibly non-Abelian) gauge transformation acting on a  component field . The main examples in field theory have a compact gauge group and we write the symmetry operator as  where  is an element of the Lie algebra associated with the Lie group of symmetry transformations, and can be expressed in terms of the hermitian generators of the Lie algebra (i.e. upto a factor , the infinitesimal generators of the gauge group), , as . 

It acts on the field  as

Now the partial derivative  transforms, accordingly, as
.
Therefore, a kinetic term of the form  in a Lagrangian is not invariant under gauge transformations.

Definition of the gauge covariant derivative

The root cause of the non gauge invariance is that in writing the field  as a row vector or in index notation , we have implicitly made a choice of basis frame field i.e. a set of fields  
such that every field can be uniquely expressed as  
for functions  (using Einstein summation), and assumed the frame fields  are constant. 
Local (i.e.  dependent) gauge invariance can be considered as invariance under the choice of frame. 
However, if one basis frame is as good as any gauge equivalent other one, we can not assume a frame fields to be constant without 
breaking local gauge symmetry.    

We can introduce the gauge covariant derivative   as a 
a generalisation of the partial derivative  that acts directly on the field  rather than its components  with respect to a choice of frame.  
A gauge covariant derivative is defined as an operator satisfying a product rule  
 
for every smooth function  (this is the defining property of a connection). 

To go back to index notation we use the product rule
.
For a fixed ,   is a field, so can be expanded w.r.t. the frame field.
Hence a gauge covariant derivative and frame field defines a (possibly non Abelian) gauge potential

(the factor  is conventional for compact gauge groups and is interpreted as a coupling constant). 
Conversely given the frame  and a gauge potential ,
this uniquely defines the gauge covariant derivative. 
We then get
.
and with suppressed frame fields this gives in index notation 

which by abuse of notation is often written as 
.
This is the definition of the gauge covariant derivative as usually presented in physics.  

The gauge covariant derivative is often assumed to satisfy additional conditions making additional structure "constant" in the sense that the covariant derivative vanishes.
For example if we  have a Hermitian product  on the fields (e.g the Dirac conjugate inner product  for spinors) reducing the gauge group to a unitary group, we can impose the further condition

making the Hermitian product "constant". Writing this out with respect to a local -orthonormal frame field gives  
,
and using the above we see that  must be Hermitian i.e.  (motivating the extra factor ). The Hermitian matrices are (up to the factor ) the generators of the unitary group.  More generally if the gauge covariant derivative preserves a gauge group  acting with representation , 
the gauge covariant connection can be written as

where  is representation of the Lie algebra associated to the group representation  (loc. cit.).

Note that including the gauge covariant derivative (or its gauge potential), as a physical field, "field with zero gauge covariant derivative along the tangent of a curve "  

is a physically meaningful definition of a field  constant along a (smooth) curve.
Hence the gauge covariant derivative defines (and is defined by) parallel transport.

Gauge Field Strength
Unlike the partial derivatives, the gauge covariant derivatives do not commute. However they almost do in the sense that the commutator is not an operator of order 2 but of order 0, i.e. is linear over functions:
. 
The linear map 
 
is called the gauge field strength (loc. cit).
In index notation, using the gauge potential 
.
If  is a G covariant derivative, one can interpret the latter term as a commutator in the Lie algebra of G and  as Lie algebra valued (loc. cit).

Invariance under gauge transformations

The gauge covariant derivative transforms covariantly under Gauge transformations, i.e. for all 

which in operator form takes the form

or 

In particular (suppressing dependence on )
.
Further, (suppressing indices and replacing them by matrix multiplication) if  is of the form above, 
 is of the form
  
or using , 
 
which is also of this form.  

In the Hermitian case with a unitary gauge group  and we have found a first order differential operator  with  as first order term such that
.

Gauge theory
In gauge theory, which studies a particular class of fields which are of importance in quantum field theory, different fields are used in Lagrangians that are invariant under local gauge transformations. Kinetic terms involve derivatives of the fields which by the above arguments need to involve gauge covariant derivatives.

Abelian Gauge Theory  

the gauge covariant derivative  on a complex scalar field  (i.e. ) of charge  is a  connection. The gauge   potential  is a (1 x 1) matrix, i.e. a scalar. 

The gauge field strength is

The gauge potential can be interpreted as electromagnetic four-potential and the gauge field strength as the electromagnetic field tensor. Since this only involves the charge of the field and not higher multipoles like the magnetic moment (and in a loose and non unique way, because it replaces  by  ) this is called minimal coupling.

Fora Dirac spinor field  of charge  the covariant derivative is also a  connection (because it has to commute with the gamma matrices) and is defined as

where again  is interpreted as the electromagnetic four-potential and  as the electromagnetic field tensor. 
(The minus sign is a convention valid for a Minkowski metric signature , which is common in general relativity and used below. For the particle physics convention , it is .  The electron's charge is defined negative as , while the Dirac field is defined to transform positively as )

Quantum electrodynamics
If a gauge transformation is given by

and for the gauge potential 

then  transforms as
,
and  transforms as

and  transforms as

so that

and  in the QED Lagrangian is therefore gauge invariant, and the gauge covariant derivative is thus named aptly.

On the other hand, the non-covariant derivative  would not preserve the Lagrangian's gauge symmetry, since
.

Quantum chromodynamics
In quantum chromodynamics, the gauge covariant derivative is

where  is the coupling constant of the strong interaction,  is the gluon gauge field, for eight different gluons , and where  is one of the eight Gell-Mann matrices. The Gell-Mann matrices give a representation of the color symmetry group SU(3). For quarks, the representation is the fundamental representation, for gluons, the representation is the adjoint representation.

Standard Model
The covariant derivative in the Standard Model combines the electromagnetic, the weak and the strong interactions. It can be expressed in the following form:

The gauge fields here belong to the fundamental representations of the electroweak Lie group  times the color symmetry Lie group SU(3). The coupling constant  provides the coupling of the hypercharge  to the  boson and  the coupling via the three vector bosons   to the weak isospin, whose components are written here as the Pauli matrices . Via the Higgs mechanism, these boson fields combine into the massless electromagnetic field  and the fields for the three massive vector bosons  and .

General relativity
The covariant derivative in general relativity is a special example of the gauge covariant derivative. It corresponds to the Levi Civita connection (a special Riemannian connection) on the tangent bundle (or the frame bundle) i.e it acts on tangent vector fields or more generally, tensors. It is usually written as  instead of . In this special case, a choice of (local) coordinates  not only gives partial derivatives , but they double as a frame of tangent vectors  in which a vector field  can be uniquely expressed as  (this uses the definition of a vector field as an operator on smooth functions that satisfies a product rule i.e. a derivation). Therefore in this case "the internal indices are also space time indices". 
Upto slightly different normalisation (and notation) the gauge potential  is the Christoffel symbol defined by
.
It gives the covariant derivative 
.
The formal similarity with the gauge covariant derivative is more clear when the choice of coordinates is decoupled from the choice of frame of vector fields . Especially when the frame is orthonormal, such a frame is usually called a d-Bein. Then  

where .
The direct analogue of the "gauge freedom" of the gauge covariant derivative is the arbitrariness of the choice of an orthonormal d-Bein at each point in space-time: local Lorentz invariance . However, in this case the more general independence of the choice of coordinates for the definition of the Levi Civita connection gives diffeomorphism or general coordinate invariance.

Fluid dynamics

In fluid dynamics, the gauge covariant derivative of a fluid may be defined as

where  is a velocity vector field of a fluid.

See also
Kinetic momentum
Connection (mathematics)
Minimal coupling
Ricci calculus

References 

Tsutomu Kambe, Gauge Principle For Ideal Fluids And Variational Principle. (PDF file.)

Differential geometry
Connection (mathematics)
Gauge theories