Geometric invariant theory
In mathematics, geometric invariant theory (or GIT) is a method for constructing quotients by group actions in algebraic geometry, used to construct moduli spaces. It was developed by David Mumford in  1965, using ideas from the paper   in  classical invariant theory. Geometric invariant theory studies an action of a group  on an algebraic variety (or scheme)  and provides techniques for forming the 'quotient' of  by  as a scheme with reasonable properties. One motivation was to construct moduli spaces in algebraic geometry as quotients of schemes parametrizing marked objects. In the 1970s and 1980s the theory developed interactions with symplectic geometry and equivariant topology, and was used to construct moduli spaces of objects in differential geometry, such as instantons and monopoles. Background 

Invariant theory is concerned with a group action of a group  on an algebraic variety (or a scheme) . Classical invariant theory addresses the situation when  is a vector space and  is either a finite group, or one of the classical Lie groups that acts linearly on . This action induces a linear action of  on the space of polynomial functions  on  by the formula

 

The polynomial invariants of the -action on  are those polynomial functions  on  which are fixed under the 'change of variables' due to the action of the group, so that  for all  in . They form a commutative algebra , and this algebra is interpreted as the algebra of functions on the 'invariant theory quotient'  because any one of these functions gives the same value for all points that are equivalent (that is,  for all ). In the language of modern algebraic geometry,

 

Several difficulties emerge from this description. The first one, successfully tackled by Hilbert in the case of a general linear group, is to prove that the algebra  is finitely generated. This is necessary if one wanted the quotient to be an affine algebraic variety. Whether a similar fact holds for arbitrary groups  was the subject of Hilbert's fourteenth problem, and Nagata demonstrated that the answer was negative in general. On the other hand, in the course of development of representation theory in the first half of the twentieth century, a large class of groups for which the answer is positive was identified; these are called reductive groups and include all finite groups and all classical groups. The finite generation of the algebra  is but the first step towards the complete description of , and progress in resolving this more delicate question was rather modest. The invariants had classically been described only in a restricted range of situations, and the complexity of this description beyond the first few cases held out little hope for full understanding of the algebras of invariants in general. Furthermore, it may happen that any polynomial invariant  takes the same value on a given pair of points  and  in , yet these points are in different orbits of the -action. A simple example is provided by the multiplicative group  of non-zero complex numbers that acts on an -dimensional complex vector space  by scalar multiplication. In this case, every polynomial invariant is a constant, but there are many different orbits of the action. The zero vector forms an orbit by itself, and the non-zero multiples of any non-zero vector form an orbit, so that non-zero orbits are parametrized by the points of the complex projective space . If this happens (different orbits having the same function values), one says that "invariants do not separate the orbits", and the algebra  reflects the topological quotient space  rather imperfectly. Indeed, the latter space, with the quotient topology, is frequently non-separated (non-Hausdorff). (This is the case in our example – the null orbit is not open because any neighborhood of the null vector contains points in all other orbits, so in the quotient topology any neighborhood of the null orbit contains all other orbits.) In 1893 Hilbert formulated and proved a criterion for determining those orbits which are not separated from the zero orbit by invariant polynomials. Rather remarkably, unlike his earlier work in invariant theory, which led to the rapid development of abstract algebra, this result of Hilbert remained little known and little used for the next 70 years. Much of the development of invariant theory in the first half of the twentieth century concerned explicit computations with invariants, and at any rate, followed the logic of algebra rather than geometry. Mumford's book 

Geometric invariant theory was founded and developed by Mumford in a monograph, first published in 1965, that applied ideas of nineteenth century invariant theory, including some  results of Hilbert, to modern algebraic geometry questions. (The book was greatly expanded in two later editions, with extra appendices by Fogarty and Mumford, and a chapter on symplectic quotients by Kirwan.) The book uses both scheme theory and  computational techniques available in examples. The abstract setting used is that of a group action on a scheme . The simple-minded idea of an orbit space

i.e. the quotient space of  by the group action, runs into difficulties in algebraic geometry, for reasons that are explicable in abstract terms. There is in fact no general reason why equivalence relations should interact well with the (rather rigid) regular functions (polynomial functions), which are at the heart of algebraic geometry. The functions on the orbit space  that should be considered are those on  that are invariant under the action of . The direct approach can be made, by means of the function field of a variety (i.e. rational functions): take the G-invariant rational functions on it, as the function field of the quotient variety. Unfortunately this — the point of view of birational geometry — can only give a first approximation to the answer. As Mumford put it in the Preface to the book:The problem is, within the set of all models of the resulting birational class, there is one model whose geometric points classify the set of orbits in some action, or the set of algebraic objects in some moduli problem. In Chapter 5 he isolates further the specific technical problem addressed, in a moduli problem of quite classical type — classify the big 'set' of all algebraic varieties subject only to being non-singular (and a requisite condition on polarization). The moduli are supposed to describe the parameter space. For example, for algebraic curves it has been known from the time of Riemann that there should be connected components of dimensions

according to the genus ,and the moduli are functions on each component. In the coarse moduli problem Mumford considers the obstructions to be:

non-separated topology on the moduli space (i.e. not enough parameters in good standing)
infinitely many irreducible components (which isn't avoidable, but local finiteness may hold)
failure of components to be representable as schemes, although representable topologically. It is the third point that motivated the whole theory. As Mumford puts it, if the first two difficulties are resolved [the third question] becomes essentially equivalent to the question of whether an orbit space of some locally closed subset of the Hilbert or Chow schemes by the projective group exists. To deal with this he introduced a notion (in fact three) of stability. This enabled him to open up the previously treacherous area — much had been written, in particular by Francesco Severi, but the methods of the literature had limitations. The birational point of view can afford to be careless about subsets of codimension 1. To have a moduli space as a scheme is on one side a question about characterising schemes as representable functors (as the Grothendieck school would see it); but geometrically it is more like a compactification question, as the stability criteria revealed. The restriction to non-singular varieties will not lead to a compact space in any sense as moduli space: varieties can degenerate to having singularities. On the other hand, the points that would correspond to highly singular varieties are definitely too 'bad' to include in the answer. The correct middle ground, of points stable enough to be admitted, was isolated by Mumford's work. The concept was not entirely new, since certain aspects of it were to be found in David Hilbert's final ideas on invariant theory, before he moved on to other fields. The book's Preface also enunciated the Mumford conjecture, later proved by William Haboush. Stability 
 
If a reductive group  acts linearly on a vector space , then a non-zero point  of  is called
unstable if 0 is in the closure of its orbit,
semi-stable if 0 is not in the closure of its orbit,
stable if its orbit is closed, and its stabilizer is finite. There are equivalent ways to state these (this criterion is known as the Hilbert–Mumford criterion):
A non-zero point  is unstable if and only if there is a 1-parameter subgroup of  all of whose weights with respect to  are positive. A non-zero point  is unstable if and only if every invariant polynomial has the same value on 0 and . A non-zero point  is semistable if and only if there is no 1-parameter subgroup of  all of whose weights with respect to  are positive. A non-zero point  is semistable if and only if some invariant polynomial has different values on 0 and . A non-zero point  is stable if and only if every 1-parameter subgroup of  has positive (and negative) weights with respect to . A non-zero point  is stable if and only if for every  not in the orbit of  there is some invariant polynomial that has different values on  and , and the ring of invariant polynomials has transcendence degree . A point of the corresponding projective space of  is called unstable, semi-stable, or stable if it is the 
image of a point in  with the same property. "Unstable" is the opposite of "semistable" (not "stable"). The unstable points form a Zariski closed set of projective space, while the semistable and stable points both form Zariski open sets (possibly empty). These definitions are from  and are not equivalent to the ones in the  first edition of Mumford's book. Many moduli spaces can be constructed as the quotients of the space of stable points of some subset of projective space by some group action. These spaces can often be compactified by adding certain equivalence classes of semistable points. Different stable orbits correspond to different points in the quotient, but two different semistable orbits may correspond to the same point in the quotient if their closures intersect. Example: 
A stable curve is a reduced connected curve of genus ≥2 such that its only singularities are ordinary double points and every non-singular rational component meets the other components in at least 3 points. The moduli space of stable curves of genus  is the quotient of a subset of the Hilbert scheme of curves in  with Hilbert polynomial  by the group . Example:
A vector bundle  over an algebraic curve (or over a Riemann surface) is a stable vector bundle 
if and only if

for all proper non-zero subbundles  of  and is semistable if this condition holds with < replaced by ≤. See also
GIT quotient
Geometric complexity theory
Geometric quotient
Categorical quotient
Quantization commutes with reduction
K-stability
K-stability of Fano varieties
Bridgeland stability condition
Stability (algebraic geometry)

References 

 Kirwan, Frances, Cohomology of quotients in symplectic and algebraic geometry. Mathematical Notes, 31. Princeton University Press, Princeton, NJ, 1984. i+211 pp. Kraft, Hanspeter, Geometrische Methoden in der Invariantentheorie. (German) (Geometrical methods in invariant theory) Aspects of Mathematics, D1. Friedr. Vieweg & Sohn, Braunschweig, 1984. x+308 pp. ;  (1st ed 1965);  (2nd ed) 
  V. L. Popov, E. B. Vinberg, Invariant theory, in Algebraic geometry. IV. Encyclopaedia of Mathematical Sciences, 55 (translated from 1989 Russian edition) Springer-Verlag, Berlin, 1994. vi+284 pp. Moduli theory
Scheme theory
Algebraic groups
Invariant theory