Q:

What is Representation Theory?

I'm beginning a course that uses representation theory, but I do not really understand what that is about. In the text I am following, I have the following definition:

A representation of the Lie group $G$ on the vector space $V$ is a continuous mapping $\cdot \colon G \times V \to V$ such that

for each $g \in G$, the translation $T_{g} \colon V \to V$ given by $T_g(v) = g \cdot v$, $v \in V$, is a linear map;
$T_{e} = \mathrm{Id}$ where $e$ is the identity element of $G$;
$T_{gh} = T_{g} T_{h}$ for $g, h \in G$. We call the pair $(V,\cdot)$ a real (resp. complex) representation and $V$ the representation space. 

What is the motivation behind this sort of definition? From my google searches I have seen different definitions, but I still don't really know why what I am reading is defined that way. Why a Lie group and not a regular group? etc. 

A:

One of the most common ways that groups arise "in the wild" is as sets of symmetries of an object. For example

The symmetry group $S_n$ is the group of all permutations of $\{1,\ldots n\}$
The dihedral group $D_{2n}$ is the group of symmetries of a regular $n$-gon
The Lie group $\mathrm{GL}_n(\mathbb R)$ is the group of invertible linear maps on $\mathbb R^n$

More generally, given a general abstract group $G$, we regularly consider the case of $G$ acting on a set $X$, and we might ask the question: given some set $X$, what is its "group of symmetries".
Representation theory asks the converse to this question:

Given a group $G$, what sets does it act on?

Whilst it is possible to attempt to answer this general, a useful starting point is to restrict the sets in question to sets we know an awful lot about: vector spaces.

Definition: Let $G$ be a group, and $V/k$ be a vector space. A representation of $G$ is a group action of $G$ on $V$ that is linear (so preserves the vector space structure of $V$) - i.e. for every $g\in G$, $u,v\in V$, $\mu,\lambda\in k$ $$g(\lambda u+\mu v) = \lambda g(\mu) + \mu g(v).$$

This is the definition that you have been given. With $V$ as before, an equivalent definition is this:

A representation of $G$ is a group homormophism
  $$\rho: G\to\mathrm{GL}(V)$$

Indeed, a group action of $G$ on $V$ assigns to each $g$ an invertible linear map. And given a homomorphism $\rho$, $G$ acts on $V$ via $g\cdot v = \rho(g)v$.
In the case that $G$ is a Lie group (or more generally a topological group), then we require this action/representation to be continuous.
Representation theory allows us to translate our viewpoint by viewing (a quotient of) our group as a group of linear maps on a vector space. This allows us to tackle problems in group theory using the familiar and powerful tools of linear algebra. For example, we can take the trace of a linear map, and the identity $\mathrm{tr}(ABA^{-1}) = \mathrm{tr}(B)$ tells us that the trace of a representation (called the character of the representation) is constant on the conjugacy classes of a group. We can also consider determinants, characteristic polynomials, dual vector spaces (or the dual representation), dimension and many more of our favourite concepts from linear algebra.
Representations are certainly powerful:

There are theorems (for example, concerning Frobenius groups) whose only known proofs use representation theory.
There are groups (such as $\mathrm{Gal}(\overline{\mathbb Q}/\mathbb Q)$) which we only really know how to study via their representations.

A:

Rough answer. Before there was an abstract definition of "Lie group" mathematicians studied groups of matrices. A Lie group is a generalization of a group of matrices. It turns out that one way to try to understand a  Lie group is to look at all the ways to "represent" it as a group of matrices. Each mapping in the definition in your question does just that.

