Here's a supplement to ACuriousMind's excellent answer, which:
* Goes into more depth about the meaning of "walking in the direction specified by [an] operator."
* Connects this to the more familiar operation of exponentiating a complex number.
* Gives another take on the physical meaning of exponentiation.
Sorry it's so long! If you don't feel like reading the whole thing, you might want to skip to the last section, which doesn't depend too much on the earlier ones.
Let's start out looking at the Lie group $U(1)$, the group of rotations of the complex plane. Geometrically, $U(1)$ looks like a circle. In fact, it's natural to identify $U(1)$ with the unit circle in $\mathbb{C}$, because if $a$ is a complex number with $|a| = 1$, the transformation "multiply by $a$" is a rotation about the origin—the rotation that sends $1$ to $a$. The identity $\mathbf{1} \in U(1)$ corresponds to the point $1$ on the unit circle, because "multiply by $1$" is rotation by zero radians.
As ACuriousMind said, the Lie algebra of a Lie group can be defined as the tangent space at the identity—the space of possible velocities of paths through the identity. If you think about $U(1)$ as the unit circle in $\mathbb{C}$, you can see this tangent space very explicitly: it's the line tangent to the unit circle at $1$, which can be naturally identified with the "imaginary line" $i\mathbb{R}$.
Now, here's a cool fact about Lie groups: if you pick a velocity at the identity element, it naturally extends to a velocity field defined everywhere in the group! Here's how.
If you're walking along a Lie group $G$, and you find yourself getting lonely, you can imagine you have a twin walking a path which is just like yours, but "translated" by an element $g$ of $G$: when you're at $h$, your twin is at $gh$. Whenever you walk through the identity, your twin walks through $g$, so your velocity at the identity corresponds to a twin velocity at $g$. The connection between your motion and your twin's motion gives a correspondence between the tangent spaces $T_{\mathbf{1}}G$ and $T_gG$.
If you're still lonely, you can imagine you have a twin for every element of $G$, all walking with you in lockstep. 499
When you're at the identity, your $g$ twin is at $g$, your $g'$ twin is at $g'$, and so on—you have a twin at every point in $G$. Thus, your velocity at the identity corresponds to a velocity at every point in $G$. Through your twins, your velocity at the identity is extended to a velocity field defined everywhere in the group.
A velocity field that comes from a bunch of people walking in lockstep like this is called a left-invariant vector field. Using this vocabulary, we can restate the cool fact from before by saying that...
In a Lie group, picking a velocity at the identity is the same as picking a left-invariant vector field.
For a concrete example, let's see how a velocity at the identity of $U(1)$ extends to a velocity field defined everywhere on $U(1)$:
As you might expect, the left-invariant vector fields on $U(1)$ are precisely the rotation-symmetric vector fields on the unit circle.
Now, let's say you have an element $v$ of the Lie algebra $\mathfrak{u}(1)$, which we just saw is the same as having a left-invariant vector field on $U(1)$. If you're standing at the identity in $U(1)$, and you're bored, you can entertain yourself by trying to walk so that your velocity always matches the velocity field $v$. It turns out there's always exactly one way to do this, so if you do it for time $t$, you'll always end up at the same place. This group element is called $\exp_t v$. You can think of $\exp_t$ as a function that turns <PERSON> algebra elements into <PERSON> group elements; it's called the Lie group exponential. 499