Question: In [J.J. Sakurai's](https://en.wikipedia.org/wiki/J._J._Sakurai) *[Modern Quantum Mechanics](https://en.wikipedia.org/wiki/J._J._Sakurai#Textbooks)*, the same operator $X$ acts on both, elements of the [ket](https://en.wikipedia.org/wiki/Bra%E2%80%93ket_notation#Ket_notation) space and the [bra](https://en.wikipedia.org/wiki/Bra%E2%80%93ket_notation) space to produce elements of the ket and bra space, respectively. Mathematically, an operator is simply a map between two spaces. 

So, how can *the same operator* act on the ket as well as the bra space?

Answer: 
This is what computer scientists would call [(ad-hoc) polymorphic](https://en.wikipedia.org/wiki/Polymorphism_(computer_science)) or “overloaded” functions: basically, an operator $X$ on the Hilbert space $\mathcal{H}$ is not just one function $X: \mathcal{H}\to\mathcal{H}$, but a family of two functions
$$
 X = \{ X\_{\mathcal{H}}, \quad X\_{\mathcal{H}^\ast} \}
$$
with
$$
 X\_{\mathcal{H}}: \mathcal{H}\to\mathcal{H}, \quad X\_{\mathcal{H}^\ast}: \mathcal{H}^\ast\to\mathcal{H}^\ast
$$
Since a Hilbert space's dual space (the “space of bra-vectors”) is a different space from $\mathcal{H}$ itself, it's always unambiguous which of the two functions you mean when applying $X$ to either a ket or a bra.

And the definition of $X\_{\mathcal{H}^\ast}$ follows directly from the one of $X\_{\mathcal{H}}$ vice versa. This is easy to see in one direction:
$$
 \bigl(X\_{\mathcal{H}^\ast}(\langle f|)\bigr)(|v\rangle) = \langle f| \bigl(X\_\mathcal{H}(|v\rangle)\bigr)
$$
In the other direction, we need to invoke the [Riesz representation theorem](https://en.wikipedia.org/wiki/Riesz_representation_theorem): for any $|v\rangle \in \mathcal{H}$, let
$$\begin{align}
 \langle X\_v^\ast| & \in\mathcal{H}^\ast
 \\ \langle X\_v^\ast| &:= X\_{\mathcal{H}^\ast}\bigl(\langle . , v\rangle\bigr).
\end{align}$$
where by $\langle . , v\rangle$ I mean the function
$$\begin{align}
 v^\ast & \in \mathcal{H}^\ast
 \\ v^\ast(w) & := \langle w,v\rangle
\end{align}$$
(This is now *not* an application of dual-vectors, but the actual scalar product that comes with the Hilbert space!)

Then Riesz tells us that this corresponds to one unique element $X\_v\in\mathcal{H}$, so we can define
$$
 X\_{\mathcal H}(|v\rangle) := X\_v.
$$
And because physicists are lazy (good trait, although some overdo it), they avoid all the “obvious” parentheses etc., and just assume the reader will know how the operator needs to be applied, which space it lives in, etc.. Swapping between a bra- and a ket-version of a state implicitly always invokes the Riesz representation theorem, but this is seldom talked about.
Just think of the operator as a matrix (as per your "matrix-elements" tag). Then it operates to the right on a column-vector $\left|\mbox{ket}\right>$, and to the left on a row-vector $\left<\mbox{bra}\right|$