Question: In [J.J. Sakurai's](https://en.wikipedia.org/wiki/J._J._Sakurai) *[Modern Quantum Mechanics](https://en.wikipedia.org/wiki/J._J._Sakurai#Textbooks)*, the same operator $X$ acts on both, elements of the [ket](https://en.wikipedia.org/wiki/Bra%E2%80%93ket_notation#Ket_notation) space and the [bra](https://en.wikipedia.org/wiki/Bra%E2%80%93ket_notation) space to produce elements of the ket and bra space, respectively. Mathematically, an operator is simply a map between two spaces. 

So, how can *the same operator* act on the ket as well as the bra space?

Answer: 
The action (to the right) on the ket space naturally induces an action (to the left) on the bra space. The bra space is the dual space of the ket space (that is the space of linear functionals over the kets). We can simply define $\left< \psi \right| X$ by its action on kets (or, since it's linear, on a basis of ket space and linear extension):
$$ \big(\left< \psi \right| X\big) \left| \phi \right> := \left< \psi \right| \big( X \left| \phi \right>\big).$$
Such definitions are also common in pure mathematics. If you want to be pedantic, you can use some notation, e.g. $\iota(X)$, for die induced operator on the dual space.
Just think of the operator as a matrix (as per your "matrix-elements" tag). Then it operates to the right on a column-vector $\left|\mbox{ket}\right>$, and to the left on a row-vector $\left<\mbox{bra}\right|$