You are essentially right. <PERSON>'s philosophy is basically this: Suppose you know (due to experimental results, say) that the physical system you are studying has symmetries described by a group $G$. There are many examples of this, e.g. rotational symmetry in 3 dimensions which is described by the group $SO(3)$. How can you construct a Hilbert space $\mathcal H$ which describes this quantum mechanical system?
Any symmetry transformation must satisfy two properties:
* It should take physical states to physical states.
* It should preserve probabilities when the transformation is applied to both the initial and final states.
These define a ray transformation. That is to say, by the first condition, we have a map from $G$ to the space of self-maps on the projective Hilbert space $P\mathcal H$ (the space whose points correspond to physical states). The second condition that probabilities are conserved means that any self-map $T:P\mathcal H \rightarrow P \mathcal H$ which is in the image of $G$ must descend from a unitary and linear or antiunitary and antilinear operator $U(T):\mathcal H \rightarrow \mathcal H$. This is the content of <PERSON>'s theorem. 976
With a bit of extra work, you can prove that this indeed defines a homomorphism. You can think of this as a 2-step homomorphism, the first map being from $G$ to the isometry group of $P\mathcal H$ with the Fubini-Study metric (this is the "group of ray transformations" in your language), and the second from there to the group of unitary-and-linear or antiunitary-and-antilinear operators on $\mathcal H$. The first step is physical, while the second is purely mathematical.
The end result is that the Hilbert space must be a projective representation of $G$. This is not an immediately obvious fact; one might have thought that a symmetry could be realized nonlinearly on the Hilbert space, but <PERSON>'s theorem tells us that can't happen. Because ray transformations are not so easy to work with in practice we generally prefer to work with the operators.
You also raise the question of whether we can construct the group $G$ from its (projective) representation. This is not really a totally sensible question as the group $G$ is part of the data of the representation. With that said, under certain conditions if you know all the representations of $G$ it can be possible to reconstruct $G$, though this is a different direction from what <PERSON> is talking about. More precisely, given the set of representations (without any of their data) together with information about how they are related (see Category of Representations), it is possible to reconstruct $G$ if it is a compact topological group (up to a central extension if we include projective representations). This is known as Tannaka-Krein duality. 364