Is there a possible distortion of XeF6 from Oh point group to reduced symmetry?
Below the molecular orbital diagram for $\ce{XeF6}$ (from What is the molecular structure of xenon hexafluoride?)
I am trying to figure out if a distortion from $O_h$ can take place and to which (lower symmetry) point group it would be. I know that it will involve the HOMO $a_g$ interacting with the $t_u$ orbital (bonding). Using a descent of symmetry table for $O_h$, I have found that the $a_g$ must remain completely symmetric. The part I am confused about is, looking at the descent of symmetry table, I am looking for an $a_g$ component of $t_u$ in a point group. I am using the following table:
$t_{1u}$ does not $a_{1u}$ in the table. Literature actually suggests $C_{3v}$ as the resulting distortion, but how would I know this when it isn't in the table? and I was wondering where I am going wrong in my approach?
Also, I am curious where the subgroups fit into the table, as $C_{3v}$ is not there but what does this mean?
inorganic-chemistry symmetry group-theory
gamma1gamma1
Now, there are a few things that must be stated very clearly. Firstly, it pays to be very careful when you are talking about point groups. It's careless (and confusing to others) to write $\mathrm{T_{2u}}$ when you mean $\mathrm{T_{1u}}$, or to write $\mathrm{A_{g}}$ when you mean $\mathrm{A_{1g}}$, so please pay attention to this next time, regardless of whether you are writing online or submitting a piece of work. Good communication in science has to be precise and clear!
Secondly, let's use this reduced descent of symmetry table (I have removed some columns and some rows, which aren't relevant now)
$$\begin{array}{ccc} \hline O_\mathrm h & T_\mathrm d & D_\mathrm{3d} \\ \hline \mathrm{A_{1g}} & \mathrm{A_1} & \mathrm{A_{1g}} \\ \mathrm{T_{1u}} & \mathrm{T_2} & \mathrm{A_{2u} + E_u} \\ \hline \end{array}$$
This tells you that if you decrease the symmetry of the system from $O_\mathrm h$ to $T_\mathrm d$, then anything that previously transformed as $\mathrm{A_{1g}}$ will now transform as $\mathrm{A_1}$. Likewise, anything that previously transformed as $\mathrm{T_{1u}}$ will now transform as $\mathrm{T_2}$.
If you are trying to look for potential mixing of orbitals in the lowered symmetry, then you have to look what the irreps in the lowered symmetry are. From what you have written, I sense that you're trying to "look for an $\mathrm{A_{1g}}$ component", which isn't what you should be looking for, because something that transformed as $\mathrm{A_{1g}}$ in $O_\mathrm h$ symmetry may not necessarily transform as "$\mathrm{A_{1g}}$" in the lower symmetry. Point in case: $T_\mathrm d$.
If you are considering the viability of a distortion to $T_\mathrm d$, then you should be looking for an $\mathrm{A_1}$ component in the second row, not $\mathrm{A_{1g}}$. Of course, $\mathrm{T_2}$ does not contain $\mathrm{A_1}$, so the distortion to $T_\mathrm{d}$ isn't favourable.
Now, in $C_\mathrm{3v}$ the correlations are $\mathrm{A_{1g}} \to \mathrm{A_1}$ and $\mathrm{T_{1u}} \to \mathrm{A_1} + \mathrm{E}$. If we add this column to the reduced table above
$$\begin{array}{cccc} \hline O_\mathrm h & T_\mathrm d & D_\mathrm{3d} & C_\mathrm{3v} \\ \hline \mathrm{A_{1g}} & \mathrm{A_1} & \mathrm{A_{1g}} & \color{red}{\mathrm{A_1}} \\ \mathrm{T_{1u}} & \mathrm{T_2} & \mathrm{A_{2u} + E_u} & \color{blue}{\mathrm{A_1 + E}} \\ \hline \end{array}$$
then it is clear that because the $\color{blue}{\text{blue}}$ $\mathrm{A_1 + E}$ shares one irrep in common with the $\color{red}{\text{red}}$ $\mathrm{A_1}$ component, the HOMO–LUMO mixing is allowed in $C_\mathrm{3v}$ symmetry, and this provides a qualitative argument for why distortion to $C_\mathrm{3v}$ symmetry is favoured.
Again, to drive home the point, you shouldn't be looking for whether the $\color{blue}{\text{blue}}$ bit contains $\mathbf{A_{1g}}$. There is no $\mathrm{A_{1g}}$ irrep in $C_\mathrm{3v}$, so of course you won't find it! You should be asking whether the $\color{blue}{\text{blue}}$ bit contains the $\color{red}{\text{red}}$ bit.
How to get those correlations, is quite a fair question. Well, I got it from this diagram in Orbital Interactions in Chemistry, 2nd ed. by Albright et al. The correlation diagram 14.4 should be self-explanatory, hopefully (you can ignore the pictures of the MOs, 14.5 and 14.6).
But if you want to obtain it without flipping through every book in the library, this is how you need to do it. Because high-symmetry point groups like $O_\mathrm h$ contain tens and tens of subgroups, books don't usually list every single possible correlation to lower symmetry, since there are far too many possibilities. The table you were given has a bunch of omitted correlations, listed under "other subgroups" (the implication being, "other subgroups that I couldn't be bothered to put in"). You'll notice that $C_\mathrm{3v}$ is one of those "other subgroups" that weren't explicitly given.
The strategy therefore is to carry out the correlation in two steps. First you go from $O_\mathrm h$ to $D_\mathrm{3d}$, and then you go from $D_\mathrm{3d}$ to $C_\mathrm{3v}$. The tables you are using don't provide that, but a more complete list of tables is provided in the Appendixes of Molecular Vibrations by Wilson et al. The $D_\mathrm{3d}$ to $C_\mathrm{3v}$ correlation is given as (again I removed the irrelevant rows)
$$\begin{array}{cc} \hline D_\mathrm{3d} & C_\mathrm{3v} \\ \hline \mathrm{A_{1g}} & \mathrm{A_1} \\ \mathrm{A_{2u}} & \mathrm{A_1} \\ \mathrm{E_u} & \mathrm{E} \\ \hline \end{array}$$
so now the correlation from $O_\mathrm h$ to $C_\mathrm{3v}$ is easy: $\mathrm{A_{1g}}$ in $O_\mathrm h$ goes to $\mathrm{A_{1g}}$ in $D_\mathrm{3d}$, which in turn goes to $\mathrm{A_1}$ in $C_\mathrm{3v}$. Likewise, $\mathrm{T_{1u}}$ in $O_\mathrm h$ goes to $\mathrm{A_{2u} + E_u}$ in $D_\mathrm{3d}$, which in turn goes to $\mathrm{A_1 + E}$ in $C_\mathrm{3v}$.
orthocresol♦orthocresol
Not the answer you're looking for? Browse other questions tagged inorganic-chemistry symmetry group-theory or ask your own question.
What is the molecular structure of xenon hexafluoride?
How are point group character tables typeset correctly?
Non-bonded orbitals in water
Potential wells and symmetry in absorption spectroscopy
Molecular orbital diagram and irreducible representations for dinitrogen
How to systematically find the reducible representation of a molecule? (eg: XeF4)
How do I have to interpret the input to the OCC flag in Molpro?
Vibronic coupling: How do I determine the new point group after vibration?
Sulfuric acid symmetry point group
Point group from mono to bulk MoS2