Since you would like to avoid mathematics, you can always break down the process into a series of steps.
1. Say you apply an upward force to the atom on the right for a very short time, the atom on the right will gain some momentum. Meanwhile the atom on the left is still stationary during this time. It hasn't felt the effects immediately. Information travels at the speed of light.
2. The right atom will move up. When the right atom moves up, the bond stretches. This causes the atom on the left to experience a component of upward force also a component of force to the right. Essentially, the left atom moves up during this small time at the start. This is expected, you apply a force on an object you at least expect the centre of mass motion to move in the direction of the force, not the opposite.
3. Since your force is off centre of mass then your combined system will also rotate about the center of mass. As I explained above you will also have other components of force not just up and down. 343
The bond is changing orientation with time. Factoring in that the atoms are moving as well. These forces (including the up and down) will contribute the oscillation of the atoms in a 2D plane. This results in rotation.
So after going through this process what we have determined is that centre of mass moves upwards however you will get also get rotation.
The 2nd picture you have drawn in the image is actually in the centre of mass frame of reference. (i.e. you are moving with the centre of mass). In the stationary frame both atoms will ultimately move upwards.
Please let me know if I have answered your question or if you would like clarification.
Edit: Additional sketch for clarification. The image shows the state of the system at different times $t_{3}>t_{2}>t_{1}>0$. At each time step I show the force due the bonding and the velocity that resulted from the prior time. Then I show the new velocity vector.For the next time, I displace the atoms in the direction the new velocity vector is pointing. I do this for a couple of times to demonstrate that you will end up getting rotation. Also I assumed I am working in the centre of mass frame when I sketched these diagram.
Also for $t_{3}$ I accidentally displaced the atoms in the same time step. The velocity directions are correct however the displacement occurs in the next time step. 766