Why does column chromatography not work like it's supposed to?

I'm a few months into working in an organic synthesis lab and I consider myself to have a solid intuition for, well, how chemistry works. I have ran at least 50 columns at this point but no matter what experiment I run, there's always some concepts that seem off.


To start, I don't think my technique is the problem. I know for a fact I am packing the column well. I dry pack but run plenty of solvent cycles through it before loading. I try to load with as little solvent as possible, to the point where the top of the sand is still dry sometimes. I load with the same solvent I use for elution, so it is not too polar in comparison. I run the column slow (like abysmally slow, 2 drips a second) to not "flush" things out. I try to perfect technique and minimize other factors.


But I still run into problems. For example, everyone uses TLC plates to test different solvent systems for maximum separation. If my desired product dot is smack in the middle with Rf of 0.5 on the plate, I'm expecting it to come out within two cycle volumes on the column (1/Rf). But regardless, it seems like it always comes out right after the first cycle and it catches me by surprise. The compounds I'm isolating have very close Rfs, and it would be great if I could at least predict when it will come out so I can switch to smaller collection tubes. I realize that not using the recommended 0.2 Rf means I will need to be very precise with collection but I don't mind. What I DO mind is that the product doesn't even match the tested Rf.


Also, it seems like if I use any solvent system more polar than the recommended 0.2 Rf TLC, the product and all the impurities will wash out together with almost no separation at all. It's like all basic concepts of chromatography have gone out the window. The separation was incredible on the TLC plate, like I'm talking the product at an Rf of 0.5 and the impurities had Rf of 0.2. Then I try a column with 50 mL cycle volume and everything started coming out within 10mL of each other?


Also, collection tube TLC doesn't make much sense either. I take a TLC of a collection tube with the product, with the product showing Rf of, say, 0.5, but the tube also contains an impurity with an Rf of 0.05. How is that even possible? According to my cycle volume calculation (CV \* 1/Rf), an impurity with that Rf should come out many many collection tubes afterwards.


I just want to understand the actual reason why this is happening. I don't mind changing my procedures, but it is rather frustrating to not understand the science.


