Question: I wondered this since my teacher told us about half life of radioactive materials back in school. It seems intuitive to me to think this way, but I wonder if there's a deeper explanation which proves me wrong.

When many atoms are involved, half life can statistically hold, but since decaying of an individual atom is completely random and stateless, can't all the atoms in a 1 kg of matter just decide to decay in the next minute, even if the probability of this event occurring is extremely small?

Answer: 
TLDR: statistical models are models, and thus by definition not a perfect reflection of reality.

Nihar's answer is good but I'm going to tackle it from a different direction.

First off, if we only look at statistical mechanics you can run through the math and of course you will find an extremely small probability. You might stop there. But statistical mechanics uses statistical models, and all models are wrong. They make assumptions and necessarily simplify reality to solve complicated problems. There could very well be some physical processes unaccounted for in statistical mechanics that negate any possibility of such a rapid decay.

A classic example is having a room and figuring out the probability that all the oxygen all of a sudden is only in one half of the room. From a stat mechanics standpoint, it's basically the probability of flipping a fair coin an unimaginably large number of times and having them all land the same way. But in reality, the unimaginably small number you would calculate wouldn't actually be correct, because the assumptions made by your model wouldn't perfectly reflect reality (particles interact with each other, for one). Much like the ideal gas law, these things are useful but can completely fail if you deviate too far from the assumptions made. This is true of all statistical models, of course.

**So if we assume that the stat model of half life is a completely accurate representation of reality, the answer to your question is technically yes. Of course we know it's not, so that leads me to my final point.**

There's also a heavy philosophical component to these sorts of questions since we are dealing with probabilities that are so small they are effectively 0. If someone flips a coin a billion times and it lands tails every time no one is going to think it's a fair coin, because it's obviously not\*. You could also consider state of the art cryptography. The odds of successfully randomly guessing a key is so low that for all intents and purposes it is 0. Or imagine watching a video of a bunch of shattered glass forming into a vase. Your conclusion wouldn't be 'see ya thermodynamics, wouldn't want to be ya', it would be 'I'm watching a video of a vase shattering in reverse'. Yes, there are technically tiny probabilities associated with these events but it is so small that saying they are technically possible is more of a philosophical statement than anything else.

\* The idea of a fair coin is a rabbit hole on its own. How do you determine that a coin is fair? By tossing it a bunch of times and observing a nearly equal number of tails and heads. If it deviates too much from 50/50, we declare it to be biased. But of course no matter what outcome we observe, there's always a chance it was a fair coin, so technically we can never know for sure. In order to make use of statistics then, we must arbitrarily pick a cut off point for random chance. Usually this is 2 sigma, maybe 3. CERN uses 5 sigma for new particle detection but again, this is arbitrary. Applied statistics is very much an art as much as it a branch of math.
One thing to keep in mind is that this is not only a statistics question and the analogy of atoms decaying and flipping coins can be misleading.

For example, [uranium 235](https://en.wikipedia.org/wiki/Uranium-235) has a half life of more than 700 million years, but when brought in the right configuration (close packed) and in the right amount (above critical mass), it decays practically in an instant... Simply because one atom decaying can trigger another to decay and so on in a chain reaction.

So, if you can assume that all decays happen independently of each other, then the answers based purely on statistics are valid. If more physics than statistics is involved, then it depends on the exact material, i.e. what material, is it pure, in what configuration, etc.