The Cows
First and foremost, unless your oil cows only have sloth-like mobility, they will likely have survived fires (natural and synthetic) by simply running away.
To further jump on the "probably not as big a deal as you think" train, fuel-oils (like gasoline and diesel) are atomized, mixed with an oxidizer, compressed, and then ignited. The liquid itself isn't very combustible; it's when the fuel is vaporized (which greatly increases the surface area of fuel exposed to the oxidizer) that it becomes dangerous. You don't have to compress the fuel before burning it, but the energy released from burning a compressed fuel is more easily harnessed to do useful work (just some additional background). Gasoline's flashpoint (essentially the temperature at which it vaporizes) is >-23C, which is why it burns readily when ignited pretty much anywhere on Earth's surface. Diesel doesn't burn nearly as easily; it's flashpoint is >55C, which means it has to be heated up before it will ignite.
Even if a cow was full to bursting with oil (4L), that's actually not very much fuel, and wouldn't burn for very long. According to this, refined fuel oil (which is similar to diesel in consistency but has a higher flashpoint of >90C) has a density of a little less than 1 kg/l, and if spread out over a 10 m diameter circle, the entire 4L would burn up in about 1 second (in your atmosphere, this would be even faster; see Other Thoughts below). Spreading all the cow's oil over a 10 m diameter circle and lighting it up is a little ridiculous, so let's more reasonably assume a 0.5 m diameter circle (someone slashes a cow's udder and the oil spills out on the ground underneath it, then they light it on fire). Even after correcting for the reduced surface area of the burn (the burn rate is proportional to the surface area of the fuel burning), the fire will still only last about 7 minutes (on Earth) without additional fuel. If you've ever lit a camp fire with dry twigs and brush, then tried to throw a log on it, you'll know it doesn't go well--the twigs burn hot and fast, but the log doesn't catch. Without amassing quite the pool of fuel and other combustibles, you'll end up with the same thing from your oil cows. 479
Plus, they probably won't take kindly to having their udders slashed.
It is also important to remember that in order for the oil to burn, it has to be exposed to the oxidizing agent (which is typically oxygen in the atmosphere). Unless your cows absorb oxygen into their oil glands at a prodigious rate, even if the temperature of the cow rises to the oil's flash point, it won't combust. An oil cow caught in a fire would have the water in it's body boil (at 100C) before the oil caught fire. This is essentially the same reason the propane or LP in the rubber hose that goes to the grill doesn't all ignite and cause the tank to explode--since there's no oxygen to react with except at the burner (where the gas meets the atmosphere), it doesn't burn (yes, the positive pressure of the tank compared to the atmosphere is also a part of that equation; it's not a perfect metaphor).
From a logistical standpoint, your oil cows aren't any different from any other livestock throughout the centuries. Sounds like they're a valuable resource, and since they're domesticated, there's vested interest in ensuring that at least a breeding pair survive, which means oil cow farmers will do what they need to in order to protect their herd. Plus, a dead oil cow is only worth the meat (assuming it's edible), byproducts (bone, ivory, etc.), and whatever tiny fraction of that 4L of oil you can squeeze out of its glands. Presumably, they are significantly more valuable alive, and unless the cultures of your planet have a prevailing "scorched earth" policy in all their conflicts, they'll probably see an oil cow herd as part of the spoils of war. Plus, if you're a bandit or raider, why kill what you can conquer and use? Raiding cultures were not generally motivated by a strong desire to kill the people they were raiding; they wanted to take their stuff, wait for the oppressed people to make new stuff, and then come back and take more stuff.
From a completely different perspective, even if your cows produce oil which requires no refining, they themselves do not have to be particularly combustible. If you're really concerned about it, your oil cows might have evolved a particularly flame-resistant skin. Maybe they sweat profusely as a defense mechanism, which makes it really hard for them to be set alight without devoting a lot of time to holding the flame against them. 121