Smokers will be prescribed with nicotine replacement patches to help them cope during their stay at affected hospital sites, but e-cigarettes will not be allowed.
Staff were already banned from smoking within the grounds but there were smoking shelters for patients.
The changes, adopted by other Welsh boards, was approved on Tuesday.
Ahead of the meeting, Dr Sharon Hopkins, public health director of Cardiff and Vale University Health Board, said the proposals had been 18 months in the planning because officials recognised a ban would "not be easy and patients and the public would have to be supported".
She added: "We recognise that this has potential issues and have taken actions to address this.
"There is no point us saying that you can't smoke on our sites full stop - we have to make sure that we are able to support people with their addiction."
Dr Hopkins said patients in hospital currently receive support to cope with any existing addictions and the same help would be offered to smokers.
She said they would be given support from the health board's smoking cessation service and smokers could be prescribed with nicotine replacement patches to help them cope during their stay.
"We know from evidence elsewhere that a total ban is much more likely to decrease the number of cigarettes people smoke and that they are much more likely to try and quit," she said.
E-cigarettes will also be banned because while they are being promoted as aids to stopping smoking Dr Sian Griffiths of the health board's public health team claimed there was little evidence to suggest they worked.
Nicotine therapy will be used instead.
The changes come into force from 1 October.
A full smoking ban at all hospital sites in Cardiff and the Vale of Glamorgan has been given the go-ahead.