Scottish Health Secretary Alex Neil said the move would help meet the cost of so-called orphan medicines.
These are drugs used to treat illnesses affecting fewer than one in 2,000 people.
The new fund will only cover the cost of medicines whose routine use has been rejected by the Scottish Medicines Consortium (SMC).
This could include drugs such as Kalydeco - also known as ivacaftor - which was rejected for use in Scotland by the SMC on Monday largely on the grounds of cost.
The NHS in England decided to provide funding for Kalydeco from 1 January this year.
Mr Neil said it was only right that Scottish patients with rare conditions had access to innovative medicines which were clinically justified, and that they were not disadvantaged due to the very high cost of these treatments.
He added: "I am therefore pleased to confirm today that the Scottish government will establish a fund which will ensure that the cost of successful new individual patient treatment requests for orphan medicines are met."
MSPs heard last year that the Individual Patient Treatment Requests (IPTR) system, which patients can use to get drugs not yet approved for general use by the SMC, is an "inequitable process" that favours young, middle-class and articulate applicants over more suitable vulnerable people who cannot speak up for themselves.
In November, Mr Neil said a review would be carried out to look at current systems for making new medicines available across the NHS in Scotland.
A separate review is examining if the IPTR arrangements can be improved, with the new fund established after interim advice from Professor Charles Swainson - a former medical director at NHS Lothian who is now an independent consultant.
The £21m is available from March through to April next year and will help cover the cost of successful requests under the IPTR system.
Mr Neil said: "This fund bridges the period to the establishment of next year's value-based pricing for medicines and any changes that are made following the completion of the ongoing access to new medicines review.
"We recognise that the Scottish Medicines Consortium (SMC) is globally respected and has the fastest and most efficient medicine review process anywhere in the UK but that for some individual patients with rare conditions, there is a need for further support. This new fund will complement it by making the IPTR processes fairer."
Labour's health spokeswoman Jackie Baillie welcomed the announcement but said it showed up the problems in the system.
She said: "We were promised a full review of how drugs are approved, but this is merely a sticking plaster on the problem.
"If nothing else, it is an acceptance that the current process is deeply flawed and is literally buying time while the SNP try to sort out the mess."
Kalydeco treats people who have the G551D cystic fibrosis mutation, which affects about 11% of people with cystic fibrosis in Scotland, compared with about 4% in the UK, according to the Cystic Fibrosis Trust.
About 80 people in Scotland would have been eligible for treatment with Kalydeco, the trust estimates.
A spokesman for the SMC said it was aware that the decision not to recommend the drug for use in the NHS would "come as a disappointment".
He said Kalydeco was a "promising new medicine" but claims that it could increase life expectancy by as much as 18 years were "highly uncertain".
The spokesman said: "This weakness, combined with the very high cost of the medicine, were significant factors in the SMC decision.
"The cost of ivacaftor in NHS Scotland would be around £180,000 per year for each patient for life. The recent decision to make ivacaftor available in England is based on a discounted price that has not been offered to NHS Scotland.
"SMC aims to make decisions on the cost-effectiveness of new medicines in a way that is fair for all patients with all conditions and for the sake of patients with cystic fibrosis, we would welcome the opportunity to evaluate an improved case for this new medicine."
Ed Owen, chief executive of the Cystic Fibrosis Trust, welcomed the Scottish government's decision to make "additional funding available to enable Kalydeco to be made available to those that need it".
He said: "We now appeal to Vertex, the manufacturers of Kalydeco, and the NHS in Scotland to work together to reach a fair and affordable solution as soon as possible.
"The CF Trust has campaigned for all people with the G551D mutation of cystic fibrosis who are eligible to receive Kalydeco to be able to receive it as soon as possible."
A £21m fund has been launched to help improve access to treatment for people with rare medical conditions.