The drugs are normally used to treat people with mental illness
Increasing numbers of people are using prescription drugs like Ritalin to boost alertness and brain power, say experts.
Up to a fifth of adults, including college students and shift workers, may be using cognitive enhancers, a poll of 1,400 by Nature journal suggests.
Neuropsychologist Professor Barbara Sahakian of Cambridge University said safety evidence is urgently needed.
Experts gather to debate this topic at a meeting in London on Monday evening.
Professor Sahakian's own work shows 17% of students in some US universities admit to using the stimulant Ritalin (methylphenidate) - a drug designed to treat hyperactive children - to maximise their learning power.
One in five of the 1,400 people who responded to the Nature survey said they had taken Ritalin, Provigil (modafinil) or beta-blockers for non-medical reasons. They used them to stimulate focus, concentration or memory.
Of that one in five, 62% had taken Ritalin and 44% Provigil - a drug normally prescribed to alleviating daytime tiredness in people suffering from the rare sleep disorder narcolepsy.
Most users had somehow obtained their drugs on prescription or else bought them over the internet.
Although these are only snapshots of use, Professor Sahakian says it does suggest these drugs are becoming more popular.
Professor Sahakian said given the increasing use of these drugs outside of their intended clinical setting, safety trials were urgently needed.
"We do not really have long-term efficacy and safety data in healthy people. These are studies that really need to be done.
"The use of these cognitive enhancing drugs is spreading to younger and younger people. That's a concern.
"Methylphenidate does have substantial abusive potential so we have to be worried about substance abuse problems and the use of these drugs in the developing brain in children."
John Harris, professor of bioethics at the University of Manchester said people should be allowed to make their own minds up about these drugs.
He said: "If these cognitive enhancing drugs make our lives better and make us better able to concentrate and better able to perform, this would surely be a good thing."
The debate will be heard at Kings Place, London.