The low-budget Ouija, starring Bates Motel actress Olivia Cooke, made $20m (Â£12.4m) between Friday and Sunday, according to studio estimates.
That put it ahead of John Wick, Keanu Reeves' latest thriller, which made $14.2m (Â£8.8m) over the same period.
Last week's top film, Brad Pitt's World War II tank drama Fury, fell to three.
Made for just $5m (Â£3.1m), Ouija's success is testament to US moviegoers' appetite for modestly-budgeted supernatural fare without big-name stars.
Blumhouse Productions, the company behind the film, has enjoyed similar success with the Insidious, Purge and Paranormal Activity series.
Meanwhile, John Wick, in which Reeves plays a retired assassin out for revenge, exceeded industry expectations with its second place opening.
This week's top five was rounded out by David Fincher's Gone Girl and animated title The Book of Life.
Next week's chart is likely to be dominated by Interstellar, the latest sci-fi epic from British director Christopher Nolan.
Matthew McConaughey and Anne Hathaway star in this tale of deep space exploration, which has its UK premiere in London on Wednesday.

The hoard, which includes silver pennies dating back to the 10th and 11th centuries, was discovered by Walter Hanks in Llandwrog in March.
National Museum Wales said some of the coins were made under the ruler Sihtric Anlafsson and were a type rarely found on the British mainland.
It said they were likely to have been hidden or lost between 1020 and 1030.
Dr Mark Redknap, of the department of history and archaeology at National Museum Wales, said the mixed nature of the collection showed bullion played an active role in the 11th Century economy and gave an idea of the wealth of Gwynedd at the time.
The museum now hopes to buy the coins and put them on display.

The New York-based singer won the award for her Orchestre Philharmonique du Luxembourg collaboration, Sings.
"I want to dedicate this Grammy to all the traditional musicians in Africa, in my country, to all the young generation," Kidjo said.
Her album beat one by Malawi's Zomba Prison Project and South Africa's Ladysmith Black Mambazo.
Kidjo, one of Africa's most prominent musicians, has now won the award for the second consecutive year.
The album merges African song writing and rhythms with European classical instrumentation, a fusion on which Kidjo has repeatedly experimented.
Kidjo described the album as an artistic challenge as traditional African bands follow the lead of the soloist much more closely, unlike Western orchestras that generally play off refined scores.
The singer, who has worked for a long time with Philip Glass, a leading US composer, said Africa was on the rise.
"Africa is positive, Africa is joyful," she said after collecting the award.
The 55-year-old singer added that she has been fighting for a positive image of Africa for a long time and believed music could connect the world and served as a tool for peace.
She later told the BBC: "I have to continue to working...  to open the way for many artists from Africa to come."
African-American rapper Kendrick Lamar won the most prizes on the night with five awards.
The 29-year-old performed The Blacker The Berry, walking on stage as part of a chain gang from inside a prison, before transitioning to Alright, in front of a huge bonfire and African dancers.
His performance ended with the unveiling of a new track with Lamar in front of a map of the African continent and the word "Compton" - his hometown in California - written on it.
A horror film about young people who use a Ouija board to contact the spirit world has topped the North American box office in the run-up to Halloween.
[NEXT_CONCEPT]
Viking coins and ingots found by a metal detectorist in Gwynedd have been declared treasure by a coroner.
[NEXT_CONCEPT]
Benin's Angelique Kidjo has won the best world music album of the year at this year's Grammys in Los Angeles.