In 1902-3, he added a sumptuous dining room to the back of the Clubhouse fitting it with stained glass windows and an early form of air conditioning. He ordered marble statuary for the Italian gardens in the northeast corner of Congress Park. The elegant atmosphere made the cream of society feel welcome to bet their money on the Clubhouses’s many games of chance. Canfield was recognized as the King of the Gamblers; Saratoga Springs was seen as the American Monte Carlo. In Saratoga Springs, he kept the Clubhouse going until 1907. The clientele during this period included not only members of wealthy families like the Whitneys, Vanderbilts and J. P. Morgan's, but gambling legends like Diamond Jim Brady and John Warne "Bet-a-Million" Gates, and prominent entertainers like Gate's girlfriend Lillian Russell and impresario Florenz Ziegfeld. This socially distinctive era, regarded as the city's golden age, ended in 1907 when reformers succeeded in banning gambling in the city. Canfield retired and sold the hotel and grounds to the city four years later, in 1911. The Pure Food and Drug Act hurt sales of bottled Saratoga Water, and the year after buying from Canfield, the city bought the Congress Hall hotel and bottling plant and demolished them. The Canfield Casino in Saratoga Springs now houses the Historical Society. Canfield owned a number of fashionable gambling houses in New York, Rhode Island, Saratoga Springs and Newport. In December 1902 his New York clubhouse at 5 East 44th Street in Manhattan was raided, and Canfield, who was not actually arrested, escaped to England where he lived for the next four and half months. Canfield's friend and fellow businessman, Charles Lang Freer, introduced Canfield to James Abbott McNeill Whistler in 1899. As a personal friend and patron of Whistler's, Canfield possessed the second largest and most important Whistler collection in the world prior to his death. A few months before his death, he sold his collection of etchings, lithographs, drawings and paintings by Whistler to the American art dealer Roland F. Knoedler for $300,000. Three of Canfield's Whistler paintings hang in the Frick Museum in New York City. Canfield came to own a number of paintings by Whistler including La Mère Gérard (2) (YMSM 27), Symphony in Grey and Green: The Ocean (YMSM 72), Arrangement in Black: Portrait of F. R. Leyland (YMSM 97), Arrangement in Brown and Black: Portrait of Miss Rosa Corder (YMSM 203), Nocturne in Blue and Silver: The Lagoon,Venice (YMSM 212), Arrangement in Black and Gold: Comte Robert de Montesquiou-Fezensac (YMSM 398), Rose et or: La Napolitaine (YMSM 505) and Grey and Gold: The Golden Bay, Ireland (YMSM 537).