He held the composite living for the rest of his life. Stebbing died at St. James's parsonage, Hampstead Road, London, on 22 September 1883, and was buried on 27 September in Kensal Green Cemetery, London. Stebbing's first poem, "The Wanderers", was printed at the end of 1817 and circulated to friends. In August 1818 he published "Minstrel of the Glen and other Poems". After his marriage he wrote for money. Stebbing was connected with the "Athenæum" magazine from its foundation. He was engaged by James Silk Buckingham in its early planning. A notice by him of Renn Hampden work on "Butler's Analogy, or Philosophical Evidences of Christianity" was the opening review in the first number of 2 January 1828, and his article on Richard Whately's "Rhetoric" led the second number. After three or four issues he became the working editor. A moderate churchman, inclining to evangelicalism, in 1847 Stebbing published "A Letter to Lord John Russell on the Established Church", in which he argued for a reform of the system of patronage. In 1848 he owned and edited the "Christian Enquirer and the Literary Companion", of which seven issues were published. Stebbing's major works were:    He wrote a continuation to the "Death of William IV", of David Hume and Tobias Smollett's "History of England". His "Essay on the Study of History", which appeared as an addition to Hume, was published separately in 1841. From 1834 to 1836 Stebbing edited, with Richard Cattermole, thirty volumes of the "Sacred Classics" of England. He was editor of the "Diamond Bible" (1834, 1840, and 1857), "Diamond New Testament" (1835), "Charles Knight's Pictorial Edition of the Book of Common Prayer" (1838–1840), Tate and Brady's "Psalms" (1840), "Psalms and Hymns, with some original Hymns" (1841), and many modern theological works. He also edited the works of Josephus (1842) and of John Bunyan, John Milton's "Poems" (1839 and 1851), and Daniel Defoe's "Journal of the Plague Year" (1830), and "Robinson Crusoe" (1859). Stebbing knew many literary figures.