I've built an API that delivers live data all at once when a user submits a search for content. I'd like to take this API to the next level by delivering the API content to a user as the content is received instead of waiting for all of the data to be received before displaying. 
How does one go about this?
From your explanation 

We're pulling our data from multiple sources with each user search.
  Being directly connected to the scrapers for those sources, we display
  the content as each scraper completes content retrieval. I was
  originally looking to mimic this in the API, which is obviously quite
  different from traditional pagination - hope this clarifies.

So you in your API, you want to 

take query from user 
initiate live scrapers
get back the data to the user when scrapers finish the job !
(correct me if im wrong)

My Answer
This might feel little complicated, but this is the best one I can think of.
1. When user submits the query: 
1. Initiate the live scrapers in to celery queue (take care of the priority). 
2. Once the queue is finished, get back to the user with the information you have via sockets(this is how facebook or any website sends users notifications`. But in your case you will send the results html data in the socket.
3. Since you will have the data already, moved into the db as you scraped, you will can paginate it like normal db.
But this approach gives you a lag of a few seconds or a minute to reply back to the user, meanwhile you keep theuser busy with something on the UI front.
Building an API