Answers.Body: <p>First, you'll want to make sure you're on the latest FeedMe release as a bug around this was recently fixed: <a href="https://github.com/craftcms/feed-me/issues/694" rel="nofollow noreferrer">https://github.com/craftcms/feed-me/issues/694</a></p>
  <p>You might also need a plugin that hooks into the <code>EVENT_AFTER_FETCH_FEED</code> event to do some offset transformation based on the tokens passed in.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-09T03:15:32.827Z
Answers.LastActivityDate: 2020-10-09T03:15:32.827Z
Answers.UserDisplayName: Brad Bell
Answers.Body: <p>I'm the one who posted that bug report regarding cursor pagination.  I just wanted to share a simple version of the code I'm using in case it's useful for anyone. I'm using Airtable so you can see I'm only running the function if the feed source is an Airtable URL.</p>
  <p>For reference, I'm getting an offset value like <code>abcdefg</code> from the API and I'm changing it to <code>https://my-api-url.com/feed1/abcdefg</code> to get the next page of results.</p>
  <p>You can run this from a module like the one you can generate at <a href="https://pluginfactory.io/" rel="nofollow noreferrer">https://pluginfactory.io/</a></p>
  <pre><code>use craft\feedme\events\FeedDataEvent;
  use craft\feedme\services\DataTypes;
  use yii\base\Event;
  
  
  class Module extends \yii\base\Module
  {
      public function init()
      {
        Craft::setAlias('@modules', __DIR__);
        if (Craft::$app-&gt;getRequest()-&gt;getIsConsoleRequest()) {
            $this-&gt;controllerNamespace = 'modules\\console\\controllers';
        } else {
            $this-&gt;controllerNamespace = 'modules\\controllers';
        }
  
        parent::init();
  
        /* Changes the Airtable API pagination key on the fly */
  
        Event::on(DataTypes::class, DataTypes::EVENT_AFTER_FETCH_FEED, function(FeedDataEvent $event) {
          $response = $event-&gt;response;
          $feedUrl = $event-&gt;url;
          
          /* Only run if this is an Airtable feed with a response */
          if ( strpos($feedUrl, &quot;airtable&quot;) &amp;&amp; isset($response['data']) ) : 
            $data = json_decode($response['data']);
            
            /* We only need to update the event if there's an offset key */
            if ( isset($data-&gt;offset) ) {
  
              /* Break the URL up into it's components */
              $url = parse_url($feedUrl);
              $baseUrl = $url['scheme'] . '://' . $url['host'] . $url['path'];
              parse_str($url['query'], $params);
  
              /* Update the current query with the new offset value */
              $params['offset'] = $data-&gt;offset;
  
              /* Rebuild the pagination URL */
              $data-&gt;offset = $baseUrl . '?' . http_build_query($params, $enc_type = PHP_QUERY_RFC3986);
  
              /* Return the data to the event */
              $response['data'] = json_encode($data);
              $event-&gt;response = $response;
            }
          endif;
        });
      }
  }
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-15T04:28:20.630Z
Answers.Text: Thanks for sharing!
Answers.UserDisplayName: Brad Bell
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-23T18:02:59.353Z
Answers.Text: I just want to say how much I appreciate this! It's saved me hours of time.
Answers.UserDisplayName: dmatthams
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-23T22:56:56.723Z
Answers.Text: @dmatthams - Awesome, glad to hear it!
Answers.UserDisplayName: Dalton Rooney
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-13T20:17:39.473Z
Answers.LastActivityDate: 2020-10-13T20:17:39.473Z
Answers.UserDisplayName: Dalton Rooney
Body: <p>Has anyone used a pagination token when paginating Feedme? I currently use Firestore and they don't use a full pagination URL but a token you append onto your next URL.</p>
  <p>Initial URL
  <code>https://firestore.googleapis.com/v1/projects/YOUR_PROJECT_ID/databases/(default)/documents/cities/LA</code></p>
  <p>Pagination URL
  <code>https://firestore.googleapis.com/v1/projects/YOUR_PROJECT_ID/databases/(default)/documents/cities/LA?pageToken=Ab7e2991jej91od727</code></p>
  <p>In the Response</p>
  <pre><code>&quot;nextPageToken&quot;: &quot;AFTOeJwxKBJYSXXPpKyhOmB4_-SifKCDzUH0mhJchn1u5hknl3iutUKaPez8OVyR99PZM2014ZU1EpZt4GnUHwHBnwM0P1B9Z-yDhTyqTrmsVt4SIffOXCyxq1Yl7iZQJ0c1ltGHpuV9KWgUH6mvl6rHdCA&quot;
  </code></pre>
  
Tags: <plugin-feedme>
Title: Feedme Pagination by Token
ThreadText: # Feedme Pagination by Token
  
  (Asked by: username_0 on 2020-10-07)
  
  Has anyone used a pagination token when paginating Feedme? I currently use Firestore and they don't use a full pagination URL but a token you append onto your next URL.
  
  Initial URL
  `https://firestore.googleapis.com/v1/projects/YOUR_PROJECT_ID/databases/(default)/documents/cities/LA`
  
  Pagination URL
  `https://firestore.googleapis.com/v1/projects/YOUR_PROJECT_ID/databases/(default)/documents/cities/LA?pageToken=Ab7e2991jej91od727`
  
  In the Response
  
  ```
  "nextPageToken": "AFTOeJwxKBJYSXXPpKyhOmB4_-SifKCDzUH0mhJchn1u5hknl3iutUKaPez8OVyR99PZM2014ZU1EpZt4GnUHwHBnwM0P1B9Z-yDhTyqTrmsVt4SIffOXCyxq1Yl7iZQJ0c1ltGHpuV9KWgUH6mvl6rHdCA"
  
  ```
  
  **Tags:** plugin-feedme
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2020-10-13. Score: 4 | Accepted answer
  
  I'm the one who posted that bug report regarding cursor pagination. I just wanted to share a simple version of the code I'm using in case it's useful for anyone. I'm using Airtable so you can see I'm only running the function if the feed source is an Airtable URL.
  
  For reference, I'm getting an offset value like `abcdefg` from the API and I'm changing it to `https://my-api-url.com/feed1/abcdefg` to get the next page of results.
  
  You can run this from a module like the one you can generate at <https://pluginfactory.io/>
  
  ```
  use craft\feedme\events\FeedDataEvent;
  use craft\feedme\services\DataTypes;
  use yii\base\Event;
  
  class Module extends \yii\base\Module
  {
      public function init()
      {
        Craft::setAlias('@modules', __DIR__);
        if (Craft::$app->getRequest()->getIsConsoleRequest()) {
            $this->controllerNamespace = 'modules\\console\\controllers';
        } else {
            $this->controllerNamespace = 'modules\\controllers';
        }
  
        parent::init();
  
        /* Changes the Airtable API pagination key on the fly */
  
        Event::on(DataTypes::class, DataTypes::EVENT_AFTER_FETCH_FEED, function(FeedDataEvent $event) {
          $response = $event->response;
          $feedUrl = $event->url;
          
          /* Only run if this is an Airtable feed with a response */
          if ( strpos($feedUrl, "airtable") && isset($response['data']) ) : 
            $data = json_decode($response['data']);
            
            /* We only need to update the event if there's an offset key */
            if ( isset($data->offset) ) {
  
              /* Break the URL up into it's components */
              $url = parse_url($feedUrl);
              $baseUrl = $url['scheme'] . '://' . $url['host'] . $url['path'];
              parse_str($url['query'], $params);
  
              /* Update the current query with the new offset value */
              $params['offset'] = $data->offset;
  
              /* Rebuild the pagination URL */
              $data->offset = $baseUrl . '?' . http_build_query($params, $enc_type = PHP_QUERY_RFC3986);
  
              /* Return the data to the event */
              $response['data'] = json_encode($data);
              $event->response = $response;
            }
          endif;
        });
      }
  }
  
  ```
  
  > **Comments:**
  > **username_1**: Thanks for sharing!  
  > **username_3**: I just want to say how much I appreciate this! It's saved me hours of time.  
  > **username_2**: @username_3 - Awesome, glad to hear it!  
  
  ---
  
  ### Answer by username_1 on 2020-10-09. Score: 2
  
  First, you'll want to make sure you're on the latest FeedMe release as a bug around this was recently fixed: <https://github.com/craftcms/feed-me/issues/694>
  
  You might also need a plugin that hooks into the `EVENT_AFTER_FETCH_FEED` event to do some offset transformation based on the tokens passed in.
  
  
  ---
  
  