store the last_state property of the loader and pass it in when creating the loader again. This will ensure that only new records are loaded.last_state = loader.last_state # store safelyincremental_loader = AirbyteHubspotLoader(config=config, stream_name="products", state=last_state)new_docs = incremental_loader.load()PreviousAirbyte GongNextAirbyte JSONInstallationExampleIncremental loadsCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright ¬© 2023 LangChain, Inc.
Airbyte is a data integration platform for ELT pipelines from APIs, databases & files to warehouses & lakes. It has the largest catalog of ELT connectors to data warehouses and databases.
Airbyte is a data integration platform for ELT pipelines from APIs, databases & files to warehouses & lakes. It has the largest catalog of ELT connectors to data warehouses and databases. ->: store the last_state property of the loader and pass it in when creating the loader again. This will ensure that only new records are loaded.last_state = loader.last_state # store safelyincremental_loader = AirbyteHubspotLoader(config=config, stream_name="products", state=last_state)new_docs = incremental_loader.load()PreviousAirbyte GongNextAirbyte JSONInstallationExampleIncremental loadsCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright ¬© 2023 LangChain, Inc.