I have a python script that extracts product data from an ecommerce website. However, one essential piece of information is missing from the page - delivery cost. This is not provided on any of the product pages, and is only available when you add the product to the shopping basket in order to test how much the product costs to deliver. Complexity is also added due to different delivery rules - e.g free delivery on orders over £100, different delivery prices for different items, or a flat rate of shipping for multiple products.
Is there a way that I can easily obtain this delivery cost data? Are there any services that anyone knows of through which I can obtain this data more easily, or suggestions on a script that I could use?
Thanks in advance.
You shouldn't try to fetch information about delivery price from a cart or any other pages, because like you see it depends on cart amount or other conditions on e-commerce site. It means only one right way here is to emulate this rules/conditions when you try to calculate total price of an order on your side. Do it like this and you'll avoid too many problems with the correct calculations of delivery prices.
Scraping / Data extraction of shipping price not on product page (only available on trolley)