I am trying to find a way to extract the depth of a website using python.
Depth of a subwebsite is equal to the number of clicks required from the main website (e.g. www.ualberta.ca) in order for a user to get to the subwebsite (e.g. www.ualberta.ca/beartracks). so for instance if it takes one additional click to get to a subwebsite from the main domain, the depth of the subwebsite would be 1.
is there anyway for me to measure this using python? thank you!
It sounds like you want to write a spider to do breadth-first search from the first url until you find a link to the second url.
I suggest you look at the Scrapy package; it makes it very easy to do.
measuring a depth of a website using python