I am trying to do some web scraping but I have some problems in joining relative and root urls
for example the root url is: http://www.jmlr.org/proceedings/papers/v2
and the relative url is: ../v2/meila07a/meila07a.pdf
As I use urljoin in urlparse: the result is odd:
http://www.jmlr.org/proceedings/v2/meila07a/meila07a.pdf
Which is not a valid link. Can anybody help me with that?
Two dots (..) means go back once in the hierarchy, change the second link to ./v2/meila07a/meila07a.pdf and it should be working fine.
Or you can also change the root one to http://www.jmlr.org/proceedings/papers/v2/, thanks to this change it will no longer dispose of v2 at the end because the root was not set to a proper directory.
joining urls with urljoin in python