Take the 2-minute tour ×
Stack Overflow is a question and answer site for professional and enthusiast programmers. It's 100% free, no registration required.

I have a bunch of (ugly if I may say) urls, which I would like to clean up using python regex. So, my urls look something like:

http://www.thisislink1.com/this/is/sublink1/1
http://www.thisislink2.co.uk/this/is/sublink1s/klinks
http://www.thisislinkd.co/this/is/sublink1/hotlinks/2
http://www.thisislinkf.com.uk/this/is/sublink1d/morelink
http://www.thisislink1.co.in/this/is/sublink1c/mylink
....

What I'd like to do is clean up these urls, so that the final link looks like:

http://www.thisislink1.com
http://www.thisislink2.co.uk
http://www.thisislinkd.co
http://www.thisislinkf.de
http://www.thisislink1.us
....

and I was wondering how I can achieve this in a pythonic way. Sorry if this is a 101 question - I am new to pytho regex structures.

share|improve this question
add comment

4 Answers

Why use regex?

>>> import urlparse
>>> url = 'http://www.thisislinkd.co/this/is/sublink1/hotlinks/2'
>>> urlparse.urlsplit(url)
SplitResult(scheme='http', netloc='www.thisislinkd.co', path='/this/is/sublink1/hotlinks/2', query='', fragment='')
share|improve this answer
add comment

Use urlparse.urlsplit:

In [3]: import urlparse    

In [8]: url = urlparse.urlsplit('http://www.thisislink1.com/this/is/sublink1/1')

In [9]: url.netloc
Out[9]: 'www.thisislink1.com'

In Python3 it would be

import urllib.parse as parse
url = parse.urlsplit('http://www.thisislink1.com/this/is/sublink1/1')
share|improve this answer
add comment

You should use a URL parser like others have suggested but for completeness here is a solution with regex:

import re

url='http://www.thisislink1.com/this/is/sublink1/1'

re.sub('(?<![/:])/.*','',url)

>>> 'http://www.thisislink1.com'

Explanation:

Match everything after and including the first forwardslash that is not preceded by a : or / and replace it with nothing ''.

(?<![/:]) # Negative lookbehind for '/' or ':'
/.*       # Match a / followed by anything
share|improve this answer
add comment

Maybe use something like this:

result = re.sub(r"(?m)(http://(www)?\..*?)/", r"\1", subject)
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.
{"cc_re__cc_re__cc_by_sa_3_0": ["        <li><a href=\"http://area51.stackexchange.com\" title=\"proposing new sites in the Stack Exchange network\">Area 51</a></li>\r\n                        <li><a href=\"http://careers.stackoverflow.com\">Stack Overflow Careers</a></li>\r\n                    \r\n                </ol>\r\n            </td>\r\n    </tr>\r\n</table>\r\n                </div>\r\n            </div>\r\n\r\n            <div id=\"copyright\">\r\n                site design / logo &#169; 2014 stack exchange inc; user contributions licensed under <a href=\"http://creativecommons.org/licenses/by-sa/3.0/\" rel=\"license\">cc by-sa 3.0</a> \r\n                with <a href=\"http://blog.stackoverflow.com/2009/06/attribution-required/\" rel=\"license\">attribution required</a>\r\n            </div>\r\n            <div id=\"svnrev\">\r\n                rev 2014.3.17.1466\r\n            </div>\r\n            \r\n        </div>\r\n    </div>\r\n    <noscript>\r\n        <div id=\"noscript-warning\">Stack Overflow works best with JavaScript enabled<img src=\"http://pixel.quantserve.com/pixel/p-c1rF4kxgLUzNc.gif\" alt=\"\" class=\"dno\"></div>\r\n    </nos"], "copyright__copyright__copyright": ["by a : or / and replace it with nothing ''.\n\n(?<![/:]) # Negative lookbehind for '/' or ':'\n/.*       # Match a / followed by anything\nshare|improve this answer\nadd comment\n\nMaybe use something like this:\n\nresult = re.sub(r\"(?m)(http://(www)?\\..*?)/\", r\"\\1\", subject)\nshare|improve this answer\nadd comment\n\nYour Answer\n\n\u00a0\ndiscard\n\nBy posting your answer, you agree to the privacy policy and terms of service.\n\nNot the answer you're looking for? Browse other questions tagged or ask your own question."]}