Question: I'm trying to use sed to clean up lines of URLs to extract just the domain.

So from:

```
http://www.suepearson.co.uk/product/174/71/3816/

```

I want:

```
http://www.suepearson.co.uk/

```

(either with or without the trailing slash, it doesn't matter)

I have tried:

```
 sed 's|\(http:\/\/.*?\/\).*|\1|'

```

and (escaping the non-greedy quantifier)

```
sed 's|\(http:\/\/.*\?\/\).*|\1|'

```

but I can not seem to get the non-greedy quantifier (`?`) to work, so it always ends up matching the whole string.

Answer: 
With sed, I usually implement non-greedy search by searching for anything except the separator until the separator :

```
echo "http://www.suon.co.uk/product/1/7/3/" | sed -n 's;\(http://[^/]*\)/.*;\1;p'

```

Output:

```
http://www.suon.co.uk

```

this is:

* don't output `-n`
* search, match pattern, replace and print `s/<pattern>/<replace>/p`
* use `;` search command separator instead of `/` to make it easier to type so `s;<pattern>;<replace>;p`
* remember match between brackets `\(` ... `\)`, later accessible with `\1`,`\2`...
* match `http://`
* followed by anything in brackets `[]`, `[ab/]` would mean either `a` or `b` or `/`
* first `^` in `[]` means `not`, so followed by anything but the thing in the `[]`
* so `[^/]` means anything except `/` character
* `*` is to repeat previous group so `[^/]*` means characters except `/`.
* so far `sed -n 's;\(http://[^/]*\)` means search and remember `http://`followed by any characters except `/` and remember what you've found
* we want to search untill the end of domain so stop on the next `/` so add another `/` at the end: `sed -n 's;\(http://[^/]*\)/'` but we want to match the rest of the line after the domain so add `.*`
* now the match remembered in group 1 (`\1`) is the domain so replace matched line with stuff saved in group `\1` and print: `sed -n 's;\(http://[^/]*\)/.*;\1;p'`

If you want to include backslash after the domain as well, then add one more backslash in the group to remember:

```
echo "http://www.suon.co.uk/product/1/7/3/" | sed -n 's;\(http://[^/]*/\).*;\1;p'

```

output:

```
http://www.suon.co.uk/

```
The following solution works for matching / working with multiply present (chained; tandem; compound) HTML or other tags. For example, I wanted to edit HTML code to remove `<span>` tags, that appeared in tandem.

**Issue:** regular `sed` regex expressions greedily matched over all the tags from the first to the last.

**Solution:** non-greedy pattern matching (per discussions elsewhere in this thread; e.g. <https://stackoverflow.com/a/46719361/1904943>).

**Example:**

```sh
echo '<span>Will</span>This <span>remove</span>will <span>this.</span>remain.' | \
sed 's/<span>[^>]*>//g' ; echo

This will remain.

```

**Explanation:**

* `s/<span>` : find `<span>`
* `[^>]` : followed by anything that is not `>`
* `*>` : until you find `>`
* `//g` : replace any such strings present with nothing.

---

**Addendum**

I was trying to clean up URLs, but I was running into difficulty matching / excluding a word - `href` - using the approach above. I briefly looked at negative lookarounds ([Regular expression to match a line that doesn't contain a word](https://stackoverflow.com/questions/406230/regular-expression-to-match-a-line-that-doesnt-contain-a-word)) but that approach seemed overly complex and did not provide a satisfactory solution.

I decided to replace `href` with ``` (backtick), do the regex substitutions, then replace ``` with `href`.

**Example** (formatted here for readability):

```sh
printf '\n
<a aaa h href="apple">apple</a>
<a bbb "c=ccc" href="banana">banana</a>
<a class="gtm-content-click"
   data-vars-link-text="nope"
   data-vars-click-url="https://blablabla"
   data-vars-event-category="story"
   data-vars-sub-category="story"
   data-vars-item="in_content_link"
   data-vars-link-text
   href="https:example.com">Example.com</a>\n\n' |
sed 's/href/`/g ;
     s/<a[^`]*`/\n<a href/g'

<a href="apple">apple</a> 
<a href="banana">banana</a> 
<a href="https:example.com">Example.com</a>

```

**Explanation:** basically as above. Here,

* `s/href/``  : replace `href` with ``` (backtick)
* `s/<a` : find start of URL
* `[^`]` : followed by anything that is not ``` (backtick)
* `*``  : until you find a ```
* `/<a href/g` : replace each of those found with `<a href`