Answers.Body: <p>You could do this with the <a href="https://github.com/CGATOxford/cgat" rel="nofollow noreferrer"><code>CGAT</code> toolkit</a>:</p>
  
  <p><code>cgat bed2bed --method=merge --merge-by-name -I bed_with_gene_ids.bed</code></p>
  
  <p>Installing such a massive package might be overkill for this task though. </p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-10T15:32:06.393Z
Answers.Text: It happens that cgat is already installed on my computer (though I forgot for what purpose). I tried the command you suggest, and I end up with a duplicate of `I 3746 3909 "WBGene00023193" . -
  `. Granted, there were duplicate lines in the original bed. But is this behaviour expected?
Answers.UserDisplayName: bli
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-10T15:45:53.230Z
Answers.Text: Also, If I run this on the whole file and not just the first 6 lines, after a while, the program fails on `TypeError: '<' not supported between instances of 'Bed' and 'Bed'`. I'm upgrading cgat to see if the error persists.
Answers.UserDisplayName: bli
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-10T16:27:43.270Z
Answers.Text: I reported the issues here: https://github.com/CGATOxford/cgat/issues/347
Answers.UserDisplayName: bli
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-10T16:27:57.673Z
Answers.Text: Your first problem is not as far as I am aware the intended behaviour. And I'm pretty sure the second is not intended. I suggest you file a bug report.
Answers.UserDisplayName: Ian Sudbery
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-10T16:30:00.707Z
Answers.Text: We crossed posts!
Answers.UserDisplayName: Ian Sudbery
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-10T13:45:37.637Z
Answers.LastActivityDate: 2017-08-10T13:45:37.637Z
Answers.UserDisplayName: Ian Sudbery
Answers.Body: <p>You can do this easily with <a href="https://hail.is" rel="nofollow noreferrer">Hail</a>.  Hail primarily uses BED files to annotate genetic datasets (see the last <a href="https://hail.is/docs/stable/hail.VariantDataset.html#hail.VariantDataset.annotate_variants_table" rel="nofollow noreferrer">annotate_variants_table</a> example), but you can manipulate BED files using Hail's general facilities for manipulating delimited text files.  For example:</p>
  
  <pre><code>$ cat genes.bed
  I   3746    3909    "WBGene00023193"    .   -
  I   3746    3909    "WBGene00023193"    .   -
  I   4118    4220    "WBGene00022277"    .   -
  I   4118    4358    "WBGene00022277"    .   -
  I   4118    10230   "WBGene00022277"    .   -
  I   4220    4223    "WBGene00022277"    .   -
  </code></pre>
  
  <p>The Hail script (python code):</p>
  
  <pre><code>from hail import *
  hc = HailContext()
  (hc
   .import_table('genes.bed', impute=True, no_header=True)
   .aggregate_by_key('f0 = f0, f3 = f3',
      'f1 = f1.min(), f2 = f2.max(), f4 = ".", f5 = "-"')
   .select(['f0', 'f1', 'f2', 'f3', 'f4', 'f5'])
   .export('genes_merged.bed', header=False))
  </code></pre>
  
  <p>The result:</p>
  
  <pre><code>$ cat genes_merged.bed 
  I   3746    3909    WBGene00023193  .   -
  I   4118    10230   WBGene00022277  .   -
  </code></pre>
  
  <p>I aggregate over chrom and name so this solution won't merge entries on different chromosomes. The <code>select</code> is necessary to reorder the fields because <code>aggregate_by_key</code> places the keys being aggregated over first.</p>
  
  <p>Disclosure: I work on Hail.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-13T22:27:07.477Z
Answers.LastActivityDate: 2017-08-13T22:27:07.477Z
Answers.UserDisplayName: Cotton Seed
Answers.Body: <pre><code>$ cut -f4-6 in.bed | sed 's/\t/_/g' | sort | uniq | awk -F'_' '{ system("grep "$1" in.bed | bedops --merge - "); print $0; }' | paste -d "\t" - - | sed 's/_/\t/g' | sort-bed - &gt; answer.bed
  </code></pre>
  
  <p>Given your sample input:</p>
  
  <pre><code>$ more in.bed
  I   3746    3909    "WBGene00023193"    .   -
  I   3746    3909    "WBGene00023193"    .   -
  I   4118    4220    "WBGene00022277"    .   -
  I   4118    4358    "WBGene00022277"    .   -
  I   4118    10230   "WBGene00022277"    .   -
  I   4220    4223    "WBGene00022277"    .   -
  </code></pre>
  
  <p>The <code>answer.bed</code> file:</p>
  
  <pre><code>$ more answer.bed
  I   3746    3909    "WBGene00023193"    .   -
  I   4118    10230   "WBGene00022277"    .   -
  </code></pre>
  
  <p>Sorting with <code>sort-bed</code> is useful at the end, so that you can pipe it or work with it with other BEDOPS tools, or other tools that now accept sorted BED input.</p>
  
  <p>Streaming is a pretty efficient way to do things, generally.</p>
  
  <hr>
  
  <p><strong>How this works</strong></p>
  
  <p>Here's the pipeline again:</p>
  
  <pre><code>$ cut -f4-6 in.bed | sed 's/\t/_/g' | sort | uniq | awk -F'_' '{ system("grep "$1" in.bed | bedops --merge - "); print $0; }' | paste -d "\t" - - | sed 's/_/\t/g' | sort-bed - &gt; answer.bed
  </code></pre>
  
  <p>We start by cutting columns 4 through 6 (id, score and strand), replacing tabs with underscores, sorting and removing duplicates:</p>
  
  <pre><code>cut -f4-6 in.bed | sed 's/\t/_/g' | sort | uniq
  </code></pre>
  
  <p>What we get out of this is a sorted list of "needles" — one for each ID-score-strand combination: an <em>ID-needle</em> — that we can use to <code>grep</code> or filter the original BED file.</p>
  
  <p>This list is piped to <code>awk</code> which, for each ID-needle, runs <code>grep</code> against the original BED file and pipes the subset to <code>bedops --merge -</code>, which merges overlapping intervals. </p>
  
  <p>Note that merging only works for overlapping intervals. <em>Merging is not necessarily the same as returning a min-max pair, and this pipeline will break if there are intervals that do not overlap.</em> But you could modify the <code>awk</code> statement to process the input intervals and return the minimum and maximum interval coordinates, if that is really what you want, by tracking the min and max values over all intervals that come into <code>awk</code>, and printing a final interval with an <code>END</code> block.</p>
  
  <p>The <code>system</code> command prints the merged interval on one line. The following <code>print $0</code> statement prints the needle on the next line:</p>
  
  <pre><code>awk -F'_' '{ system("grep "$1" in.bed | bedops --merge - "); print $0; }'
  </code></pre>
  
  <p>We take each pair of alternating lines and re-linearize them with <code>paste</code>. This result now contains four columns: the three columns of each merged interval, and the ID-needle. </p>
  
  <p>We then use <code>sed</code> to replace underscores with tabs, so that we turn the ID-needle back into three, tab-separated ID-score-strand columns:</p>
  
  <pre><code>paste -d "\t" - - | sed 's/_/\t/g'
  </code></pre>
  
  <p>The output is now a six-column BED file, but it is ordered by the sort order we applied on ID-needles further up the pipeline, which we don't want. What we really want is BED that is sorted per BEDOPS <code>sort-bed</code>, so that we can do more set operations and get a correct result. So we pipe this to <code>sort-bed -</code> to write a sorted file to <code>answer.bed</code>:</p>
  
  <pre><code>sort-bed - &gt; answer.bed
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-11T10:58:11.707Z
Answers.Text: Thanks for the answer, it works, and I think I understood how. Maybe some explanations about the different steps could be useful.
Answers.UserDisplayName: bli
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-10T18:48:21.233Z
Answers.LastActivityDate: 2017-08-11T16:51:49.820Z
Answers.LastEditDate: 2017-08-11T16:51:49.820Z
Answers.UserDisplayName: Alex Reynolds
Answers.Body: <p>If you're 100% sure that everything but the start and end positions will be the same for all lines sharing a name, you could just do it yourself. For example, in Perl:</p>
  
  <pre><code>$ perl -lane '$start{$F[3]}||=$F[1]; 
                if( $F[1] &lt; $start{$F[3]} ){
                   $start{$F[3]} = $F[1]
                } 
                if( $F[2] &gt; $end{$F[3]} ){
                   $end{$F[3]} = $F[2]
                } 
                $chr{$F[3]} = $F[0]; 
                $rest{$F[3]} = join "\t", @F[4,$#F]; 
                END{
                   foreach $n (keys %chr){
                    print "$chr{$n}\t$start{$n}\t$end{$n}\t$n\t$rest{$n}"
                   }
                }' file.bed 
  I   3746    3909    "WBGene00023193"    .   -
  I   4118    10230   "WBGene00022277"    .   -
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-10T15:03:28.297Z
Answers.Text: I was hoping that an efficient tool already existed and would avoid me re-inventing the wheel in a slow scripting language.
Answers.UserDisplayName: bli
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-10T15:12:50.457Z
Answers.Text: @bli absolutely, that makes much more sense. I just figured this is simple enough, so I may as well give a scripting solution. But yes, this will be slow and is also very naive so it will break if your files are slightly different.
Answers.UserDisplayName: terdon
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-10T13:59:48.527Z
Answers.LastActivityDate: 2017-08-10T13:59:48.527Z
Answers.UserDisplayName: terdon
Answers.Body: <p>Although you don't mention it, I'm guessing you're using bedtools v2.26.0. Version 2.26.0 of groupBy has a bug in it, which you've encountered (it was fixed shortly after release, so you'll either have to use a version before the bug was introduced, or compile the current source code yourself from <a href="https://github.com/arq5x/bedtools2" rel="nofollow noreferrer">https://github.com/arq5x/bedtools2</a>) </p>
  
  <p>v2.26.0: </p>
  
  <pre><code>local10:~/Documents/tmp$ cat asdf.bed 
  I 3746 3909 WBGene00023193 . -
  I 3746 3909 WBGene00023193 . -
  I 4118 4220 WBGene00022277 . -
  I 4118 4358 WBGene00022277 . -
  I 4118 10230 WBGene00022277 . -
  I 4220 4223 WBGene00022277 . -
  local10:~/Documents/tmp$ groupBy -i asdf.bed -g 4 -c 2,3 -o min,max 
  3746    10230 
  </code></pre>
  
  <p>v2.26.0-125-g52db654 (I.E. compiling the source code from github): </p>
  
  <pre><code>local10:~/Documents/tmp$ bedtools2/bin/groupBy -i asdf.bed -g 4 -c 2,3 -o min,max
  WBGene00023193  3746    3909
  WBGene00022277  4118    10230
  </code></pre>
  
  <hr>
  
  <p>To answer your questions: </p>
  
  <p>1) You might notice that my output above gives the grouped columns first; you'll have to reorder the output via awk in order to get it back in order. As for why they chose to group on both columns 1 and 4: if you have the same name on multiple chromosomes, you may want to treat them as separate features.</p>
  
  <p>2) Version differences, as stated in the first part of my answer. </p>
  
  <hr>
  
  <p>To actually merge the file: </p>
  
  <p>Make sure to run this with a version other than v2.26.0 (As Devon Ryan writes in the comments, you may want to add column 6 to <code>-g</code> to make it strand-specific):</p>
  
  <pre><code>./bedtools2/bin/groupBy -i asdf.bed -g 1,4 -c 2,3,5,6 -o min,max,first,first \
       | awk -v OFS='\t' '{print $1, $3, $4, $2, $5, $6}'
  I   3746    3909    WBGene00023193  .   -
  I   4118    10230   WBGene00022277  .   -
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-10T18:30:28.567Z
Answers.Text: If you include 6 in `-g 1,4` then you benefit by not merging genes on different strands. UCSC sometimes has those and they really aren't the same gene and shouldn't be merged together. You don't need 1 in `-c`, or 6 if you add it to `-g`.
Answers.UserDisplayName: Devon Ryan
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-10T17:40:41.057Z
Answers.LastActivityDate: 2017-08-11T17:36:04.583Z
Answers.LastEditDate: 2017-08-11T17:36:04.583Z
Answers.UserDisplayName: OrdiNeu
Body: <p>I generated a file starting with the following bed lines:</p>
  <pre><code>$ head -6 /tmp/bed_with_gene_ids.bed
  I   3746    3909    &quot;WBGene00023193&quot;    .   -
  I   3746    3909    &quot;WBGene00023193&quot;    .   -
  I   4118    4220    &quot;WBGene00022277&quot;    .   -
  I   4118    4358    &quot;WBGene00022277&quot;    .   -
  I   4118    10230   &quot;WBGene00022277&quot;    .   -
  I   4220    4223    &quot;WBGene00022277&quot;    .   -
  </code></pre>
  <p>I would like to merge them based on the name field (the 4-th column), taking the min for the start and the max for the end. Other fields are expected to be the same for all records having the same name.</p>
  <p>Expected result:</p>
  <pre><code>I   3746    3909    &quot;WBGene00023193&quot;    .   -
  I   4118    10230   &quot;WBGene00022277&quot;    .   -
  </code></pre>
  <p>I found a potential solution based on <code>bedtools groupby</code> here: <a href="https://www.biostars.org/p/145751/#145775" rel="nofollow noreferrer">https://www.biostars.org/p/145751/#145775</a></p>
  <hr />
  <blockquote>
  <p>Sample data:</p>
  </blockquote>
  <pre><code>cat genes.bed
  chr14    49894259    49895806    ENSMUST00000053290    0.000000    ...
  chr14    49894873    49894876    ENSMUST00000053290    0.000000    ...
  chr14    49894876    49895800    ENSMUST00000053291    0.000000    ...
  chr14    49895797    49895800    ENSMUST00000053291    0.000000    ...
  chr14    49901908    49901941    ENSMUST00000053291    0.000000    ...
  </code></pre>
  <blockquote>
  <p>Example output:</p>
  </blockquote>
  <pre><code>sort -k4,4 genes.bed \
  | groupBy -g 1,4 -c 4,2,3 -o count,min,max \
  | awk -v OFS='\t' '{print <span class="math-container">$1, $</span>4, <span class="math-container">$5, $</span>2, $3}'
  
  chr14    49894259    49895806    ENSMUST00000053290    2
  chr14    49894876    49901941    ENSMUST00000053291    3
  </code></pre>
  <hr />
  <p>However:</p>
  <ol>
  <li><p>I don't understand the groupBy behaviour (Why <code>-g 1,4</code> and not just <code>-g 4</code>?, Why <code>-c 4,2,3</code> in this order and then rearrange things using <code>awk</code>?)</p>
  </li>
  <li><p>This code doesn't work for me.</p>
  </li>
  </ol>
  <p>Here is what happens when I try the solution given above:</p>
  <pre><code><span class="math-container">$ head -3 /tmp/bed_with_gene_ids.bed | bedtools groupby -g 1,4 -c 4,2,3 -o count,min,max | awk -v OFS='\t' '{print $1, $4, $5, $2, $</span>3}'
  3           3746    4220
  </code></pre>
  <p>Here are attempt based on what I thought could work according to <a href="http://bedtools.readthedocs.io/en/latest/content/tools/groupby.html" rel="nofollow noreferrer">the documentation</a>:</p>
  <pre><code>$ head -6 /tmp/bed_with_gene_ids.bed | bedtools groupby -g 4 -c 1,2,3,4,5,6 -o first,min,max,distinct,first,first
  I   3746    10230   &quot;WBGene00022277&quot;,&quot;WBGene00023193&quot;   .   -
  
  $ head -6 /tmp/bed_with_gene_ids.bed | bedtools groupby -g 4 -c 1,2,3,4,5,6 -o first,min,max,last,first,first
  I   3746    10230   &quot;WBGene00022277&quot;    .   -
  
  $ head -6 /tmp/bed_with_gene_ids.bed | bedtools groupby -g 4 -c 1,2,3,5,6 -o first,min,max,first,first
  I   3746    10230   .   -
  </code></pre>
  <p>I don't get why when I group based on the 4-th column, for which I have two distinct values, I cannot obtain two lines in the resulting output.</p>
  <p>I understand based on the comments on the documentation page that the documentation is not up-to-date. In particular, there is a <code>-full</code> option that is needed if one wants all fields to be outputted. Re-reading the solution mentioned above, I think I now understand the reason for the multiple columns for the <code>-g option</code> and for the <code>awk</code> rearrangement. Hence the following attempt.</p>
  <pre><code>$ head -6 /tmp/bed_with_gene_ids.bed | bedtools groupby -g 1,4,5,6 -c 2,3 -o min,max -full
      I   3746    3909    &quot;WBGene00023193&quot;    .   -   3746    10230
  </code></pre>
  <p>But this still doesn't give me two lines.</p>
  <p>Are there other tools that could do what I want efficiently?</p>
  <hr />
  <h3>Edit: Solution</h3>
  <p>According to <a href="https://bioinformatics.stackexchange.com/a/2271/292">this answer</a>, the problem with bedtools is that there is a bug in the latest release (2.26.0 as of august 2017). In order to have a functional <code>bedtools groupby</code>, one needs to get the development version from github.</p>
  <p>With the <a href="https://github.com/arq5x/bedtools2/commit/52db65490b196f95bcd141a20c79ed36d9989496" rel="nofollow noreferrer">github version of bedtools</a>, I can now get the expected result as follows:</p>
  <pre><code><span class="math-container">$ head -6 /tmp/bed_with_gene_ids.bed | bedtools groupby -g 1,4,5,6 -c 2,3 -o min,max | awk -v OFS="\t" '{print $1,$5,$6,$2,$3,$</span>4}'
  I   3746    3909    &quot;WBGene00023193&quot;    .   -
  I   4118    10230   &quot;WBGene00022277&quot;    .   -
  </code></pre>
  <p>I include fields 1, 5 and 6 in <code>-g</code> (besides field 4) in order to have them printed out. In my bed file, they should be the same for a given value of field 4. The <code>awk</code> part is needed because one has apparently not total control on the output order: the <code>-g</code> fields come before the <code>-c</code> fields.</p>
  <p>Note (added 23/12/2020): As specified in <code>bedtools groupby</code> documentation, the input bed file should be sorted according to (chromosome (1) and) name (4) field(s) for this to work properly. Otherwise, duplicate (i.e. non-merged) names may appear in the output.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-08-10T12:55:00.507Z
Comments.Text: What do you want to do with the score and strand fields if they're different between lines, or does that never happen?
Comments.UserDisplayName: Devon Ryan
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-08-10T15:06:54.250Z
Comments.Text: Actually, I don't care about the score field, and I would ideally set it to "." if it is not already the case. I cannot guarantee that the strand field will always be the same, but since these bed lines come from transcript annotations whose gene_id I've put in the name field, I suppose it will generally be true that for a same name, the strand will be the same. I should check this, though.
Comments.UserDisplayName: bli
Tags: <bed><bedtools>
Title: Merging bed records based on name
ThreadText: # Merging bed records based on name
  
  (Asked by: username_0 on 2017-08-10)
  
  I generated a file starting with the following bed lines:
  
  ```
  $ head -6 /tmp/bed_with_gene_ids.bed
  I   3746    3909    "WBGene00023193"    .   -
  I   3746    3909    "WBGene00023193"    .   -
  I   4118    4220    "WBGene00022277"    .   -
  I   4118    4358    "WBGene00022277"    .   -
  I   4118    10230   "WBGene00022277"    .   -
  I   4220    4223    "WBGene00022277"    .   -
  
  ```
  
  I would like to merge them based on the name field (the 4-th column), taking the min for the start and the max for the end. Other fields are expected to be the same for all records having the same name.
  
  Expected result:
  
  ```
  I   3746    3909    "WBGene00023193"    .   -
  I   4118    10230   "WBGene00022277"    .   -
  
  ```
  
  I found a potential solution based on `bedtools groupby` here: <https://www.biostars.org/p/145751/#145775>
  
  ---
  
  > Sample data:
  
  ```
  cat genes.bed
  chr14    49894259    49895806    ENSMUST00000053290    0.000000    ...
  chr14    49894873    49894876    ENSMUST00000053290    0.000000    ...
  chr14    49894876    49895800    ENSMUST00000053291    0.000000    ...
  chr14    49895797    49895800    ENSMUST00000053291    0.000000    ...
  chr14    49901908    49901941    ENSMUST00000053291    0.000000    ...
  
  ```
  
  > Example output:
  
  ```
  sort -k4,4 genes.bed \
  | groupBy -g 1,4 -c 4,2,3 -o count,min,max \
  | awk -v OFS='\t' '{print $1, $4, $5, $2, $3}'
  
  chr14    49894259    49895806    ENSMUST00000053290    2
  chr14    49894876    49901941    ENSMUST00000053291    3
  
  ```
  
  ---
  
  However:
  
  1. I don't understand the groupBy behaviour (Why `-g 1,4` and not just `-g 4`?, Why `-c 4,2,3` in this order and then rearrange things using `awk`?)
  2. This code doesn't work for me.
  
  Here is what happens when I try the solution given above:
  
  ```
  $ head -3 /tmp/bed_with_gene_ids.bed | bedtools groupby -g 1,4 -c 4,2,3 -o count,min,max | awk -v OFS='\t' '{print $1, $4, $5, $2, $3}'
  3           3746    4220
  
  ```
  
  Here are attempt based on what I thought could work according to [the documentation](http://bedtools.readthedocs.io/en/latest/content/tools/groupby.html):
  
  ```
  $ head -6 /tmp/bed_with_gene_ids.bed | bedtools groupby -g 4 -c 1,2,3,4,5,6 -o first,min,max,distinct,first,first
  I   3746    10230   "WBGene00022277","WBGene00023193"   .   -
  
  $ head -6 /tmp/bed_with_gene_ids.bed | bedtools groupby -g 4 -c 1,2,3,4,5,6 -o first,min,max,last,first,first
  I   3746    10230   "WBGene00022277"    .   -
  
  $ head -6 /tmp/bed_with_gene_ids.bed | bedtools groupby -g 4 -c 1,2,3,5,6 -o first,min,max,first,first
  I   3746    10230   .   -
  
  ```
  
  I don't get why when I group based on the 4-th column, for which I have two distinct values, I cannot obtain two lines in the resulting output.
  
  I understand based on the comments on the documentation page that the documentation is not up-to-date. In particular, there is a `-full` option that is needed if one wants all fields to be outputted. Re-reading the solution mentioned above, I think I now understand the reason for the multiple columns for the `-g option` and for the `awk` rearrangement. Hence the following attempt.
  
  ```
  $ head -6 /tmp/bed_with_gene_ids.bed | bedtools groupby -g 1,4,5,6 -c 2,3 -o min,max -full
      I   3746    3909    "WBGene00023193"    .   -   3746    10230
  
  ```
  
  But this still doesn't give me two lines.
  
  Are there other tools that could do what I want efficiently?
  
  ---
  
  ### Edit: Solution
  
  According to [this answer](https://bioinformatics.stackexchange.com/a/2271/292), the problem with bedtools is that there is a bug in the latest release (2.26.0 as of august 2017). In order to have a functional `bedtools groupby`, one needs to get the development version from github.
  
  With the [github version of bedtools](https://github.com/arq5x/bedtools2/commit/52db65490b196f95bcd141a20c79ed36d9989496), I can now get the expected result as follows:
  
  ```
  $ head -6 /tmp/bed_with_gene_ids.bed | bedtools groupby -g 1,4,5,6 -c 2,3 -o min,max | awk -v OFS="\t" '{print $1,$5,$6,$2,$3,$4}'
  I   3746    3909    "WBGene00023193"    .   -
  I   4118    10230   "WBGene00022277"    .   -
  
  ```
  
  I include fields 1, 5 and 6 in `-g` (besides field 4) in order to have them printed out. In my bed file, they should be the same for a given value of field 4. The `awk` part is needed because one has apparently not total control on the output order: the `-g` fields come before the `-c` fields.
  
  Note (added 23/12/2020): As specified in `bedtools groupby` documentation, the input bed file should be sorted according to (chromosome (1) and) name (4) field(s) for this to work properly. Otherwise, duplicate (i.e. non-merged) names may appear in the output.
  
  **Tags:** bed, bedtools
  
  > **Comments:**
  > **username_1**: What do you want to do with the score and strand fields if they're different between lines, or does that never happen?  
  > **username_0**: Actually, I don't care about the score field, and I would ideally set it to "." if it is not already the case. I cannot guarantee that the strand field will always be the same, but since these bed lines come from transcript annotations whose gene_id I've put in the name field, I suppose it will generally be true that for a same name, the strand will be the same. I should check this, though.  
  
  ---
  
  ## 5 answers
  
  ### Answer by username_6 on 2017-08-10. Score: 6 | Accepted answer
  
  Although you don't mention it, I'm guessing you're using bedtools v2.26.0. Version 2.26.0 of groupBy has a bug in it, which you've encountered (it was fixed shortly after release, so you'll either have to use a version before the bug was introduced, or compile the current source code yourself from <https://github.com/arq5x/bedtools2>)
  
  v2.26.0:
  
  ```
  local10:~/Documents/tmp$ cat asdf.bed 
  I 3746 3909 WBGene00023193 . -
  I 3746 3909 WBGene00023193 . -
  I 4118 4220 WBGene00022277 . -
  I 4118 4358 WBGene00022277 . -
  I 4118 10230 WBGene00022277 . -
  I 4220 4223 WBGene00022277 . -
  local10:~/Documents/tmp$ groupBy -i asdf.bed -g 4 -c 2,3 -o min,max 
  3746    10230 
  
  ```
  
  v2.26.0-125-g52db654 (I.E. compiling the source code from github):
  
  ```
  local10:~/Documents/tmp$ bedtools2/bin/groupBy -i asdf.bed -g 4 -c 2,3 -o min,max
  WBGene00023193  3746    3909
  WBGene00022277  4118    10230
  
  ```
  
  ---
  
  To answer your questions:
  
  1) You might notice that my output above gives the grouped columns first; you'll have to reorder the output via awk in order to get it back in order. As for why they chose to group on both columns 1 and 4: if you have the same name on multiple chromosomes, you may want to treat them as separate features.
  
  2) Version differences, as stated in the first part of my answer.
  
  ---
  
  To actually merge the file:
  
  Make sure to run this with a version other than v2.26.0 (As username_1 username_1 writes in the comments, you may want to add column 6 to `-g` to make it strand-specific):
  
  ```
  ./bedtools2/bin/groupBy -i asdf.bed -g 1,4 -c 2,3,5,6 -o min,max,first,first \
       | awk -v OFS='\t' '{print $1, $3, $4, $2, $5, $6}'
  I   3746    3909    WBGene00023193  .   -
  I   4118    10230   WBGene00022277  .   -
  
  ```
  
  > **Comments:**
  > **username_1**: If you include 6 in `-g 1,4` then you benefit by not merging genes on different strands. UCSC sometimes has those and they really aren't the same gene and shouldn't be merged together. You don't need 1 in `-c`, or 6 if you add it to `-g`.  
  
  ---
  
  ### Answer by username_2 on 2017-08-10. Score: 4
  
  You could do this with the [`CGAT` toolkit](https://github.com/CGATOxford/cgat):
  
  `cgat bed2bed --method=merge --merge-by-name -I bed_with_gene_ids.bed`
  
  Installing such a massive package might be overkill for this task though.
  
  > **Comments:**
  > **username_0**: It happens that cgat is already installed on my computer (though I forgot for what purpose). I tried the command you suggest, and I end up with a duplicate of `I 3746 3909 "WBGene00023193" . - `. Granted, there were duplicate lines in the original bed. But is this behaviour expected?  
  > **username_0**: Also, If I run this on the whole file and not just the first 6 lines, after a while, the program fails on `TypeError: '<' not supported between instances of 'Bed' and 'Bed'`. I'm upgrading cgat to see if the error persists.  
  > **username_0**: I reported the issues here: https://github.com/CGATOxford/cgat/issues/347  
  > **username_2**: Your first problem is not as far as I am aware the intended behaviour. And I'm pretty sure the second is not intended. I suggest you file a bug report.  
  > **username_2**: We crossed posts!  
  
  ---
  
  ### Answer by username_3 on 2017-08-13. Score: 3
  
  You can do this easily with [Hail](https://hail.is). Hail primarily uses BED files to annotate genetic datasets (see the last [annotate\_variants\_table](https://hail.is/docs/stable/hail.VariantDataset.html#hail.VariantDataset.annotate_variants_table) example), but you can manipulate BED files using Hail's general facilities for manipulating delimited text files. For example:
  
  ```
  $ cat genes.bed
  I   3746    3909    "WBGene00023193"    .   -
  I   3746    3909    "WBGene00023193"    .   -
  I   4118    4220    "WBGene00022277"    .   -
  I   4118    4358    "WBGene00022277"    .   -
  I   4118    10230   "WBGene00022277"    .   -
  I   4220    4223    "WBGene00022277"    .   -
  
  ```
  
  The Hail script (python code):
  
  ```
  from hail import *
  hc = HailContext()
  (hc
   .import_table('genes.bed', impute=True, no_header=True)
   .aggregate_by_key('f0 = f0, f3 = f3',
      'f1 = f1.min(), f2 = f2.max(), f4 = ".", f5 = "-"')
   .select(['f0', 'f1', 'f2', 'f3', 'f4', 'f5'])
   .export('genes_merged.bed', header=False))
  
  ```
  
  The result:
  
  ```
  $ cat genes_merged.bed 
  I   3746    3909    WBGene00023193  .   -
  I   4118    10230   WBGene00022277  .   -
  
  ```
  
  I aggregate over chrom and name so this solution won't merge entries on different chromosomes. The `select` is necessary to reorder the fields because `aggregate_by_key` places the keys being aggregated over first.
  
  Disclosure: I work on Hail.
  
  
  ---
  
  ### Answer by username_4 on 2017-08-10. Score: 2
  
  ```
  $ cut -f4-6 in.bed | sed 's/\t/_/g' | sort | uniq | awk -F'_' '{ system("grep "$1" in.bed | bedops --merge - "); print $0; }' | paste -d "\t" - - | sed 's/_/\t/g' | sort-bed - > answer.bed
  
  ```
  
  Given your sample input:
  
  ```
  $ more in.bed
  I   3746    3909    "WBGene00023193"    .   -
  I   3746    3909    "WBGene00023193"    .   -
  I   4118    4220    "WBGene00022277"    .   -
  I   4118    4358    "WBGene00022277"    .   -
  I   4118    10230   "WBGene00022277"    .   -
  I   4220    4223    "WBGene00022277"    .   -
  
  ```
  
  The `answer.bed` file:
  
  ```
  $ more answer.bed
  I   3746    3909    "WBGene00023193"    .   -
  I   4118    10230   "WBGene00022277"    .   -
  
  ```
  
  Sorting with `sort-bed` is useful at the end, so that you can pipe it or work with it with other BEDOPS tools, or other tools that now accept sorted BED input.
  
  Streaming is a pretty efficient way to do things, generally.
  
  ---
  
  **How this works**
  
  Here's the pipeline again:
  
  ```
  $ cut -f4-6 in.bed | sed 's/\t/_/g' | sort | uniq | awk -F'_' '{ system("grep "$1" in.bed | bedops --merge - "); print $0; }' | paste -d "\t" - - | sed 's/_/\t/g' | sort-bed - > answer.bed
  
  ```
  
  We start by cutting columns 4 through 6 (id, score and strand), replacing tabs with underscores, sorting and removing duplicates:
  
  ```
  cut -f4-6 in.bed | sed 's/\t/_/g' | sort | uniq
  
  ```
  
  What we get out of this is a sorted list of "needles" — one for each ID-score-strand combination: an *ID-needle* — that we can use to `grep` or filter the original BED file.
  
  This list is piped to `awk` which, for each ID-needle, runs `grep` against the original BED file and pipes the subset to `bedops --merge -`, which merges overlapping intervals.
  
  Note that merging only works for overlapping intervals. *Merging is not necessarily the same as returning a min-max pair, and this pipeline will break if there are intervals that do not overlap.* But you could modify the `awk` statement to process the input intervals and return the minimum and maximum interval coordinates, if that is really what you want, by tracking the min and max values over all intervals that come into `awk`, and printing a final interval with an `END` block.
  
  The `system` command prints the merged interval on one line. The following `print $0` statement prints the needle on the next line:
  
  ```
  awk -F'_' '{ system("grep "$1" in.bed | bedops --merge - "); print $0; }'
  
  ```
  
  We take each pair of alternating lines and re-linearize them with `paste`. This result now contains four columns: the three columns of each merged interval, and the ID-needle.
  
  We then use `sed` to replace underscores with tabs, so that we turn the ID-needle back into three, tab-separated ID-score-strand columns:
  
  ```
  paste -d "\t" - - | sed 's/_/\t/g'
  
  ```
  
  The output is now a six-column BED file, but it is ordered by the sort order we applied on ID-needles further up the pipeline, which we don't want. What we really want is BED that is sorted per BEDOPS `sort-bed`, so that we can do more set operations and get a correct result. So we pipe this to `sort-bed -` to write a sorted file to `answer.bed`:
  
  ```
  sort-bed - > answer.bed
  
  ```
  
  > **Comments:**
  > **username_0**: Thanks for the answer, it works, and I think I understood how. Maybe some explanations about the different steps could be useful.  
  
  ---
  
  ### Answer by username_5 on 2017-08-10. Score: 0
  
  If you're 100% sure that everything but the start and end positions will be the same for all lines sharing a name, you could just do it yourself. For example, in Perl:
  
  ```
  $ perl -lane '$start{$F[3]}||=$F[1]; 
                if( $F[1] < $start{$F[3]} ){
                   $start{$F[3]} = $F[1]
                } 
                if( $F[2] > $end{$F[3]} ){
                   $end{$F[3]} = $F[2]
                } 
                $chr{$F[3]} = $F[0]; 
                $rest{$F[3]} = join "\t", @F[4,$#F]; 
                END{
                   foreach $n (keys %chr){
                    print "$chr{$n}\t$start{$n}\t$end{$n}\t$n\t$rest{$n}"
                   }
                }' file.bed 
  I   3746    3909    "WBGene00023193"    .   -
  I   4118    10230   "WBGene00022277"    .   -
  
  ```
  
  > **Comments:**
  > **username_0**: I was hoping that an efficient tool already existed and would avoid me re-inventing the wheel in a slow scripting language.  
  > **username_5**: @username_0 absolutely, that makes much more sense. I just figured this is simple enough, so I may as well give a scripting solution. But yes, this will be slow and is also very naive so it will break if your files are slightly different.  
  
  ---
  
  