Answers.Body: <p>The short answer is yes, and I have ranted about this in <a href="http://anthon.home.xs4all.nl/rants/2013/polishing_epub/">parallelizing EPUB polishing</a>.</p>
  
  <hr>
  
  <p>The problem with some Linux systems (Ubuntu) is that there is a limit to the message popup queue. Since ebook polishing (using calibre) is done quicker than the time a message is displayed, this queue overflows and calibre cannot write messages to it any more, and (on last try at least) calibre did lock up.</p>
  
  <p>If you look in any directory where calibre stores the EPUB, you can see that there is a <code>metadata.opf</code> file that contains the metadata for the corresponding ebook. You have to specify this file as argument for the <code>ebook-polish</code> program together (<code>-o</code> option), together with the corresponding EPUB file.</p>
  
  <p>What you therefore need to do is make list of 'outdated' EPUB files (outdated compared to the <code>metadata.opf</code> files in the same directory), and then call:</p>
  
  <pre><code>ebook-polish -o path/to/metadata.opf path/to/book.epub
  </code></pre>
  
  <p>and rename the output (<code>path/to/book_polished.epub</code>) to <code>path/to/book.epub</code>.</p>
  
  <p>The <a href="http://anthon.home.xs4all.nl/rants/2013/_static/ebook_parallel_polish">script</a> referred to in the above rant does exactly that. As prerequisite you need to have <a href="http://www.gnu.org/software/parallel/">GNU parallel</a> installed¹.</p>
  
  <p>With the script I was able to process 1329 polishes in 3m25. Searching and updating 4 changed files in 2400+ item database with 1600+ books now takes less than 2 seconds². Of course files in the calibre database that are not supported by ebook-publish (such as PDF) are completely skipped.</p>
  
  <p>¹ <sub>In case you wonder, yes the script, via <code>parallel</code>, calls itself for each file to call<code>ebook-polish</code> and rename the output</sub><br>
  ² <sub>Fast enought to always try and polish the EPUB files before I synchronise the selected files with the reader devices</sub><br></p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-03-09T02:42:27.807Z
Answers.Text: ubuntu has a message pop up queue?  Please elaborate.
Answers.UserDisplayName: Thufir
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-03-10T07:24:14.573Z
Answers.Text: @Thufir What is there to elaborate? Messages are, or at least used to be when I was still using Ubuntu, displayed one after another after having been put in a queue by the OS or the display program. If you have further questions about Ubuntu put it on [ubuntu.se]
Answers.UserDisplayName: Anthon
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2013-12-27T15:31:54.057Z
Answers.LastActivityDate: 2013-12-27T15:40:11.827Z
Answers.LastEditDate: 2013-12-27T15:40:11.827Z
Answers.UserDisplayName: Anthon
Body: <p>As described in <a href="https://ebooks.stackexchange.com/a/379/60">this answer</a>, you can manually polish EPUB files so they are updated with the metadata you added or changed in calibre, without converting the EPUB files.</p>
  
  <p>At least on Linux there is a problem with this functionality if you have many files:</p>
  
  <ul>
  <li>You cannot select all EPUB files (after searching for <code>format:epub</code>) and start the conversion of them all if there are more than 25-50 files.</li>
  <li>You cannot find which files need polishing and which not, so if you lost track after updating several books, you need to start from scratch.</li>
  <li>You cannot easily run <code>ebook-polish</code> from the commandline on all EPUB files,  using <code>find</code>. As files with name <code>...polished.epub</code> might exists and because the output files cannot be specified and need to be renamed.</li>
  </ul>
  
  <p>Is there a more easy way to polish all and only the EPUB files that need updating?</p>
  
Tags: <epub><calibre><metadata><linux>
Title: How can I keep a large number of EPUB files, managed in calibre, "polished"
UserDisplayName: Anthon
ThreadText: # How can I keep a large number of EPUB files, managed in calibre, "polished"
  
  (Asked by: username_0 on 2013-12-27)
  
  As described in [this answer](https://ebooks.stackexchange.com/a/379/60), you can manually polish EPUB files so they are updated with the metadata you added or changed in calibre, without converting the EPUB files.
  
  At least on Linux there is a problem with this functionality if you have many files:
  
  * You cannot select all EPUB files (after searching for `format:epub`) and start the conversion of them all if there are more than 25-50 files.
  * You cannot find which files need polishing and which not, so if you lost track after updating several books, you need to start from scratch.
  * You cannot easily run `ebook-polish` from the commandline on all EPUB files, using `find`. As files with name `...polished.epub` might exists and because the output files cannot be specified and need to be renamed.
  
  Is there a more easy way to polish all and only the EPUB files that need updating?
  
  **Tags:** epub, calibre, metadata, linux
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2013-12-27. Score: 15
  
  The short answer is yes, and I have ranted about this in [parallelizing EPUB polishing](http://username_0.home.xs4all.nl/rants/2013/polishing_epub/).
  
  ---
  
  The problem with some Linux systems (Ubuntu) is that there is a limit to the message popup queue. Since ebook polishing (using calibre) is done quicker than the time a message is displayed, this queue overflows and calibre cannot write messages to it any more, and (on last try at least) calibre did lock up.
  
  If you look in any directory where calibre stores the EPUB, you can see that there is a `metadata.opf` file that contains the metadata for the corresponding ebook. You have to specify this file as argument for the `ebook-polish` program together (`-o` option), together with the corresponding EPUB file.
  
  What you therefore need to do is make list of 'outdated' EPUB files (outdated compared to the `metadata.opf` files in the same directory), and then call:
  
  ```
  ebook-polish -o path/to/metadata.opf path/to/book.epub
  
  ```
  
  and rename the output (`path/to/book_polished.epub`) to `path/to/book.epub`.
  
  The [script](http://username_0.home.xs4all.nl/rants/2013/_static/ebook_parallel_polish) referred to in the above rant does exactly that. As prerequisite you need to have [GNU parallel](http://www.gnu.org/software/parallel/) installed¹.
  
  With the script I was able to process 1329 polishes in 3m25. Searching and updating 4 changed files in 2400+ item database with 1600+ books now takes less than 2 seconds². Of course files in the calibre database that are not supported by ebook-publish (such as PDF) are completely skipped.
  
  ¹ In case you wonder, yes the script, via `parallel`, calls itself for each file to call`ebook-polish` and rename the output  
  ² Fast enought to always try and polish the EPUB files before I synchronise the selected files with the reader devices
  
  > **Comments:**
  > **username_1**: ubuntu has a message pop up queue?  Please elaborate.  
  > **username_0**: @username_1 What is there to elaborate? Messages are, or at least used to be when I was still using Ubuntu, displayed one after another after having been put in a queue by the OS or the display program. If you have further questions about Ubuntu put it on [ubuntu.se]  
  
  ---
  
  