Answers.Body: <p>My solution to this issue is fairly complex but stable.  I feel I went well beyond the initial request here.</p>
  
  <p>I built a MySQL database with a table of jobs.  In that table there are columns for status, the text that needs to be in each video and then further more with youtube metadata.  The table has an index ID value that auto increments with each added row.</p>
  
  <p>I manipulate the data via microsoft excel and save it as a CSV. I wrote a python script that reads the CSV and then imports into MySQL.  I do this because the youtube metadata column in each row is stored in JSON format for ease during the upload phase.</p>
  
  <p>Once in the MySQL database, I have a script that runs once daily that generates any videos listed with the status code corresponding to needing to be rendered.</p>
  
  <p>I use ffmpeg with drawtext to push the text desired during the timeline desired on top of a template version of the video that has whitespace in the timeline desired.</p>
  
  <p>Files are named corresponding to the Row ID# in the database to make it easier to pick up later on in the upload phase.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-11T05:23:09.007Z
Answers.LastActivityDate: 2017-06-11T05:23:09.007Z
Answers.UserDisplayName: Cas
Body: <p>I'm doing freelance work producing product videos for a company that sells millions of products (sub components for telecom equipment).  They want me to produce a video for each part number.  The video template they've provided to me has a roll about their company, contact info, etc and in the middle is a single title that has the static text of "We buy/sell: " followed by the part number the video corresponds to.  There are millions of part numbers provided to me via many spreadsheets.</p>
  
  <p>I'm not a video editor/producer by trade - I do cyber security work with freelance advanced data entry such as this.  I used my cyber security skills to mostly automate the process, which now looks like this:</p>
  
  <ol>
  <li>Using Adobe Premiere CC, create a sequence for the part number and insert a title with the part number in it into the sequence.  (~20 seconds total PER sequence).  I generate a project file with several thousand sequences.</li>
  <li>Batch encode all of the sequences in a single project via Adobe Media Encoder.</li>
  <li>Use a script I wrote to leverage YouTube API to upload all of the video files and pull the data from the spreadsheet (converted to CSV) and match the video's filename with the corresponding line in the CSV to add the YouTube metadata for the particular file.</li>
  </ol>
  
  <p>My time sink is just the 20 seconds per video of data entry - everything else is done via semi-automated job batching.</p>
  
  <p>I'm looking for a way to automate the last remaining data entry portion.  Adobe premiere doesn't reliably import formats that I can generate, such as XML.  (It does, but it fails 90% of the time - title objects are offline/missing.  It fails to import an unmodified XML file that it generated.)</p>
  
  <p>I'm not all that familiar with other video editing/production softwares.  I'm thinking I can combine the encoding and data entry portion into a process that I can script.  I started looking into VirtualDub but I decided I should ask the experts before trying to invent something that has already been invented.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-01-16T19:18:40.553Z
Comments.Text: Can you link to one output video, and identify the fixed and variable portions?
Comments.UserDisplayName: Gyan
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-01-16T21:49:32.510Z
Comments.Text: https://www.youtube.com/watch?v=fxifEOqKiYs
  
  At 44 seconds you will find the title that is variable.  The rest of the video is static.
  
  I may have a solution that I can script via python using ImageMagick and FFMPEG.  I'll develop it a little tonight and hopefully be able to test.
Comments.UserDisplayName: Cas
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-01-16T23:25:39.843Z
Comments.Text: Look into blender - free 3d app w/ python scripting, it has basic 2d timeline editing
Comments.UserDisplayName: MmmHmm
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-01-17T04:47:05.033Z
Comments.Text: Easily doable using ffmpeg alone. No IM required. Render the whole thing once from Premiere with the 44s-55s part with no text. Then use drawtext filter with right font to overlay text at that time.
Comments.UserDisplayName: Gyan
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-01-19T00:53:28.553Z
Comments.Text: I'm still going to use IM to generate the static images because I'm looking at the potential scale of a render job:  While 1 million videos isn't a realistic number, it shows how the scale can affect production time.
  
  If I have 1 million videos to render and it takes ffmpeg 2 addtional seconds to render the 44-55 second mark, that's an extra 555 hours of rendering time needed.  Using IM, I can offload the image rendering to another server I have in my server farm and then stagger the start of the ffmpeg script so the IM script has time to get some buffer.  Needs benchmarking.
Comments.UserDisplayName: Cas
Tags: <premiere><virtualdub>
Title: Scripting Video Production for data entry
UserDisplayName: Cas
ThreadText: # Scripting Video Production for data entry
  
  (Asked by: username_0 on 2017-01-16)
  
  I'm doing freelance work producing product videos for a company that sells millions of products (sub components for telecom equipment). They want me to produce a video for each part number. The video template they've provided to me has a roll about their company, contact info, etc and in the middle is a single title that has the static text of "We buy/sell: " followed by the part number the video corresponds to. There are millions of part numbers provided to me via many spreadsheets.
  
  I'm not a video editor/producer by trade - I do cyber security work with freelance advanced data entry such as this. I used my cyber security skills to mostly automate the process, which now looks like this:
  
  1. Using Adobe Premiere CC, create a sequence for the part number and insert a title with the part number in it into the sequence. (~20 seconds total PER sequence). I generate a project file with several thousand sequences.
  2. Batch encode all of the sequences in a single project via Adobe Media Encoder.
  3. Use a script I wrote to leverage YouTube API to upload all of the video files and pull the data from the spreadsheet (converted to CSV) and match the video's filename with the corresponding line in the CSV to add the YouTube metadata for the particular file.
  
  My time sink is just the 20 seconds per video of data entry - everything else is done via semi-automated job batching.
  
  I'm looking for a way to automate the last remaining data entry portion. Adobe premiere doesn't reliably import formats that I can generate, such as XML. (It does, but it fails 90% of the time - title objects are offline/missing. It fails to import an unmodified XML file that it generated.)
  
  I'm not all that familiar with other video editing/production softwares. I'm thinking I can combine the encoding and data entry portion into a process that I can script. I started looking into VirtualDub but I decided I should ask the experts before trying to invent something that has already been invented.
  
  **Tags:** premiere, virtualdub
  
  > **Comments:**
  > **username_1**: Can you link to one output video, and identify the fixed and variable portions?  
  > **username_0**: https://www.youtube.com/watch?v=fxifEOqKiYs  At 44 seconds you will find the title that is variable.  The rest of the video is static.  I may have a solution that I can script via python using ImageMagick and FFMPEG.  I'll develop it a little tonight and hopefully be able to test.  
  > **username_2**: Look into blender - free 3d app w/ python scripting, it has basic 2d timeline editing  
  > **username_1**: Easily doable using ffmpeg alone. No IM required. Render the whole thing once from Premiere with the 44s-55s part with no text. Then use drawtext filter with right font to overlay text at that time.  
  > **username_0**: I'm still going to use IM to generate the static images because I'm looking at the potential scale of a render job:  While 1 million videos isn't a realistic number, it shows how the scale can affect production time.  If I have 1 million videos to render and it takes ffmpeg 2 addtional seconds to render the 44-55 second mark, that's an extra 555 hours of rendering time needed.  Using IM, I can offload the image rendering to another server I have in my server farm and then stagger the start of the ffmpeg script so the IM script has time to get some buffer.  Needs benchmarking.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2017-06-11. Score: 1 | Accepted answer
  
  My solution to this issue is fairly complex but stable. I feel I went well beyond the initial request here.
  
  I built a MySQL database with a table of jobs. In that table there are columns for status, the text that needs to be in each video and then further more with youtube metadata. The table has an index ID value that auto increments with each added row.
  
  I manipulate the data via microsoft excel and save it as a CSV. I wrote a python script that reads the CSV and then imports into MySQL. I do this because the youtube metadata column in each row is stored in JSON format for ease during the upload phase.
  
  Once in the MySQL database, I have a script that runs once daily that generates any videos listed with the status code corresponding to needing to be rendered.
  
  I use ffmpeg with drawtext to push the text desired during the timeline desired on top of a template version of the video that has whitespace in the timeline desired.
  
  Files are named corresponding to the Row ID# in the database to make it easier to pick up later on in the upload phase.
  
  
  ---
  
  