events.action: opened
events.author: gavanderhoorn
events.masked_author: username_0
events.text: Unfinished, but wanted to get some early feedback.
  
  Docs have not been updated.
events.title: WIP: Add pre 2014 support -- second attempt
events.type: issue
events.action: created
events.author: gavanderhoorn
events.masked_author: username_0
events.text: @wasowski @username_2 @username_1
  
  To use:
  
   - build Docker image (use build script)
   - check help output of `rosinstall_generator_tm.sh`
  
  Note: this new version of the time machine does not accept URLs to issues/PRs as input any more. It only accepts iso8601 timestamps.
  
  The `get_issue_creation_date.py` script can be used to find the `created_at` datetime stamp, but that requires `PyGitHub`. That is present in the Docker image, but I don't provide an easy way to run that script inside a container yet.
events.type: comment
events.action: created
events.author: gavanderhoorn
events.masked_author: username_0
events.text: Related: #1, #2, #3 and #4.
events.type: comment
events.action: created
events.author: gavanderhoorn
events.masked_author: username_0
events.text: Next goal would be to add the time machine script itself to the Docker image. That way it would be completely self-contained.
events.type: comment
events.action: created
events.author: gavanderhoorn
events.masked_author: username_0
events.text: A simple test -- to see if at least the infrastructure of the new version works on machines-not-my-own:
  
  ```
  rosinstall_generator_tm.sh "2016-01-01T00:00:00+00:00" indigo roscpp --deps
  ```
  
  That should get you a `.rosinstall` file (but printed to your console) with `roscpp` itself *and* all of its (transitive) dependencies.
  
  I've been creating some more rosdistro caches and uploading them so ideally generating that `.rosinstall` file takes very little time, as it should be able to reuse a cache I've already created.
events.type: comment
events.action: created
events.author: git-afsantos
events.masked_author: username_1
events.text: The new changes look good, but once again my inexperience with Docker strikes.
  
  From what I understood, `virtualenv` has been replaced with a Docker image, which contains all the dependencies of the time machine. Including the time machine within said image is the end goal.
  So, how does this work in practice for the newbies? Do I build the image and run a container, or does the time machine interact with the image in a more transparent way?
  
  Sorry if the help output of the script answers this, have not tried it yet, but will soon.
events.type: comment
events.action: created
events.author: gavanderhoorn
events.masked_author: username_0
events.text: For the moment you only interact with the script. It does everything for you.
  
  Except building the Docker image it needs that will contain the necessary `virtualenv`s.
  
  So as I wrote in https://github.com/rosin-project/rosinstall_generator_time_machine/pull/8#issuecomment-437100402, you run `build.sh` in the `docker` sub dir and you're set (`rosinstall_generator_tm.sh` will also complain if it can't find the Docker image it needs).
  
  ---
  
  For the next version I'll add `rosinstall_generator_tm.sh` to the image and add a very thin shell script that will basically do something like this:
  
  ```
  docker run \
    -it \
    --rm \
    -v <map some dirs in your homedir to inside the container> \
    robust-rosin/rosinstall_generator_time_machine:02 \
    /rgtm/rosinstall_generator_tm.sh "$*"
  ```
  
  So you still only interact with a bash script, but that just forwards everything to the script inside the Docker image.
events.type: comment
events.action: created
events.author: ChrisTimperley
events.masked_author: username_2
events.text: This is awesome. Thanks! I'll update the `build-rosinstall.py` script on a separate branch.
events.type: comment
events.action: created
events.author: ChrisTimperley
events.masked_author: username_2
events.text: One observation: It looks like git versions are now being returned instead of archives? Using the archives might help to reduce the size of the Docker image for each scenario?
events.type: comment
events.action: created
events.author: git-afsantos
events.masked_author: username_1
events.text: Need to clone rosdistro .. done
  Need to retrieve some rosdistro caches ..done
  Requested timepoint: '2016-01-01T00:00:00+00:00' (1451606400)
  Resetting local rosdistro clone ..
  Already on 'master'
  Your branch is up-to-date with 'origin/master'.
  Determined rosdistro commit: a772634f (authored: 1451597278)
  Reverting to ros/rosdistro@a772634f
  Cache already exists for (distro; stamp) tuple, skipping generation
  Updating local rosdistro index.yaml to use cache from the past ..
  Invoking: rosinstall_generator --rosdistro=indigo roscpp --deps --tar
  Using ROS_DISTRO: indigo
  - tar:
      local-name: catkin
      uri: https://github.com/ros-gbp/catkin-release/archive/release/indigo/catkin/0.6.16-0.tar.gz
      version: catkin-release-release-indigo-catkin-0.6.16-0
  - tar:
      local-name: cmake_modules
      uri: https://github.com/ros-gbp/cmake_modules-release/archive/release/indigo/cmake_modules/0.3.3-0.tar.gz
      version: cmake_modules-release-release-indigo-cmake_modules-0.3.3-0
  - tar:
      local-name: gencpp
      uri: https://github.com/ros-gbp/gencpp-release/archive/release/indigo/gencpp/0.5.3-0.tar.gz
      version: gencpp-release-release-indigo-gencpp-0.5.3-0
  - tar:
      local-name: genlisp
      uri: https://github.com/ros-gbp/genlisp-release/archive/release/indigo/genlisp/0.4.15-0.tar.gz
      version: genlisp-release-release-indigo-genlisp-0.4.15-0
  - tar:
      local-name: genmsg
      uri: https://github.com/ros-gbp/genmsg-release/archive/release/indigo/genmsg/0.5.6-0.tar.gz
      version: genmsg-release-release-indigo-genmsg-0.5.6-0
  - tar:
      local-name: genpy
      uri: https://github.com/ros-gbp/genpy-release/archive/release/indigo/genpy/0.5.7-0.tar.gz
      version: genpy-release-release-indigo-genpy-0.5.7-0
  - tar:
      local-name: message_generation
      uri: https://github.com/ros-gbp/message_generation-release/archive/release/indigo/message_generation/0.2.10-0.tar.gz
      version: message_generation-release-release-indigo-message_generation-0.2.10-0
  - tar:
      local-name: message_runtime
      uri: https://github.com/ros-gbp/message_runtime-release/archive/release/indigo/message_runtime/0.4.12-0.tar.gz
      version: message_runtime-release-release-indigo-message_runtime-0.4.12-0
  - tar:
      local-name: ros/rosbuild
      uri: https://github.com/ros-gbp/ros-release/archive/release/indigo/rosbuild/1.11.11-0.tar.gz
      version: ros-release-release-indigo-rosbuild-1.11.11-0
  - tar:
      local-name: ros/roslang
      uri: https://github.com/ros-gbp/ros-release/archive/release/indigo/roslang/1.11.11-0.tar.gz
      version: ros-release-release-indigo-roslang-1.11.11-0
  - tar:
      local-name: ros/roslib
      uri: https://github.com/ros-gbp/ros-release/archive/release/indigo/roslib/1.11.11-0.tar.gz
      version: ros-release-release-indigo-roslib-1.11.11-0
  - tar:
      local-name: ros/rosunit
      uri: https://github.com/ros-gbp/ros-release/archive/release/indigo/rosunit/1.11.11-0.tar.gz
      version: ros-release-release-indigo-rosunit-1.11.11-0
  - tar:
      local-name: ros_comm/rosconsole
      uri: https://github.com/ros-gbp/ros_comm-release/archive/release/indigo/rosconsole/1.11.16-0.tar.gz
      version: ros_comm-release-release-indigo-rosconsole-1.11.16-0
  - tar:
      local-name: ros_comm/roscpp
      uri: https://github.com/ros-gbp/ros_comm-release/archive/release/indigo/roscpp/1.11.16-0.tar.gz
      version: ros_comm-release-release-indigo-roscpp-1.11.16-0
  - tar:
      local-name: ros_comm/xmlrpcpp
      uri: https://github.com/ros-gbp/ros_comm-release/archive/release/indigo/xmlrpcpp/1.11.16-0.tar.gz
      version: ros_comm-release-release-indigo-xmlrpcpp-1.11.16-0
  - tar:
      local-name: ros_comm_msgs/rosgraph_msgs
      uri: https://github.com/ros-gbp/ros_comm_msgs-release/archive/release/indigo/rosgraph_msgs/1.11.1-0.tar.gz
      version: ros_comm_msgs-release-release-indigo-rosgraph_msgs-1.11.1-0
  - tar:
      local-name: roscpp_core/cpp_common
      uri: https://github.com/ros-gbp/roscpp_core-release/archive/release/indigo/cpp_common/0.5.6-0.tar.gz
      version: roscpp_core-release-release-indigo-cpp_common-0.5.6-0
  - tar:
      local-name: roscpp_core/roscpp_serialization
      uri: https://github.com/ros-gbp/roscpp_core-release/archive/release/indigo/roscpp_serialization/0.5.6-0.tar.gz
      version: roscpp_core-release-release-indigo-roscpp_serialization-0.5.6-0
  - tar:
      local-name: roscpp_core/roscpp_traits
      uri: https://github.com/ros-gbp/roscpp_core-release/archive/release/indigo/roscpp_traits/0.5.6-0.tar.gz
      version: roscpp_core-release-release-indigo-roscpp_traits-0.5.6-0
  - tar:
      local-name: roscpp_core/rostime
      uri: https://github.com/ros-gbp/roscpp_core-release/archive/release/indigo/rostime/0.5.6-0.tar.gz
      version: roscpp_core-release-release-indigo-rostime-0.5.6-0
  - tar:
      local-name: rospack
      uri: https://github.com/ros-gbp/rospack-release/archive/release/indigo/rospack/2.2.5-0.tar.gz
      version: rospack-release-release-indigo-rospack-2.2.5-0
  - tar:
      local-name: std_msgs
      uri: https://github.com/ros-gbp/std_msgs-release/archive/release/indigo/std_msgs/0.5.9-1.tar.gz
      version: std_msgs-release-release-indigo-std_msgs-0.5.9-1
  </details>
events.type: comment
events.action: created
events.author: gavanderhoorn
events.masked_author: username_0
events.text: Thanks @username_1.
  
  re: did not take long: that is good to hear and was one of the reasons I wanted to improve the script. Notice how it skips generating the cache because it already exists.
  
  The output also looks like I would have expected it.
events.type: comment
events.action: created
events.author: gavanderhoorn
events.masked_author: username_0
events.text: O and one thing: if/when you have generated a few `.rosinstall` files -- and the accompanying `rosdistro` caches -- could you open a PR against [username_0/rgtm_rosdistro_caches](https://github.com/username_0/rgtm_rosdistro_caches) contributing the new caches.
  
  That way things will go faster for everyone.
events.type: comment
events.action: created
events.author: git-afsantos
events.masked_author: username_1
events.text: Not sure if I should open a new issue, of if it is intended behaviour (for now), and I just missed it.
  
  When the generator needs to build a new cache, output redirection should be avoided - or else, you do not see the prompts for github credentials, which require you to hit Enter.
  
  So the best workflow for now is to run once without redirection, to build the cache if needed, and then run again, redirecting to the intended file.
events.type: comment
events.action: created
events.author: gavanderhoorn
events.masked_author: username_0
events.text: As also mentioned in https://github.com/robust-rosin/robust/issues/56#issuecomment-438438255, redirection should not be affecting the git pw prompts, as they should be appearing on `stderr`, which is not redirected with a regular `>`.
  
  I'll see if I can figure out what is going on.
events.type: comment
events.action: created
events.author: gavanderhoorn
events.masked_author: username_0
events.text: @username_1: I've added c7f97015. That appeared to be missing the redirect to `stderr`. A test on my machine with:
  
  ```
  rosinstall_generator_tm.sh "2014-05-11T00:00:00+00:00" hydro > blaa.txt
  ```
  
  shows all the output as I expected, including the pw and user prompts.
events.type: comment
events.action: created
events.author: git-afsantos
events.masked_author: username_1
events.text: This works for me too. :+1:
events.type: comment
events.action: created
events.author: ChrisTimperley
events.masked_author: username_2
events.text: I'm experiencing some unexpected behaviour with this version of the time machine:
  
  ```
  building rosinstall file for file: mavros/101c09b/101c09b.bug
  executing command: rosinstall_generator_tm.sh 2013-06-30 groovy mavros --deps --tar
  time machine failed for bug [mavros/101c09b/101c09b.bug]:
  Requested timepoint: '2013-06-30' (1372564800)
  Switching to pre-REP-141 infrastructure ..
  Resetting local rosdistro clone ..
  Previous HEAD position was d0158a278... Merge pull request #1148 from 130s/bloom-patch-4
  Switched to branch 'master'
  Your branch is up-to-date with 'origin/master'.
  Determined rosdistro commit: d0158a27 (authored: 1372611034)
  Reverting to ros/rosdistro@d0158a27
  Cache already exists for (distro; stamp) tuple, skipping generation
  Updating local rosdistro index.yaml to use cache from the past ..
  Invoking: rosinstall_generator --rosdistro=groovy mavros --deps --tar
  Using ROS_DISTRO: groovy
  The following not released packages/stacks will be ignored: mavros
  No packages/stacks left after ignoring not released
  ```
events.type: comment
events.action: created
events.author: gavanderhoorn
events.masked_author: username_0
events.text: This is documented [here](https://github.com/rosin-project/rosinstall_generator_time_machine#no-packagesstacks-left-after-ignoring-not-released), but it basically means that `mavros` was not released in `groovy` at the time you requested.
  
  Note btw: the code expects a ISO 8601 timestamp, which I don't believe `2013-06-30` is. Is this the json/Python parsing of the yaml again?
  
  `date` and `git` can be flexible when parsing date-times, but just to avoid possible issues I would *really* recommend to use iso8601 stamps. They are much less ambiguous: `2016-06-30` does not include any info on timezones, which will make datetime parsing dependent on local time+date configuration.
events.type: comment
events.action: created
events.author: ChrisTimperley
events.masked_author: username_2
events.type: comment
events.action: created
events.author: gavanderhoorn
events.masked_author: username_0
events.text: Are you not using the `get_issue_creation_date.py` script? I would not use the datetimes in the `.bug` files. They are too imprecise. Bugs can be very dependent on the state of the pkgs (and thus on the state of `rosdistro`). Afaik, @wasowski retrieved most of those stamps by looking at the date that github renders on mouse-over.
events.type: comment
events.action: created
events.author: gavanderhoorn
events.masked_author: username_0
events.text: O it could of course be that something is not working correctly. That's why it is a `WIP`.
  
  If you'd manually checkout ros/rosdistro@d0158a27 you could see whether it's the correct state and whether `mavros` is listed in the relevant yaml files.
events.type: comment
events.action: created
events.author: gavanderhoorn
events.masked_author: username_0
events.text: Just did this (through Github, just click the link) and I cannot find `mavros` in https://github.com/ros/rosdistro/blob/d0158a278ddee40b3408ec4111bdc777cf62803f/groovy/release.yaml.
events.type: comment
events.action: created
events.author: gavanderhoorn
events.masked_author: username_0
events.text: I think I'm about ready to merge this PR.
  
  Migrating everything to be included in the Docker image is something I'll do in a separate PR.
  
  Things work sufficiently well for me right now with this new implementation.
  
  @username_2, @username_1: would you have any comments?
events.type: comment
events.action: created
events.author: git-afsantos
events.masked_author: username_1
events.text: :+1: I have not run into any issues since the one concerning output redirection.
events.type: comment
events.action: created
events.author: gavanderhoorn
events.masked_author: username_0
events.text: Thanks for the "review" @username_1 :)
events.type: comment