Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

I am following along with Twitter's developer and want to do a very basic call against their filter real-time API. The code below almost gets me there:

curl -s -u twitterusername:twitterpassword https://stream.twitter.com/1/statuses/sample.json -o "somefile.txt"

My hope is dynamically name the file such that hourly logs of the data are captured.

EDIT: It is worth nothing that I am hoping this command remains open, and that that the data I receive are continuous. I am looking to redirect the output every hour to different files.

I am completely new to command line and ubuntu, so I don't even know where to start. Any help will be much appreciated.

share|improve this question

1 Answer 1

up vote 3 down vote accepted
curl -s -u twitterusername:twitterpassword https://stream.twitter.com/1/statuses/sample.json -o "somefile $(date + format).txt"

Where, format can be any one of the following:

%a : Abbreviated weekday name (Sun..Sat)
%b : Abbreviated month name (Jan..Dec)
%B : Full month name, variable length (January..December)
%d : Day of month (01..31)
%e : Day of month, blank padded ( 1..31)
%m : Month (01..12)
%Y : Year
%d : Day of month (e.g, 01)
%H : 24 hour format (00..23)
%I : 12 hour format (01..12)
%M : Minutes of the current time (00...59)
%j : day of year (001..366)
%D : date; same as %m/%d/%y
%F : full date; same as %Y-%m-%d

So, for you this will save the file and dynamically add the hour (%H) and the minutes (%M) of the current time

curl -s -u twitterusername:twitterpassword https://stream.twitter.com/1/statuses/sample.json -o "somefile $(date +\"%H:%M\").txt"

Because you want curl to get data for 1 hour and save that data to the file resuming again operations you need to at least use a small script, this will do the job:

#! /bin/bash

while true; do
    curl -s -m 3600 -u twitterusername:twitterpassword https://stream.twitter.com/1/statuses/sample.json -o "somefile $(date +%H:%M).txt"
done

It will, while you leave the script running, execute the command, every 3600 seconds (1 hour, the -m 3600 parameter) curl will close and the command will be gain executed.

Note that this will not just slit the stream, it will actually close curl and re-open it, do not think its possible to slit the stream while curl is running.

You need to same the script somewhere, ie ~/curl_script.sh and make it executable with chmod 755 ~/curl_script.sh before using it on the terminal, to use it move the folder where the script was saved and just type ./curl_script.sh.

To interrupt the script press Ctrl+c.

If you interrupt the script and resume it on the same minute it will by default overwrite the previous collected data, so beware.

Let me know if you want to make some other modifications to the script. For further curl parameters I recommend the read of the curl man page (man curl on a terminal).

Have fun.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.
{"cc_re__cc_re__cc_by_sa_3_0": ["                  <li><a href=\"//area51.stackexchange.com\" title=\"proposing new sites in the Stack Exchange network\">Area 51</a></li>\r\n                        <li><a href=\"//careers.stackoverflow.com\">Stack Overflow Careers</a></li>\r\n                    \r\n                </ol>\r\n            </td>\r\n    </tr>\r\n</table>\r\n                </div>\r\n            </div>\r\n\r\n            <div id=\"copyright\">\r\n                site design / logo &#169; 2015 stack exchange inc; user contributions licensed under <a href=\"http://creativecommons.org/licenses/by-sa/3.0/\" rel=\"license\">cc by-sa 3.0</a> \r\n                with <a href=\"http://blog.stackoverflow.com/2009/06/attribution-required/\" rel=\"license\">attribution required</a>\r\n            </div>\r\n            <div id=\"svnrev\">\r\n                rev 2015.2.26.2347\r\n            </div>\r\n                <div id=\"additional-notices\">\r\n                    Ubuntu and Canonical are registered trademarks of Canonical Ltd.\r\n                </div>\r\n            \r\n        </div>\r\n    </div>\r\n    <noscript>\r\n        <div id=\"noscript-wa"], "copyright__copyright__copyright": [" resume it on the same minute it will by default overwrite the previous collected data, so beware.\n\nLet me know if you want to make some other modifications to the script. For further curl parameters I recommend the read of the curl man page (man curl on a terminal).\n\nHave fun.\n\nshare|improve this answer\n\nYour Answer\n\n\u00a0\ndiscard\n\nBy posting your answer, you agree to the privacy policy and terms of service.\n\nNot the answer you're looking for? Browse other questions tagged or ask your own question."]}