A lot of folks on here are saying that this is cool but useless because there are better ways to click a button on a screen.  If you read through their paper (<a href="http://groups.csail.mit.edu/uid/projects/sikuli/sikuli-uist2009.pdf" rel="nofollow">http://groups.csail.mit.edu/uid/projects/sikuli/sikuli-uist2...</a>) you'll find more practical examples of what can be done with this type of system.<p>One such example is to track real time images of a webcam pointed at a baby and using Sikuli to watch for a yellow dot placed on the baby's forehead.  Another is to track movements of something across the screen; in this case a bus moving along Google maps.<p>I agree that there are better ways to do most of the things in their examples and that they should probably re-work their videos a bit, but just because this system doesn't solve your problems the way you want it to doesn't mean its useless.