Follow y'all's nose for about five minutes until you see a sign saying Kuledibi on the left, which points the way to the tower.
Thanks to the complete lack of signs, the tower can be difficult to locate.