Google Web Toolkit GWT Java AJAX Programming — Save 50%
A practical book and eBook guide to Google Web Toolkit for creating AJAX applications with Java, fast.
.
Introduction.
To say that maps are important is an understatement almost as big as the world itself. Maps quite literally connect us to the world in which we all live, and by extension, they link us to one another. The oldest preserved maps date back nearly 4500 years. In addition to connecting us to our past, they chart much of human progress and expose the relationships among people through time.
Unfortunately, as a work of humankind, maps share many of the same shortcomings of all human endeavors. They are to some degree inaccurate and they reflect the bias of the map maker. Advancements in technology help to address the former issue, and offer us the opportunity to resist the latter. To the extent that it's possible for all of us to participate in the map making, the bias of a select few becomes less meaningful.
Google Earth and Google Maps are two applications that allow each of us to assert our own place in the world and contribute our own unique perspective. I can think of no better way to accomplish this than by combining maps and photography.
Photos reveal much about who we are, the places we have been, the people with whom we have shared those experiences, and the significant events in our lives. Pinning our photos to a map allows us to see them in their proper geographic context, a valuable way to explore and share them with friends and family. Photos can reveal the true character of a place, and afford others the opportunity to experience these destinations, perhaps faraway and unreachable for some of us, from the perspective of someone who has actually been there.
In this tutorial I'll show you how to 'pin' your photos using Google Earth and Google Maps. Both applications are free, and available for Windows, Mac OS X, and Linux. In the case of Google Earth there are requirements associated with installing and running what is a local application. Google Maps has its own requirements: primary among them a compatible web browser (the highly regarded FireFox is recommended).
In Google Earth, your photos show up on the map within the application complete with titles, descriptions, and other relevant information. You can choose to share your photos with everyone, only people you know, or even reserve them strictly for yourself.
Google Maps offers the flexibility to present maps outside of a traditional application. For example you can embed a map on a webpage pinpointing the location of one particular photo, or mapping a collection of photos to present along with a photo gallery, or even collecting all of your digital photos together on one dynamic map.
Over the course of a short couple of articles we'll cover everything you need to take advantage of both applications. I've put together two scripts to help us accomplish that goal. The first is a Perl script that works through your photos and generates a file in the proper format with all of the data necessary to include those photos in Google Earth. The second is a short bit of Javascript that works with the first file and builds a dynamic Google Map of those same photos. Both scripts are available for you to download, after which you are free to use them as is, or modify them to suit your own projects. I've purposefully kept the code as simple as possible to make them accessible to the widest audience, even those of you reading this who may be new to programming or unfamiliar with Perl, Javascript or both. I've taken care to comment the code generously so that everything is plainly obvious. I'm hopeful that you will be surprised at just how simple it is.
There a couple of preliminary topics to examine briefly before we go any further.
In the preceding paragraph I mentioned that the result of the first of our two scripts is a 'file in the proper format...'. This file, or more to the point the file format, is a very important part of the project. KML (Keyhole Markup Language) is a fairly simple XML-based format that can be considered the native "language" of Google Earth. That description begs the question, 'What is XML?'.
To oversimplify, because even a cursory discussion of XML is outside of the scope of this article, XML (Extensible Markup Language) is an open data format (in contrast to proprietary formats) which allows us to present information in such way that we communicate not only the data itself, but also descriptive information about the data and the relationships among elements of the data. One of the technical terms that applies is 'metalanguage', which approximately translates to mean a language that makes it possible to describe other languages. If you're unfamiliar with the concept, it may be difficult to grasp at first or it may not seem impressive. However, metalanguages, and specifically XML, are an important innovation (I don't mean to suggest that it's a new concept. In fact XML has roots that are quite old, relative to the brief history of computing). These metalanguages make it possible for us to imbue data with meaning such that software can make use of that data. Let's look at an example taken from the Wikipedia entry for KML.
<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="">
<Placemark>
<description>New York City</description>
<name>New York City</name>
<Point>
<coordinates>-74.006393,40.714172,0</coordinates>
</Point>
</Placemark>
</kml>
Ignore all of the pro forma stuff before and after the <Placemark>
Let's think about this for a moment. There are a very large number of recognizable places on the planet, and a provably infinite number of strings. Given a block of text, how could a computer be made to identify a place from, for example the scientific name of a particular flower, or a person's name? It would be extremely difficult.
We could try to create a database of every recognizable place and have the program check the database every time it encounters a string. That assumes it's possible to agree on a 'complete list of every place', which is almost certainly untrue. Keep in mind that we could be talking about informal places that are significant only to a small number of people or even a single person, e.g. 'the park where, as a child, I first learned to fly a kite'. It would be a very long list if we were going to include these sorts of places, and incredibly time-consuming to search.
Relying on the structure of the fragment above, we can easily write a program that can identify 'New York City' as the name of a place, or for that matter 'The park where I first learned to fly a kite'. In fact, I could write a program that pulls all of the place names from a file like this, along with a description, and a pair of coordinate points for each, and includes them on a map. That's exactly what we're going to do. KML makes it possible.
If I haven't made it clear, the structural bits of the file must be standardized. KML supports a limited set of elements (e.g. 'Placemark' is a supported element, as are 'Point' and 'coordinates'), and all of the elements used in a file must adhere to the standard for it to be considered valid.
The second point we need to address before we begin is, appropriately enough... where to begin? Lewis Carroll famously tells us to "Begin at the beginning and go on till you come to the end: then stop." Of course, Mr. Carroll was writing a book at the time. If "Alice's Adventures in Wonderland" were an article, he might have had different advice. From the beginning to the end there is a lot of ground to cover. We're going to start somewhere further along, and make up the difference with the following assumptions. For the purposes of this discussion, I am going to assume that you have:
- Perl
- Access to Phil Harvey's excellent ExifTool , a Perl library and command-line application for reading, writing, and editing metadata in images (among other file types). We will be using this library in our first script
- A publicly accessible web server. Google requires the use of an API key by which it can monitor the use of its map services. Google must be able to validate your key, and so your site must be publicly available. Note that this a requirement for Google Maps only
- Photos, preferably in a photo management application. Essentially, all you need is an app capable of generating both thumbnails and reduced size copies of your original photos . An app that can export a nice gallery for use on the web is even better
- Coordinate data as part of the EXIF metadata embedded in your photos. If that sounds unfamiliar to you, then most likely you will have to take some additional steps before you can make use of this tutorial. I'm not aware of any digital cameras that automatically include this information at the time the photo is created. There are devices that can be used in combination with digital cameras, and there are a number of ways that you can 'tag' your photos with geographic data much the same way you would add keywords and other annotations.
Let's begin!
Part 1: Google Earth
Section 1: Introduction to Part 1
Time to begin the project in earnest. As I've already mentioned we'll spend the first half of this tutorial looking at Google Earth and putting together a Perl script whichthat, given a collection of geotagged photos, will build a set of KML files so that we can browse our photos in Google Earth. These same files will serve as the data source for our Google Maps application later on.
Section 2: Some Advice Before We Begin
Take your time to make sure you understand each topic before you move on to the next. Think of this as the first step in debugging your completed code. If you go slowly enough that you are to be able to identify aspects of the project that you don't quite understand, then you'll have some idea where to start looking for problems should things not go as expected. Furthermore, going slowly will give you the opportunity to identify those parts that you may want to modify to better fit the script to your own needs.
If this is new to you, follow along as faithfully as possible with what I do here the first time through. Feel free to make notes for yourself as you go, but making changes on the first pass may make it difficult for you to catch back on to the narrative and piece together a functional script. After you have a working solution, it will be a simple matter to implement changes one at a time until you have something that works for you. Following this approach it will be easy to identify the silly mistakes that tend to creep in once you start making changes.
There is also the issue of trust. This is probably the first time we're meeting each other, in which case you should have some doubt that my code works properly to begin with. If you minimize the number of changes you make, you can confirm that this works for you before blaming yourself or your changes for my mistakes. I will tell you up front that I'm building this project myself as we go. You can be certain at least that it functions as described for me as of the date attached to the article. I realize that this is quite different from being certain that the project will work for you, but at least it's something.
The entirety of my project is available for you to download. You are free to use all of it for any legal purpose whatsoever, including my photos in addition to all of the code, icons, etc. This is so you have some samples to use before you involve your own content. I don't promise that they are the most beautiful images you have ever seen, but they are all decent photos and properly annotated with the necessary metadata, including geographic tags.
Section 3: Photos, Metadata and ExifTool
To begin, we must have a clear understanding of what the Perl script will require from us. Essentially, we need to provide it with a selection of annotated image files, and information about how to reference those files.
A simple folder of files is sufficient, and will be convenient for us, both as the programmer and end user. The script will be capable of negotiating nested folders, and if a directory contains both images and other file types, non-image types will be ignored.
Typically, after a day of taking photos I'll have 100 to 200 that I want to keep. I delete the rest immediately after offloading them from the camera. For the files that are left, I preserve the original grouping, keeping all of the files together in a single folder. I place this folder of images in an appropriate location according to a scheme that serves to keep my complete collection of photos neatly organized. These are my digital 'negatives'. I handle all subsequent organization, editing, enhancements, and annotations within my photo management application. I use Apple Inc.'s Aperture; but there are many others that do the job equally well.
Annotating your photos is well worth the investment of time and effort, but it's important that you have some strategy in mind so that you don't create meaningless tags that are difficult to use to your advantage. For the purposes of this project the tags we'll need are quite limited, which means that going forward we will be able to continue adding photos to our maps with a reasonable amount of work.
The annotations we need are:
Caption
Latitude
Longitude
Image Date *
Location/Sub-Location
City
Province/State
Country Name
Event
People
ImageWidth *
ImageHeight *
* Values for these Exif tags are generated by your camera.
Note that these are labels used in Aperture, and are not necessarily consistent from one application to the next. Some of them are more likely than others to be used reliably. 'City' for example should be dependable, while the labels 'People', 'Events', and 'Location', among others, are more likely to differ. One explanation for these variations is that the meaning of these fields are more open to interpretation. Location, for example, is likely to be used to narrow down the area where the photo was taken within a particular city, but it is left to the person who is annotating the photo to decide that the field should name a specific street address, an informal place (e.g. 'home' or 'school'), or a larger area, for example a district or neighborhood. Fortunately things aren't so arbitrary as they seem.
Each of these fields corresponds to a specific tag name that adheres to one of the common metadata formats (Exif1, IPTC2, XMP3, and there are others). These tag names are consistent as required by the standards. The trick is in determining the labels used in your application that correspond to the well-defined tag names. Our script relies on these metadata tags, so it is important that you know which fields to use in your application.
This gives us an excuse to get acquainted with ExifTool4. From the project's website, we have this description of the application:
ExifTool is a platform-independent Perl library plus a command-line application for reading, writing, and editing meta information in image, audio, and video files...
ExifTool can seem a little intimidating at first. Just keep in mind that we will need to understand just a small part of it for this project, and then be happy that such a useful and flexible tool is freely available for you to use.
The brief description above states in part that ExifTool is a Perl library and command line application that we can use to extract metadata from image files.
With a single short command, we can have the app print all of the metadata contained in one of our image files. First, make sure ExifTool is installed. You can test for this by typing the name of the application at the command line.
$ exiftool
If it is installed, then running it with no options should prompt the tool to print its documentation. If this works, there will be more than one screen of text. You can page through it by pressing the spacebar. Press the 'q' key at any time to stop.
If the tool is not installed, you will need to add it before continuing. See the appendix at the end of this tutorial for more information.
Having confirmed that ExifTool is installed, typing the following command will result in a listing of the metadata for the named image:
$ exiftool -f -s -g2 /path/image.jpg
Where 'path' is an absolute path to image.jpg or a relative path from the current directory, and 'image.jpg' is the name of one of your tagged image files.
We'll have more to say about ExifTool later, but because I believe that no tutorial should ask the reader to blindly enter commands as if they were magic incantations, I'll briefly describe each of the options used in the command above:
-f, forces printing of tags even if their values are not found. This gives us a better idea about all of the available tag names, whether or not there are currently defined values for those tags.
-s, prints tag names instead of descriptions. This is important for our purposes. We need to know the tag names so that we can request them in our Perl code. Descriptions, which are expanded, more human-readable forms of the same names obscure details we need. For example, compare the tag name 'GPSLatitude' to the description 'GPS Latitude'. We can use the tag name, but not the description to extract the latitude value from our files.
-g2, organizes output by category. All location specific information is grouped together, as is all information related to the camera, date and time tags, etc. You may feel, as I do, that this grouping makes it easier to examine the output. Also, this organization is more likely to reflect the grouping of field names used by your photo management application.
If you prefer to save the output to a file, you can add ExifTool's -w option with a file extension.
$ exiftool -f -s -g2 -w txt path/image.jpg
This command will produce the same result but write the output to the file 'image.txt' in the current directory; again, where 'image.jpg' is the name of the image file. The -w option appends the named extension to the image file's basename, creates a new file with that name, and sets the new file as the destination for output.
The tag names that correspond to the list of Aperture fields presented above are:
Section 4: Making Photos Available to Google Earth
We will use some of the metadata tags from our image files to locate our photos on the map (e.g. GPSLatitude, GPSLongitude), and others to describe the photos. For example, we will include the value of the People tag in the information window that accompanies each marker to identify friends and family who appear in the associated photo.
Because we want to display and link to photos on the map, not just indicate their position, we need to include references to the location of our image files on a publicly accessible web server. You have some choice about how to do this, but for the implementation described here we will (1) Display a thumbnail in the info window of each map marker and (2) include a link to the details page for the image in a gallery created in our photo management app.
When a visitor clicks on a map marker they will see a thumbnail photo along with other brief descriptive information. Clicking a link included as part of the description will open the viewer's web browser to a page displaying a large size image and additional details. Furthermore, because the page is part of a gallery, viewers can jump to an index page and step forward and back through the complete collection. This is a complementary way to browse our photos. Taking this one step further, we could add a comments section to the gallery pages or replace the gallery altogether, instead choosing to display each photo as a weblog post for example.
The structure of the gallery created from my photo app is as follows...
/ (the root of the gallery directory)
index.html
large-1.html
large-2.html
large-3.html
...
large-n.html
assets/
css/
img/
catalog/
picture-1.jpg
picture-2.jpg
picture-3.jpg
...
picture-n.jpg
thumbnails/
thumb-1.jpg
thumb-2.jpg
thumb-3.jpg
...
thumb-n.jpg
The application creates a root directory containing the complete gallery. Assuming we do not want to make any manual changes to the finished site, publishing is as easy as copying the entire directory to a location within the web server's document root.
assets/: Is a subfolder containing files related to the theme itself. We don't need to concern ourselves with this sub-directory.
catalog/: Contains a single catalog.plist file which is specific to Aperture and not relevant to this discussion.
pictures/: Contains the large size images included on the detail gallery pages.
thumbnails/: This subfolder contains the thumbnail images corresponding to the large size images in pictures/.
Finally, there are a number of files at the root of the gallery. These include index pages and files named 'large-n.html', where n is a number starting at 1 and increasing sequentially e.g. large-1.html, large-2.html, large-3.html, ...
The index files are the index pages of our gallery. The number of index pages generated will be dictated by the number of image files in the gallery, as well as the gallery's layout and design. index.html is always the first gallery page.
The large-n.html files are the details pages of our gallery. Each page features an individual photo with links to the previous and next photos in sequence and a link back to the index. You can see the gallery I have created for this tutorial here:
If you take the time to look through the gallery, maybe you can appreciate the value of viewing these photos on a map. Web-based photo galleries like this one are nice enough, but the photos are more interesting when viewed in some meaningful context.
There are a couple of things to notice about this gallery.
Firstly, picture-1.jpg, thumb-1.jpg, and large-1.html all refer to the same image. So if we pick one of the three files we can easily predict the names of the other two. This relationship will be useful when it comes to writing our script.
There is another important issue I need to call to your attention because it will not be apparent from looking only at the gallery structure. Aperture has renamed all of my photos in the process of exporting them.
The name of the original file from which picture-1.jpg was generated (as well as large-1.html and thumb-1.jpg) is 'IMG_0277.JPG', which is the filename produced by my camera. Because I want to link to these gallery files, not my original photos which will stay safely tucked away on a local drive, I must run the script against the photos in the gallery. I cannot run it against the original image files because the filenames referenced in the output are unrelated to the files in the gallery.
If my photo management app provided me the option of preserving the original filenames for the corresponding photos in the gallery, then I could run the script against the original image files or the gallery photos because all of the filenames would be consistent, but this is not the case. I don't have a problem as long as I run the script on the exported photos.
However, if I'm running the script against the photos in the web gallery, either the pictures or thumbnail images must contain the same metadata as the original image files. Aperture preserves the metadata in both. Your application may not.
A simple, dependable way to confirm that the metadata is present in the gallery files is to run ExifTool first against the original file and then the same photo in the pictures/ and thumbnails/ directories in the gallery. If ExifTool reports identical metadata, then you will have no trouble using one of pictures/ or thumbnails/ as your source directory. If the metadata is not present or not complete in the gallery files, you may need to use the script on your original image files. As has already been explained, this isn't a problem unless the gallery produces filenames that are inconsistent with the original filenames, as Aperture does. In this case you have a problem. You won't be able to run the script on the original image files because of the naming issue or on the gallery photos because they don't contain metadata.
Make sure that you understand this point.
If you find yourself in this situation, then your best bet is to generate files to use with your maps from your original photos in some other way, bypassing your photo management app's web gallery features altogether in favor of a solution that preserves the filenames, the metadata, or both. There is another option which involves setting up a relationship between the names of the original files and the gallery filenames. This tutorial does not include details about how to set up this association.
Finally, keep in mind that though we've looked at the structure of a gallery generated by Aperture, virtually all photo management apps produce galleries with a similar structure. Regardless of the application used, you should find:
- A group of html files including index and details pages
- A folder of large size image files
- A folder of thumbnails
Once you have identified the structure used by your application, as we have done here, it will be a simple task to translate these instructions.
Section 5: Referencing Files Over the Internet
Now we can talk about how to reference these files and gallery pages so that we can create a script to generate a KML file that includes these references.
When we identify a file over the internet, it is not enough to use the filename, e.g. 'thumb-1.jpg', or even the absolute and relative path to the file on the local computer. In fact these paths are most likely not valid as far as your web server is concerned. Instead we need to know how to reference our files such that they can be accessed over the global internet, and the web more specifically. In other words, we need to be able to generate a URL (Uniform Resource Locator) which unambiguously describes the location of our files. The formal details of exactly what comprises a URL5; are more complicated than may be obvious at first, but most of us are familiar with the typical URL, like this one:
which describes the location of a document titled "Universal Resource Identifiers in WWW" that just so happens to define the formal details of what comprises a URL.
This portion of the address is enough to describe the location of a particular web server over the public internet. In fact it does a little more than just specify the location of a machine. The http:// portion is called the scheme and it identifies a particular protocol (i.e. a set of rules governing communication) and a related application, namely the web. What I just said isn't quite correct; at one time, HTTP was used exclusively by the web, but that's no longer true. Many internet-based applications use the protocol because the popularity of the web ensures that data sent via HTTP isn't blocked or otherwise disrupted. You may not be accustomed to thinking of it as such, but the web itself is a highly-distributed, network-based application.
/rfc/
This portion of the address specifies a directory on the server. It is equivalent to an absolute path on your local computer. The leading forward slash is the root of the web server's public document directory. Assuming no trickiness on the part of the server, all content lives under the document root.
This tells us that rfc/ is a sub-directory contained within the document root. Though this directory happens to be located immediately under the root, this certainly need not be the case. In fact these paths can get quite long.
We have now discussed all of the URL except for:
rfc1630.txt
which is the name of a specific file. The filename is no different than the filenames on your local computer. Let's manually construct a path to one of the large-n.html pages of the gallery we have created.
The address of my server is robreed.net, so I know that the beginning of my URL will be:
I keep all of my galleries together within a photos/ directory, which is contained in the document root.
Within photos/, each gallery is given its own folder. The name of the folder I have created for this tutorial is 'tutorial_gallery'. Putting this all together, the following URL brings me to the root of my photo gallery:
We've already gone over the directory structure of the gallery, so it should make sense you to that when referring to the 'large-1.html' detail page, the complete URL will be:
the URL of the image that corresponds to that detail page is:
and the thumbnail can be found at:
Notice that the address of the gallery is shared among all of these resources. Also, notice that resources of each type (e.g. the large images, thumbnails, and html pages) share a more specific address with files of that same type.
If we use the term 'base address' to refer to the shared portions of these URLs, then we can talk about several significant base addresses:
The gallery base address:
The html pages base address:
The images base address:
The thumbnails base address:
Note that given the structure of this particular gallery, the html pages base address and the gallery base address are identical. This need not be the case, and may not be for the gallery produced by your application.
We can hard-code the base addresses into our script. For each photo, we need only append the associated filename to construct valid URLs to any of these resources. As the script runs, it will have access to the name of the file that it is currently evaluating, and so it will be a simple matter to generate the references we need as we go.
At this point we have discussed almost everything we need to put together our script. We have:
- Created a gallery at our server, which includes our photos with metadata in tow
- Identified all of the metadata tags we need to extract from our photos with the script and the corresponding field names in our photo management application
- Determined all of the base addresses we need to generate references to our image files
Section 6: KML
The last thing we need to understand is the format of the KML files we want to produce.
We've already looked at a fragment of KML. The full details can be found on Google's KML documentation pages , which include samples, tutorials and a complete reference for the format.
A quick look at the reference is enough to see that the language includes many elements and attributes, the majority of which we will not be including in our files. That statement correctly implies it is not necessary for every KML file to include all elements and attributes. The converse however is not true, which is to say that every element and attribute contained in any KML file must be a part of the standard.
A small subset of KML will get us most, if not all, of what you will typically see in Google Earth from other applications.
Many of the features we will not be using deal with aspects of the language that are either:
- Not relevant to this project, e.g. ground overlays (GroundOverlay) which "draw an image overlay draped onto the terrain"
- Minute details for which the default values are sensible
There is no need to feel shortchanged because we are covering only a subset of the language. With the basic structure in place and a solid understanding of how to script the generation of KML, you will be able to extend the project to include any of the other components of the language as you see fit.
The structure of our KML file is as follows:
1. <?xml version="1.0" encoding="UTF-8"?>
2. <kml xmlns="">
3. <Document>
4. <Folder>
5. <name>$folder_name</name>
6. <description>$folder_description</description>
7. <Placemark>
8. <name>$placemark_name</name>
9. <Snippet maxLines="1">
10. $placemark_snippet
11. </Snippet>
12. <description><![CDATA[
13. $placemark_description
14. ]]></description>
15. <Point>
16. <coordinates>$longitude,$latitude
</coordinates>
17. </Point>
18. </Placemark>
19. </Folder>
20. </Document>
21. </kml>
Line 1: XML header
Every valid KML file must start with this line and nothing else is allowed to appear before it. As I've already mentioned, KML is an XML-based language and XML requires this header.
Line 2: Namespace declaration
More specifically this is the KML namespace declaration, and it is another formality. The value of the xmlns attribute must be a URI referencing a resource that identifies elements and attributes which are valid under the language. Remember that KML is one of many XML based languages, each with its own vocabulary. Making namespaces unique to each language avoids the obvious problems that would occur if all languages were required to pull from a common pool of identifiers. In such a scheme all identifiers would need to be unique among all XML based languages, —a situation which would quickly prove unworkable.
The first two lines are strictly boilerplate. They are required and will be identical in every KML file you produce. They change only when the language itself is modified.
Line 3: <Document> is a container element representing the KML file itself. If we do not explicitly name the document by including a name element then Google Earth will use the name of the KML file as the Document element <name>.
The Document container will appear on the Google Earth 'Sidebar' within the 'Places' panel. Optionally we can control whether the container is closed or open by default. (This setting can be toggled in Google Earth using a typical disclosure triangle.) There are many other elements and attributes that can be applied to the Document element. Refer to the KML Reference for the full details.
Line 4: <Folder> is another container element. The files we produce will include a single <Folder> containing all of our Placemarks, where each Placemark represents a single image. We could create multiple Folder elements to group our Placemarks according to some significant criteria. Think of the Folder element as being similar to your operating system's concept of a folder.
At this point, note the structure of the fragment. The majority of it is contained within the Folder element. Folder, in turn, is an element of Document which is itself within the <kml> container. It should make sense that everything in the file that is considered part of the language must be contained within the kml element.
From the KML reference:
A Folder is used to arrange other Features hierarchically (Folders, Placemarks, NetworkLinks, or Overlays). A Feature is visible only if it and all its ancestors are visible.
Line 5: The name element identifies an object, in this case the Folder object. The text that appears between the name tags can be any plain text that will serve as an appropriate label.
Line 6: <description> is any text that seems to adequately describe the object.
The description element supports both plain text and a subset of HTML. We'll consider issues related to using HTML in <description> at the discussion of Placemark, lines 12 - 14.
Lines 7 - 17 define a <Placemark> element.
Note that Placemark contains a number of elements that also appear in Folder, including <name> (line 8), and <description> (lines 12 - 14). These elements serve the same purpose for Placemark as they do for the Folder element, but of course they refer to a different object.
I've said that <description> can include a subset of HTML in addition to plain text. Under XML, some characters have special meaning. You may need to use these characters as part of the HTML included in your descriptions. Angle brackets (<, >) for example surround tag names in HTML, but serve a similar purpose in XML. When they are used strictly as part of the content, we want the XML parser to ignore these characters.
We can accomplish this a few different ways:
We can use entity references, either numeric character references or character entity references, to indicate that the symbol appears as part of the data and should not be treated as part of the syntax of the language. The character '<', which is required to include an image as part of the description (something we will be doing), (e.g. <img src='//dgdsbygo8mp3h.cloudfront.net/sites/default/files/blank.gif' data-original=... />) can safely be included as the character entity reference '<' or the numeric character reference '<'.
The character entity references may be easier to remember and recognize on sight but are limited to the small subset of characters for which they have been defined. The numeric references on the other hand can specify any ASCII character. Of the two types, numeric character references should be preferred. There are also Unicode entity references which can specify any character at all.
In the simple case of embedding short bits of HTML in KML descriptions, we can avoid the complication of these references altogether by enclosing the entire description in a CDATA6 element, which instructs the XML parser to ignore any special characters that appear until the end of the block set off by the CDATA tags.
Notice the string '<![CDATA[' immediately after the opening <description> tag within <Placemark>, and the string ]]> immediately before the closing </description> tag. If we simply place all of our HTML and plain text between those two strings, it will all be ignored by the parser and we are not required to deal with special characters individually. This is how we'll handle the issue.
Lines 9 - 11: <Snippet>
The KML reference does a good job of clearly describing this element.
From the KML reference:
In Google Earth, this description is displayed in the Places panel under the name of the feature. If a Snippet is not supplied, the first two lines of the <description> are used. In Google Earth, if a Placemark contains both a description and a Snippet, the <Snippet> appears beneath the Placemark in the Places panel, and the <description> appears in the Placemark's description balloon. This tag does not support HTML markup. <Snippet> has a maxLines attribute, an integer that specifies the maximum number of lines to display. Default for maxLines is 2.
Notice that in the block above at line 9, I have included a 'maxLines' attribute with a value of 1. Of course, you are free to substitute your own value for maxLines, or you can omit the attribute entirely to use the default value.
The only element we have yet to review is <Point>, and again we need only look to the official reference for a nice description.
From the KML reference:
A geographic location defined by longitude, latitude, and (optional) altitude. When a Point is contained by a Placemark, the point itself determines the position of the Placemark's name and icon.
<Point> in turn contains the element <coordinates> which is required.
From the KML reference:
A single tuple consisting of floating point values for longitude, latitude, and altitude (in that order).
The reference also informs us that altitude is optional, and in fact we will not be generating altitude values.
Finally the reference warns:
Do not include spaces between the three values that describe a coordinate.
This seems like an easy mistake to make. We'll need to be careful to avoid it.
There will be a number of <Placemark> elements, one for each of our images.
The question is how to handle these elements. The answer is that we'll treat KML as a 'fill in the blanks' style template. All of the structural and syntactic bits will be hard-coded, e.g. the XML header, namespace declaration, all of the element and attribute labels, and even the whitespace, which is not strictly required but will make it much easier for us to inspect the resulting files in a text editor. These components will form our template.
The blanks are all of the text and html values of the various elements and attributes. We will use variables as place-holders everywhere we need dynamic data, i.e. values that change from one file to the next or one execution of the script to the next.
Take a look at the strings I've used in the block above. $folder_name, $folder_description, $placemark_name, etc. For those of you unfamiliar with Perl, these are all valid variable names chosen to indicate where the variables slot into the structure. These are the same names used in the source file distributed with the tutorial.
Section 7: Introduction to the Code
At this point, having discussed every aspect of the project, we can succinctly describe how to write the code. We'll do this in 3 stages of increasing granularity.
Firstly, we'll finish this tutorial with a natural language walk-through of the execution of the script.
Secondly, if you look at the source file included with the project, you will notice immediately that comments dominate the code. Because instruction is as important an objective as the actual operation of the script, I use comments in the source to provide a running narration. For those of you who find this superabundant level of commenting a distraction, I'm distributing a second copy of the source file with much of the comments removed.
Finally, there is the code itself. After all, source code is nothing more than a rigorously formal set of instructions that describe how to complete a task. Most programs, including this one, are a matter of transforming input in one form to output in another. In this very ge
Section 8: Natural language walk-through
- We need to prime the script by hard-coding some variable assignments. These 'configurable variables' serve the same purpose as the preferences, menus and dialog boxes typical of GUI applications.
Among other values we'll specify:
- The names of the KML output files
- The directory where the script should look for our photos
- and our Google Maps API key
Important: If you haven't yet, now is a perfect time to sign up for a key.
Before running the script, open the source file and look for the 'configurable variables' section of the code. Read through the comments describing each required value, make whatever necessary changes, and save the file.
- We need to create our KML output files.
If there is a problem which prevents the script from writing these files, we can determine that at the beginning of execution before doing any other work.
- The script starts reading from the 'fill in the blanks' template included in the data section of the source file, which is read via the __DATA__ pseudo-filehandle. (See the comments in the source for more information.)
The template looks like this:
__DATA__
<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="">
<Document>
<Style id="tutorial_custom_icon">
<IconStyle>
<scale>1.0</scale>
<Icon>
<href>!</href>
</Icon>
</IconStyle>
</Style>
<Folder>
<name>!folder_name_goes_here!</name>
<description>
!folder_description_goes_here!
</description>
!placemarks_go_here!
</Folder>
</Document>
</kml>
All of the literal text in the template is written directly to the output files.
The term 'literal text' applies to those parts of our template written directly into the source code, or in other words that are not part of the metadata extracted from the image files or computed values.
Notice that there are a number of placeholder strings in the template:
!custom_icon_url_goes_here!!folder_name_goes_here!!folder_description_goes_here!!
placemarks_go_here!
Because these placeholders are all unique strings in the template we can attempt to match against of them as we read each line and, wherever a match succeeds, substitute the current value of the appropriate variable for the placeholder. Note that some of these are from the configurable variables section of the file while others are values set dynamically during execution.
We could simply include the values from the configurable variables section in the template rather than assigning these values to variables, and then substituting the values for placeholders as described.
Handling it the way it has been presented here allows us to group all of the values we must set manually together in one small section at the top. This is considered to be preferable to scattering these values throughout the source because it makes it easier to configure the script and maintain the code. For example, it minimizes the chance that we'll break something by making an inadvertent change when we only intended to update one of these values. There are a number of other benefits as well, including these commonly noted advantages:
- These values are essentially options or directives we use to modify the behavior of the script. Grouping them together makes it more efficient to use our little command line executable, just as well organized menus make it easier to work with a GUI-based app.
- Grouping these values together makes it easier to examine them.
- Because the values are defined once and stored in variables, we can repeatedly use the same values at more than one point in the code without retyping them. Reusing values whenever possible, rather than recreating them, significantly decreases the likelihood of a typo or some other careless mistake breaking the execution of the program or ruining our output.
- Defining values only once avoids a common problem where two or more independent values that are intended to be identical get out of sync when the value is not updated everywhere it is used.
When we encounter the Placemarks section of the KML template (i.e. the string !placemarks_go_here!) we need to append one Placemark element for each of our image files.
The Perl module File::Find, which is part of the standard distribution, defines a number of functions for traversing a filesystem hierarchy starting at a specified root. One of these functions, find(), does a depth-first search of the directory tree, running a routine we provide it against every item found, where an item is any of the possible filesystem entry types (these vary among platforms but always include files and directories among others).
For example, Unix-like operating systems typically allow for seven (7) unique entry types including:
- plain files
- directories
- symbolic links
- named pipes
- block special files
- character special files
- sockets
These different types of entries and the differences among platforms aren't particularly important to us. For our purposes, there are only 2 categories of entries: plain files and everything else. If the item is not a file, we can ignore it regardless of type.
We need to define a routine that can be run against each item found. We provide find() a reference to the routine and a starting location and it arranges for the routine to run repeatedly, once for each item in the directory tree starting at the root.
From the documentation for File::Find
find(&wanted, @directories_to_search);
sub wanted { ... }
"find()" does a depth-first search over the given @directories in the order they are given.For each file or directory found, it calls the &wanted subroutine.Additionally, for each directory found, it will "chdir()" into that directory and continue the search, invoking the &wanted function on each file or subdirectory in the directory.
The wanted function takes no argumentsbut rather does its work through a collection of variables.
$File::Find::dir is the current directory name, $_ is the current filename within that directory, $File::Find::name is the complete pathname to the file.Don't modify these variables.
The documentation confirms that we need to supply find() a reference to a function, &wanted, and a list of directories to search, @directories_to_search.
@directories_to_search is easy enough, we want to search the folder containing our geotagged image files. This is the value of one of our configurable variables ($photos_dir).
Let's describe the routine that find() will run against each item in $photos_dir. Make sure you understand that the routine should be designed to run against only one item. If there are many items, find() will call the routine many times, but that is irrelevant as far as the design of the routine is concerned.
For a single item:
- First, the routine runs a series of tests to determine if it is:
- a file
- the appropriate type of file
- and readable
The second test will make a determination based on the filename extension. As written, the script considers any file that ends in .jpg or .jpeg (case insensitive) a valid file type. You can expand this list to include extensions for other types of image files or alternative extensions for jpeg files.
If the item fails any of these tests then we immediately return from our routine, essentially skipping it.
- If the item passes all of the tests, we use ExifTool to extract from the image's metadata only those tag values we're interested in (detailed above).
- Next check for the 'GPSLatitude' and 'GPSLongitude' tags among the list returned from ExifTool. If there are not defined values for these tags then the tag names will not be included among the list returned. Though the item may be a valid image file, without coordinate data we can't know where to position a marker representing this photo on a map, so there's nothing we can do with it in our mapping application. We have no choice but to skip it.
Note that any photo for which there is no coordinate data will be missing from our output files.
- Assuming we have at least the two tags 'GPSLatitude' and 'GPSLongitude', we use the tag values to build a Placemark element for the image and write the element to output.
We assemble a
<description> element for the Placemark by combining several of the tag values.
We can account for optional tags, which may or may not have been populated in the metadata for the image (the tag People for example) by determining if ExifTool was able to find values for the associated tag names. If no value was found, because the tag was not used, the corresponding variable will be undefined.
The Placemark description for each of our image files can be illustrated by the following block, which is another template:
<ul>
<li>People: $People</li>
<li>Event: $Event</li>
<li>Location: $Location</li>
<li>City: $City</li>
<li>State: $State</li>
<li>Country: $Country</li>
<li>Date: $DateTimeOriginal</li>
</ul>
<p>$thumbnail_img</p>
<p>$image_link</p>
Variables are replaced by tag values extracted from the current image file. If one or more of the variables is undefined, we simply do not include the entire line.
$thumbnail_img is a complete URL to a thumbnail which will be included in the description via the <img> tag's src attribute, e.g.
<img src='//dgdsbygo8mp3h.cloudfront.net/sites/default/files/blank.gif' data-
$image_link is a complete URL to the gallery detail page featuring the photo. We include a link to the page in the description, e.g.
<a href="$image_link">Gallery details page for this image</a>
As we have just seen, among the metadata tags we want to extract are the regional tags 'City', 'State/Province' and 'Country'. Assuming values for these tags are defined, we will include the information as part of the description in our <placemark>, which means we will see it displayed in the information window when we click the marker representing this image on the map.
We also use these tags to assist us with clustering in our Google Maps application and as a first step to implementing a feature called 'Regions' in Google Earth (more on the topic of Regions later).
Throughout the tutorial I've said that we will be generating more than one output file, but we haven't identified what these files are until now.
By default, the names of the output files generated in the script accompanying the tutorial are:
- photos.kml
- cities.kml
- states.kml
- countries.kml
photos.kml is the only file of interest as far as this part of the tutorial is concerned. It is the output file we've been discussing to this point. This is the file that includes one <placemark> for each of our geotagged image files.
What are the others?
In the source we declare the three hashes:
%known_cities%known_ states%known_countries
These hashes will contain one key for each unique region represented among the geotagged images (i.e. one key in the appropriately named hash for each city, state, and country represented).
The purpose of these hashes is to enable clustering of markers on our maps around large, well-defined geographical areas at lower zoom levels.
What is clustering?
Until we are above some threshold level, we will not be able to distinguish between markers for photos first within the same country, then state/province, and finally city, assuming we have more than one photo for the region. In these cases we create a single marker at the center of the region to indicate that there are photos available which cannot be displayed at the current zoom level.
- Before we return from the routine, we check the hashes to see if the keys corresponding to the tag values from the current image file already exist. If they don't, we add them.
For example, if the current image file is a photo of Boston, Massachusetts (USA), the following is a partial list of tag values from ExifTool:
City: "Boston"
State: "MA"
Country: "USA"
Let's say that the script has already come across a photo of someplace in the United States. If that's the case, then the key "USA" will already exist in the %known_countries hash. We don't have to add it again.
If we've also seen at least one photo taken somewhere in Massachusetts, then the key "MA, USA" will exist in %known_states; but let's say this is the first photo of Boston, Massachusetts. Then the key "Boston, MA" will not exist in %known_cities, so it will be added to the hash.
If this is all a little unclear, it may help to look at the source file.
After all items have been evaluated, find() returns. At this point in execution, we've generated a Placemark element for each image, and finished populating the regional hashes.
- For each key in the regional hashes, the script creates a Placemark element in one of cities.kml, states.kml or countries.kml (these are the default output filenames). For example, for every key in the %known_countries hash, a Placemark is created in the countries.kml file.
There are a couple of important differences between photos.kml and these regional files.
- There will be only one Placemark element for each unique regional location. So if we have 100 image files and all of them are photos of Boston, MA, then we will have a single Placemark element in each of cities.kml, states.kml, and countries.kml - not 100 identical Placemark elements in each.
- Because these Placemarks represent regions, not photos, there is no metadata for us to work with. How do we get a coordinate pair for each city, state, and country?
We have two options.
The Google Maps API includes a geocoder we can use. From the documentation we have this definition of geocoding:
Geocoding is the process of converting addresses (like "1600 Amphitheatre Parkway, Mountain View, CA") into geographic coordinates (like latitude 37.423021 and longitude -122.083739), which you can use to place markers or position the map based on street addresses in your database or addresses supplied by users.
We can pass the geocoder any one of our hash keys and it will return the geographic coordinates representing the center of the named region.
The problem is that it returns coordinates for what I'll call the 'computed center' of the region, which is not always what we want. For example, the following is a link to maps.google.com identifying the computed center of Massachusetts (my home state).
&ll=42.008489,-71.71875&spn=3.98356,7.03125&z=7&om=1
The arrow identifies the computed center of the state as reported by Google's geocoder, which seems to indicate that the center is along the southern border. This makes sense when you consider that Massachusetts includes Cape Cod and Nantucket, which are much further southeast than the rest of the state.
If you aren't bothered by these odd-looking but technically accurate computed coordinates, then Google's geocoder may be all you need. Also, if your regions are regularly shaped then this issues won't be apparent to you.
Alternatively, you could build a set of associations between region names and the geographic coordinates you want to represent these places.
As written, the script allows you to do both. In addition to using the geocoder, you can manually associate regional locations with specific coordinates in the configurable variables section of the code. Instructions are included in the source file itself. If you have manually placed a particular region, then it will use your coordinates, otherwise it will use Google's geocoder.
Note: Use of the geocoder here is the reason that you must include a Google Maps API key in the script.
These regional places are useful for clustering with Google Maps and Google Earth. Clustering in Google Earth requires the use of Regions which are not implemented in this tutorial. The topic of clustering will come up again in the Google Maps portion of the project.
- After generating all of our Placemark elements, there are a couple of lines from the KML template at DATA that must still to be appended to the output before we're finished. These are nothing more than closing tags for the <Folder>, <Document> and <kml> elements and can be written directly from our template to the the output files.
- At the end of execution we print a few statistics that have been collected throughout execution to help users identify problems, or confirm that the script ran as expected. These are simple counters or based on simple counters, but are useful nonetheless.
At the end of execution we have generated our KML files:
photos.kml, cities.kml, states.kml, countries.kml
Of primary interest is photos.kml.
Because we're not dealing with Regions for clustering in Google Earth, the regional kml files aren't useful for this part of the tutorial, but don't delete them because we will be using clustering in Google Maps. Should we decide to use Regions in the future, implementing the feature will be a much simpler proposition than it would be otherwise because we've already done some of the work involved.
Section 9: Finishing up
You can, and probably should, validate your KML files to ensure that they adhere to the standard. This doesn't take a long time and it's easy to do. If you've uploaded your files to a publicly accessible web server, visit the Feed Validator.
Once at the site, type the complete URL of your KML file(s), one at a time, into the text entry field and click the validate button.
A note about the results:
The validator makes a distinction between warnings and errors.
If your file conforms to the standard, but the validator has some suggestions for improving it you'll see the message:
Warning
This feed is valid, but may cause problems for some users.
We recommend fixing these problems.
If you scroll down the page you will see a message confirming that the feed has validated.
If there are actually errors in the file it will not validate, and you will see the message:
Sorry
This feed does not validate.
In either case, the validator will identify the lines in the file that produced the messages as well as an attempted explanation of the issue.
Once your feed validates, all that's left to do is enjoy browsing and sharing your photos.
Section 10: Sharing KML Files
- You can open the KML files locally.
Doubling-clicking photos.kml should open Google Earth. Alternatively, launch Google Earth, choose 'Open...' from the 'Edit' menu and use the standard dialog box to navigate the filesystem to your photos.kml file.
- You can share the file via email.
Because the files contain only text, you can depend on file sizes being relatively small. Remember that images are referenced in the file via URL not included directly.
- If you upload the file to a public web server, something you'll want to do in preparation for the Google Maps portion of the project, then others can access your images from within Google Earth by adding a Network Link.
However you choose to share the KML file, you can select the folder containing all of your photo markers in Google Earth's Places panel and 'tour' your photos. A tour moves from one marker to the next on the map, pausing for a short amount of time at each and (optionally) displaying the marker's information window.
Finally, and as a preview of what we'll be doing next, you can display your KML file in Google Maps in either of two ways:
You can visit the Google Maps site at maps.google.com and type the full URL to your photos.kml file into the search field on the page.
Equivalently, you can pass the URL for the photos.kml file as the value to that application's 'q' parameter.
For example, the KML file I've created for this project is located at:
I can display the data from the file in Google Maps at:
We can't change anything about the Google designed page, which works well for only small numbers of images. If you had hundreds of image files on the other hand, the 'Contents' list in the sidebar on the page would be very long and unwieldy. Also, we cannot incorporate clustering or other features available to us through the API.
Take a look at my kml file and you'll see the problem that clustering solves on display around the area of Boston, MA, where the majority of the 113 photos in the collection were taken. As you zoom in, the space between the markers on the map increases. At some point, you will be able to clearly distinguish between all of them, but at the default zoom level, there are a jumble of indistinguishable, overlapping icons.
Limitations notwithstanding using Google Maps and KML in this way is useful in its own right and a good demonstration of what's possible.
Section 11: Next Steps
What could be done to improve on this script?
The next topic you should familiarize yourself with related to Google Earth is 'Regions', which make it possible to efficiently work with very large datasets in Google Earth, even when publishing data over a network.
Regions define areas in Google Earth in terms of a 'bounding box' and 'levels of detail'.
The bounding box defines the northern, southern, eastern, and western boundaries of the user's current view of the Google Earth map (and also possibly min and max altitude values if appropriate, for example when working with 3D objects).
Level of detail describes the number of pixels the Region currently occupies on screen. Keep in mind that even a large area (large in terms of coordinate boundaries) may occupy only a small portion of the map at low zoom levels. Of course the reverse is also true, i.e. very small geographic areas may fill a large portion of the display at the highest zoom levels. Or we can say that at high zoom levels the resolution of the view is much greater than it is at the lowest levels. As a result of this higher resolution, more markers can be distinguished on the map but the field of view is decreased. At lower resolutions adding a lot of detail to the map, i.e. many Placemarks, only results in overlapping, indistinguishable markers. Google Earth and your computer must work harder to display these Placemarks. In the case of Network Links, all of this data must be transmitted over relatively slow connections, significantly reducing the performance of the application and detracting from the appeal of using Google Earth as a solution for sharing your photos.
This discussion may sound confusing the first time you read it. Suffice it to say that it makes little sense to produce markers for photos on your map that are not visible to your audience because they exist outside the current view or because they are obscured by other markers. Regions solve these issues by ignoring data that is either outside of the current bounds of the map, or indistinguishable because of the current resolution. While we are dealing with only hundreds or thousands of images, this isn't necessarily a significant concern. At the point when we are sharing tens of thousands of photos, or perhaps hundreds of megabytes worth of data, Regions are critically important.
Because the use of Regions is not necessary given the scope of this project (hundreds of photos) I'll not pursue this topic further. I can say that you will be able to incorporate Regions into the rest of this project if you choose to do so. Nothing that we do here precludes the use of Regions or complicates their use.
Note that we will be implementing a similar feature in Google Maps, because the issue of inefficiency is a problem with far fewer markers. Google provides a 'marker manager' that is similar in functionality to the Regions feature in Google Earth. Allowing the application to ignore photos outside the current view and 'cluster' (i.e. reduce the number of markers generated) in response to the resolution of the map at various zoom levels. The discussion of the marker manager should help you with the concept of Regions.
Section 12: Conclusion
We've finally come to the end of the Google Earth project, which is actually the first half of a 2-part tutorial. If you haven't looked at the source code yet, now is the time.
If you followed along with the tutorial, the code will seem very familiar to you. Also, if you have some questions after reading through the tutorial, the extensive comments in the source should help. If you've successfully generated your KML files, congratulations. If you're still working on it, and especially if you're having some trouble, I would encourage you to keep at it. Maybe take a break and come back. It's always been surprising to me just how much clearer a problem becomes after spending some time away from it doing anything else.
I hope you continue on to the second project, where we look at building a Google Map from the KML files that we've generated. In the process we'll cover the Google Maps API, Javascript, object models, the DOM, and we'll talk a little bit about XHTML.
See you there.
>> Continue Reading Google Earth, Google Maps and Your Photos: a Tutorial Part II
Notes:
1 Exif: (E)(x)changeable (i)mage (f)ile format. EXIF.org is an unofficial site that at least links to the specifications. See the appendix for more information about the Exif standard.
2 (I)nternational (P)ress (T)elecommunications (C)ouncil. See the IPTC entry at Wikipedia for more information:
3 XMP home at adobe.com:
4 The ExifTool homepage at is the best reference for the project. Other than a tutorial like this one, you shouldn't need another resource.
5 RFC 1630:
6 The name CDATA is formed from the phrase 'Character DATA'
Downloads:
Rob Reed has over 10 years experience in the IT industry. He has held positions as the Director of Technology, IT Director, and Research and Development Technical Lead for various firms. Currently, he is an independent consultant and is pursuing a graduate degree in Computer Science at Tufts University (Medford, MA). His thesis work looks at issues of semantic information management. He is also interested in the topics of computing platforms, network and systems security, programming languages, and development. He maintains a weblog at. He can be reached at robreed@gmail.com.
About the Author :
Books From Packt
Post new comment