[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, {"document_url": "http://www.danielschneller.com/2007/", "unformatted_src": "http://bp1.blogger.com/_jZVk_i0f9PM/Rmq0dLCR4LI/AAAAAAAAABY/SDMKdW4U_x0/s400/hdd.png", "src": "http://bp1.blogger.com/_jZVk_i0f9PM/Rmq0dLCR4LI/AAAAAAAAABY/SDMKdW4U_x0/s400/hdd.png", "formatted_filename": "hdd", "alt_text": "sample graph"}, null, null, null, null, null, null, null, null, null, null, null]

OS X / iPod Touch DHCP timeout problem

It has been some time since I last posted, but I spent the time well, especially on a vacation in Malta. Very nice place, even in late November. Apart from one rainy day we had great weather and temperatures around 20°C. Very comfortable. Another nice aspect was the display of prices in both Maltesian Lira (Lm) and Euros, because as of January 1st, 2008 Malta will change their currency and introduce the Euro as well. Very convenient for travelers :)

Back to business: Of course I did not want to be cut off from the net completely, so apart from my girlfriend's notebook I took my iPod Touch with me. The hotel advertised free WiFi access in the lobby. The first day I went down there and connected to the network without any problems using the iPod. They provided a freely accessible, unencrypted access point.

Two days later I could not get a working link. The iPod detected the wireless network and was able to join it, however I never got an IP address from the DHCP server but ended up with one of those IPV4LL (better known as APIPA) addresses that usually don't do you any good...

Trying and renewing the DHCP lease did not help, so I decided to take the hard way and ask someone at the front desk. I was surprised when the guy I talked to showed some basic knowledge of DHCP and asked me to wait while he checked the router. A moment later he came back and told me he could not do anything, because on Sundays the technical hotline available for the staff was not available. He gave me a complementary hour of Internet access on one of the for-pay computers in the business lounge and asked me to check again on Monday.

So I did - and still had the same problem. So I went back to the front desk, again asking if they had any problems with the router leasing IP addresses. This time the person at the desk was less knowledgeable and told me "We do not have IP addresses. Just set everything to automatic and it will work.". Very helpful. I decided to try again and explained to her in layman's terms that my device being set to automatic would result in it asking their WiFi device what to do and that this did not work. She recommended moving over to the business area and try again, because "the wireless network is installed over there". Imagine the front desk being approximately 15m away from the business area. Just three steps down, no walls etc. and my iPod showing a perfectly fine signal strength for the "Lobby Wireless" network. Anyway, because she had asked nicely I went over there - and guess what - this time got an address when I hit the renew lease button...

While she was happy that she had been right and helped another one of the tech-illiterate guests, I was both happy that I finally could check my mails, but also a little confused. Trying again later with the XP notebook it dawned on me that this was simply a very slow DHCP service and apparently the iPod is not too keen on waiting longer than approximately 5 seconds (rough guess) before falling back to Zeroconf.

I went looking around the net for someone with similar problems. Turns out this does not seem to be a iPhone/iPod issue but a general Mac OS X thing. I have not tried it out (do not have a Mac), but maybe a change to /etc/dhclient.conf could work, specifying a longer timeout value - assuming that OS X uses dhclient...

Posted by Daniel Schneller am 13:23 2 comments: Labels: apple, ipod, network


New GMail rather slow

I noticed the new GMail (the one where an "Older Version" link appears in the top right part of the window) seems to be way slower than that "older version". Whenever I use the mouse to scroll through my inbox I can see the browser slowly redraw the screen from top top bottom.

Is this normal on a 3GHz HT machine? I surely hope this is still pre-release code :)

Posted by Daniel Schneller am 13:21 1 comment: Labels: firefox


iTunes depending on Internet Explorer cache?

Today I noticed that although in Firefox I could see the latest episode (#150) of The JavaPosse iTunes did not notice it, even when I called the "Update Podcast" menu. It still insisted on episode 149 to be the most recent one.

I reloaded the Feedburner feed with F5 and Strg-F5 to no avail. I suspected our company proxy to somehow misbehave and launched Internet Explorer to check what I would get there. Usually I do not use IE, so I could be quite sure to have it request the feed through the proxy and not serve it from its own cache.

To my surprise I immediately got the right, current feed displayed, including Episode #150.

Once I had seen this in Internet Explorer I tried "Update Podcast" in iTunes again and this time it started to download the episode it had not seen before.

I have not bothered to look through the iTunes documentation, so maybe I am writing about something completely normal here, however for me it is definitely counter-intuitive and took me 10 minutes to figure out by trial and error...

Posted by Daniel Schneller am 00:31 No comments: Labels: apple, firefox, internet explorer, itunes, windows


Upgrading Windows XP Home to Professional

Some time ago a friend of mine upgraded his little office network with a Windows 2000 server. Since then he had been using Windows XP Home edition - mainly to just access SMB shares on the server. There were however some pain points with that setup. Profiles were not stored on the server, so the backup to tape drives on the server could not include them. Moreover we had to manually take care of having identical user accounts on all machines to share data among them and grant access to each others printers.

So we decided to upgrade to Windows XP Pro, set up Active Directory and use roaming profiles. However because there were several products installed on the different workstations that take a long to time to configure properly we did not want to start from scratch.

Unfortunately with XP you cannot just change the product key and all of a sudden Home becomes Professional. For one the keys just do not match - there are even different keys for the retail XP Pro version and the corporate volume license installs...

So this is what we did to get to XP Pro without having to install everything anew. (Notice that all programs and settings remain intact. You will only lose all Windows updates since the state on the installation medium. You might also have to install some drivers again after the process, depending on your hardware)

Prepare the machines for upgrading

The basic idea is to do a "repair install" from the freshly bought XP CDs. That will overwrite all system files from the CD, so along the way you get the missing parts that Home does not have. However this puts you back into the stone-age, because the last XP CDs have a copyright date of 2004 and contain no fixes apart from SP2. So when the first machine failed to boot except for safe mode because of the 2007 nVidia graphics drivers not matching the 2004 system we uninstalled the drivers from the other machines upfront.

Perform "repair install" of XP Pro

Boot from the XP Pro CD. You will be asked if you want to use the repair console to fix problems with existing installations. I find this somewhat irritating, because we do want a repair install, but not with the repair console. After accepting the license with F8 the setup program will find the Windows on your hard disk. This time choose to repair that installation. At first I was somewhat shocked when I immediately saw "Deleting file:" messages flying through the status bar without further confirmation. However from here on you just have to wait. It will take about the same time as a fresh Windows install.

During the graphical phase of the installer you will possibly be asked to confirm or deny drivers that have not been digitally signed by Microsoft. These drivers are part of the Windows installation being upgraded/repaired. You should indeed follow the advice given in the confirmation box to NOT use them anyway. In our case this led to a machine blue-screening on boot because of the graphics driver! You are better served to reinstall the drivers when the upgrade is complete and all recent Windows updates have been applied again.

Optional: Prepare Offline Windows Update

As the Windows setup gave me some time to wait I decided to prepare an offline update CD to get the new installations up-to-date quickly. In case you have a real broadband connection this is not really necessary. As we only had a 1MBit DSL line at hand, downloading more than 80 updates plus several drivers on 5 individual machines did not seem like a good idea. So I downloaded "c't Offline Update" - a nifty little tool developed by the German c't magazine. The page is in German, however there are some links to English resources, too. I used the current version 4.1.

Basically what the tool does is get the list of all available upgrades for the products you select (Windows and Office in several versions and language versions) from Microsoft and downloads all updates automatically. They are then packaged into a CD or DVD image you can burn and use to update any machine without a network connection.

Install Windows Updates

As the CD install replaces lots of system files with outdated versions it is imperative to re-install all Windows updates that were lost. You can do so by either using the Offline medium created earlier (see above) or by simple visiting the Windows Update site via Internet Explorer. I noticed a problem after I had installed the CD on all machines: For some reason about 5MB of updates had not been installed offline. They showed up as new updates on windowsupdate.microsoft.com, however I could not install them. Windows Update always seemed to start but almost immediately failed with (I believe, did not write it down) error code 0x800000002.

To get Windows update working again I needed to re-register some DLLs. Fortunately I found a script that does this automatically. I duplicate it here, because I have often seen forum links go dead after some time. However I would like to make clear that I am not to credit for this. The messages in the script are in German. They basically just say: "We are going to re-register some DLLs to repair Windows Update. We do not take any responsibility in case your computer does not survive etc."

Reinstall drivers

The last thing I had to do was install some drivers. For once the ForceWare drivers I had removed before the repair installation had to be restored, and on another machine I re-installed the AVM FritzCard drivers. However scanner drivers and even some special serial ports setups worked flawlessly without any manual intervention.

Posted by Daniel Schneller am 23:07 No comments: Labels: hardware, network, nvidia, office, registry, software, windows


Leverage Eclipse perspectives for different screen layouts

At work I use a three monitor setup for Java development. The machine is based on Windows XP and Eclipse 3.3. The individual views are spread out across the middle and right screens, the left one is mostly used for browser and email windows. Because I usually use a rather large font for code editing the center screen is entirely dedicated to the code editor.  The right screen contains outline, call hierarchy, source history, a console view, the package explorer and the task/warnings lists. All in all each and every pixel of this 2560x1024 area is used effectively for Java development.

However in the evenings I sometimes like to complete some unfinished task at home. Thanks to a VPN connection I have full access to the CVS server and everything else I need. However I do not want to have to synchronize and download all changes to the laptop before I start. Usually I leave open Eclipse on my office PC all the time. To save time I just connect to it via RDP. The problem with that is however the limited resolution of the laptop screen - it is only 1400x1050. I used to have VNC running and scroll left and right to see all of the office screens, but it is just to slow and there are keyboard issues with the notebook keyboard.

The main disadvantage with this setup was Eclipse - because the RDP resolution is smaller than the office desktop I could only see a small part of the views on the right screen. The first few times I reset the Java and Debug perspectives to their default configuration which docks all views inside the main window. While this worked, it is a real pain to rearrange the views again when I came back to work the next morning.

It took some time until I remembered the "Save perspective" feature. I have now set up my "normal" Java and Debug perspectives - stretched out across two screens - and a "single window" version for each of them. The only thing that needs to be configured now when I switch from desktop to laptop mode is found in the Window - Preferences - Run/Debug - Perspectices menu: When working from home I define the "simple" perspectives as the defaults, at work I choose the multi-head versions.

Posted by Daniel Schneller am 21:24 No comments: Labels: eclipse, java, windows


How to charge the iPod Touch's battery on Linux

I have never been very much into iPods before. Although people were all excited about the click-wheel and the great user experience I tried some of them and never shared the whole fuzz.

However when I first heard about the iPod Touch I was really fascinated. I watched the iPod Keynote from September 2007 and immediately like the iPod Touch. Usually I am not a person who buys products immediately after their release, but I figured they had had some time to get the worst bugs out of the iPhone; and as the iPod is undoubtedly nothing more than the "phone without the phone" I decided to give it a go and ordered it from the Apple Online Store on September 9th. Delivery was scheduled for the week starting Oct 1st and it promptly arrived on Thursday. As I had it delivered to the office address I suddenly noticed a lot of people coming to my desk...

I connected it to the office desktop (XP) where I had been using iTunes for quite some time already to manage the music I usually listen to while coding. Syncing was a breeze and I was really happy.

Because I did not have time to fully charge it at work and of course played around with the WiFi (which is a little heavier on the battery than just listening to music). So I decided to fully charge it over night at home. However that turned out a lot more difficult than I had imagined. Up to that point I had always considered charging a mobile device via the USB port something nobody could do wrong. However when I plugged it in after I had fired up my Ubuntu desktop it just very briefly showed the "charging" icon on the iPod's screen and immediately went back to "working on battery". The Gnome desktop presented me with a camera import wizard... I tried to disconnect and reconnect, even disabled all udev rules because I suspected one of the programs started upon the discovery of the new device to somehow confuse it - all to no avail.

When I booted into the Windows partition I still keep for emergencies, it started to charge immediately... Now this is some kind of platform tie in! I knew before that Apple's support for anything but OS X or Windows is virutally non-existant, but preventing people from charging the battery, just because of the "wrong" operating system?

It took me four more days, pounding Google with all sorts of keywords I could imagine - "ipod charge linux", "ubuntu ipod battery" and so on. Only when I got the idea to search for some combination of terms that contained "iPhone" I got to this: iPhone Now Charges in Linux.

That site contains the source for a kernel module that will send a short sequence of bytes to the iPod or iPhone that will tell it to start charging the battery. I find it very annyoing that stuff like this necessary to just recharge a device... Thanks to Matt Coyler I do not have to boot Windows just to charge my MP3 player. Great job!

Posted by Daniel Schneller am 23:53 2 comments: Labels: apple, gnome, hardware, ipod, linux, software, ubuntu, windows


(Also see the follow-up post about some progress)

Today I was (again) facing a log file from a machine that had for some reason not been able to start a temporary MySQL daemon during the night to prepare for a streaming MySQL slave installation. The necessary 2nd daemon had created its new ibdata files, however just after that aborted the startup process with the following message:

As you can see, the port is a different one from the default MySQL port, so I can be sure there was no conflict with the primary instance. Even more curiously the same process has been working on that and other machines flawlessly for some time. However I remember having seen this message once before, but back then I did not have the time to look into it any further. We just restarted the streaming slave setup process and it went right through.

This time however restarting that process didn't work. It just aborted with the same message again. I especially wondered about the error message: "Bind on TCP/IP port: No such file or directory". What the hell is that supposed to mean? A colleague and I even had a look at the MySQL source code, but the "No such file or directory" message is not to be found in conjunction with the bind error message. I looked on the web but could not find any explanation where that comes from.

However because the process would fail repeatedly I had a look at port 3310:

You can tell this is a Windows machine. Use findstr is a poor man's grep for Windows. However what's more interesting is that there is a connection on 3310. However this is not due to someone having already bound that port for some sort of a service, but instead it is in use as a local endpoint for a connection to a SQL server instance (port 1433)!

It turned out to be the same problem we had some time ago with JBoss not being able to bind port 1099 on Windows 2003. We had not explicitly reserved port 3310 for private use by a user-controlled application, so Windows was free to assign it as a local endpoint to any process requesting a socket connection. This is probably due to the range of so called ephemeral ports. The name stands for a range of ports that make up the "local" end every time an application connects to a remote service. The default bounds for this range are from port 1024 up to 4999. As 3310 is right in the middle of this range, apparently chances are big enough to get bitten by this more than once in a lifetime. We have now added port 3310 (and 3306 for that matter, which was also missing) to the list of ports excluded from the dynamic assignments. Probably 3306 has never been a problem, because the primary MySQL instance is configured as an autostart service. I guess it starts early enough during boot so that the chances of anything else having claimed that port are very low. However there was most probably some luck involved, too...

This might also be a problem if you need a lot of outgoing connections on a machine. To configure the range for Windows 2003 Server, have a look at Microsoft's documentation on TCP/IP stack implementation details, specifically the section on the "TCP TIME-WAIT delay" in the Core Protocol Stack Components and the TDI Interface chapter (for Windows 2000 see this page). They are both linked from Knowledge Base article #908472

We will monitor the situation and see if we get any more troubles with this.

I still however do not get the "No such file or directory" part of the message...

To round things up here's another link on the topic of ephemeral ports and their meaning for network security: www.bsdcan.org/2006/papers/ImprovingTCPIP.pdf.

Posted by Daniel Schneller am 21:39 5 comments: Labels: database, mysql, network, standards, windows


The value of a CVS commit database

Due to some discrepancies between the Eclipse 3.2.2 compiler and Sun's javac we needed to upgrade our development environments to Eclipse 3.3. Otherwise we could not tell for sure that something that looked ok in Eclipse would compile in the daily build process.

Even though I had used 3.3 privately for some time now, there is always some tension when switching a whole bunch of developers in a really large project.

At first everything seemed fine, apart from some minor issues that could be easily worked around.

However I ran into a nasty little bug regarding the CVS integration when I had to switch a workspace that had been checked out on HEAD to another branch. That branch had been created to keep the Helpers- and Utilities-Refactoring I wrote about before separate from the HEAD until it is complete.

Within Eclipse you can just select "Switch to another branch or version" from the context menu on any project (or file for that matter) and select the branch you would like. I had done this with 3.2.2 several times without any problems. So not suspecting any problems I switched my local working copy to the branch and started checking in modified files after some more refactoring.

However shortly after I had done that a colleague complained that there were compile errors on HEAD now. Turns out that Eclipse 3.3.0 has a well hidden bug in that feature: The version switch involves some requests to the CVS server. This works alright for files that have no changes in the working copy, however for files with outgoing changes the server response is not handled correctly and those remain on HEAD. Because I had made some changes before the switch already, part of my changes went to the branch, the rest of them to HEAD, leaving both in an incomplete state. For details on the bug see Eclipse Bug #192392.

The files I had checked in spanned several projects and where of course committed in little chunks with different comments. At that point I was very glad that I had my ViewVC commit database to query for anything I had done over the last few hours that had gone to HEAD. Without it it would probably have taken me hours to just find out which files I had checked in on the wrong branch. While it was still some tedious work to actually restore everything back to the state I wanted, just identifying the files affected was done with a rather simple SQL query in no time.

I can only advise anyone working on a project with more that just a few files to set up a database that stores all commits by type (addition, change, deletion), file, branch, date and author. This wasn't just my life insurance in this case, but in combination with a full text index on the commit comment field it is also a very good base for change logs - using simple SQL they can be generated very flexibly and within just the blink of an eye.

The version of ViewVC we use is rather old and contains some custom changes that probably would not be required with a more recent release. So I recommend taking a look at the current version the project offers.

Posted by Daniel Schneller am 18:09 9 comments: Labels: bug, database, eclipse, java, mysql


Helpless Helpers and Useless Utilities

With any code base of a reasonable size there are lots of issues you would normally take care of immediately when you come across them, however often there is just no time for it. In the end you will have to live with the knowledge that you had to leave some ugly hacks in it just to meet the deadline.

Because we have recently finished development of the next major release of our software product, there is some time now to do code cleanup and get some more automated tests on the way. Because one of the bugs that almost prevented us from holding our schedule was a particularly nasty - but well hidden one - there has (again) been some discussion about coding guidelines and quality.

People always seem to agree that you need to talk to each other, think in larger terms than just your specific problem at the time and strive for code readability and re-usability. For starters I personally would sometimes even do away with just a little more of the former one...

The bug I mentioned above was in a method to determine whether a given date is a holiday or a business day. Because this is not a trivial problem to solve generically for several countries, we rely on a database table that is pre-populated with the holidays for the next few years in a given country. To tell whether or not a date is a holiday is just a mere database lookup. Nothing can really go wrong here, can it? The method in question was implemented like this:

Of course there was a corresponding method for business days, too:

Just in case you do not know: Saturday is usually considered a business day in Germany, for our customer it definitely is. So this part of the latter method alone is at the very least somewhat dubious.

By now you will probably have stopped looking at the code, trying to find out what it is you miss to make this work correctly. You are right to do so, because it is of course just plain wrong. The isHoliday method returns just the opposite to what you would expect. And the isWorkday code - apart from the wrong condition for the DAY_OF_WEEK even leverages this and works in fact (almost) correctly.

One might ask how this could go unnoticed till the last moment of development and I can tell you, the answer will not be pleasant: It didn't!

There were about 20 calls to the holiday method. 19 of these always inverted the result! The author of the 20th call had relied on this seemingly simple method to just work and only found out about the bug when he accidentally connected to a database with an empty holidays table and still got true out of it.

Both of the above methods were part of a class called "DateUtil". Apparently someone had had the idea of providing common date functions with this class, to spare everyone from writing their own methods to manipulate and use dates.

While this, of course, is generally a good idea it would have been even better to provide some thorough testing code for it, because even in the seemingly simplest of methods you have a sufficiently high chance of making a mistake while implementing it. One could point a finger to the author of that class, because he introduced the bug in the first place. But I would rather have 19 more fingers to point at each and every one who noticed the problem and just worked around it in his or her private piece of code, telling no one about the problem and letting everyone find out for him- or herself again. I will not even try and estimate how many hours were spent uselessly because of this...

But it did not end there. Alerted to this I decided to have a look at the project source tree. In the course of development I had already seen classes like StringUtil, CalendarUtil and others. So I just ran a find workspace/ -regex ".*(Util|Utility|Utilities|Helper)\.java" to look for classes matching that pattern. I ended up with over 160 helpers and utilities. Many of them were obviously duplicating functionality just by looking at their names. Several PersistenceUtilitys, StringHelpers and the like. Looking through them they often contained very similar methods, re-implementing the same task over and over again.

Because practically none of these were covered with (J)Unit tests but had only been empirically tested on the precise context of their calls, you could not even just go ahead and replace calls to a similar sounding method in a more central place - and vice versa, making almost all of them virtually useless from a more general perspective.

So the whole idea of creating utility and helper classes had in fact been reduced to absurdity. Right now we are in the process of looking through all of them and consolidating to a handful of really helpful helpers and useful utilities - each of them accompanied by a set of unit tests to prevent another "a holiday-is-a-business-day" problem in the future. We will see what other gems bubble up and see the light of day in the process...

Posted by Daniel Schneller am 21:22 No comments: Labels: bug, java, software


Hard disks are too big

I remember when I got my first hard drive. It was part of the first PC machine I ever had, a 386DX40. Before that I owned an Atari 800XL and an Amiga 500. While there were hard drives for the Amiga I couldn't afford one at the time. The 386 came with a 52MB Quantum. Back then I had to decide whether I wanted to have Windows 3.0 on the disk or Wing Commander 2.

While I'm not really too eager to go back to those times I recently noticed (again) that the vast amounts of disk space we have today are not always a good thing in a manner of speaking.

Over the years I owned several computers, with each new one came a new, bigger hard drive. As I usually sold the old machine - and the smaller hard drive with it - I just copied (almost) everything into some sort of "old_disk" folder on the new machine. When the next upgrade came, I had (of course) not reviewed everything in that folder and so, just to be on the safe side, just put it into the next "layer of archive" directory.

Currently I am using a Pentium IV machine with three hard drives. An older IDE drive with 80GB, one SATA 250GB and a third one with 400GB, also SATA. When I first installed Ubuntu in the summer of 2006 (Dapper Drake) I decided to give it the whole 80GB drive. To make space I just moved all the Windows data on that disk to the 250GB one. (Guess what... "80gb_disk" the folder was called).

Now, a little more than a year later, Ubuntu has replaced Windows as my primary and everyday operating system. I still keep Windows for the occasional game of Far Cry, but apart from that I do not really need it anymore. The last piece of hardware I cannot get to work with Linux is the CanoScan LiDE 70 scanner, but that will be replaced soon. Because the 80GB was filling up steadily I decided to do it properly this time and dedicate the whole 400GB disk to the /home partition.

Of course I needed to get the data off there, but as it was more than the already crowded 250GB disk could take, I finally had no choice but to wade through the data and see what I could throw away.

It was just unbelievable. I found things I had completely forgotten about; letters, pictures, stuff from school and university I never thought I had kept in the first place. It was like going through those boxes that somehow seem to magically appear in the attic and the basement every time you move to a new place. Some of the things were really great to find - like meeting someone you haven't seen for some time.

However most of the space was just taken up by old Windows installations, "temporary" download folders and rotting user profiles. Do you know that feeling when doing the occasional re-install of Windows to better keep the old installation somewhere, in case you need that one special settings file or have to take a look at the old registry again? Or let that ISO image lie around and burn it to a CD or DVD later and then never do it anyway? I found at least 4 Outlook PST files - apart from the last one I really used, of course

It took me more than 6 hours to go through it all and this time really deleting stuff. Most of it was complete trash and absolutely useless. I still kept all of my documents, music files and other "handmade" stuff. In total I removed around 290GB of old cruft...

Imagine yourself back to the 386 time, would you have believed anyone, had they told you that you were going to have roughly six thousand(!) 52MB disks, filled up with nothing but useless junk? :-)

Posted by Daniel Schneller am 23:36 No comments: Labels: fun, hardware, linux, ubuntu, windows


ERROR 1033 (HY000) on InnoDB configuration error

One of the key features MySQL often uses to advertise for their database is the modular architecture that allows them to have different storage engines below the same SQL layer. In practice the application and/or database designer can choose from a variety of low level data storage implementations that each offer different characteristics and may be chosen on a per table basis. (Even though I personally believe most designs will use one type of table for all tables of a particular schema).

The idea behind this is that for example people who do not need transactions should not have to worry about them at all – maybe there is a performance impact involved which they cannot afford to take. Moreover some specialized types of index or column might not be available on all engines. Basically the concept is very interesting and can be really useful for developers.

However there is a weakness that in my opinion needs some severe work to be done: The interface between the common SQL layer and the storage engines seems to be somewhat limited with respect to what storage engines can do to inform the level above about status and error conditions.

For example there is no (elegant) way to find out about the details of a constraint violation problem when using the InnoDB storage engine. While you will get an error message that some statement failed due to a violation of referential integrity constraints, you have to use the generic “show engine innodb status” command to get some details. However this will not only tell you about the error you care about at that particular moment, but will also give you lots of information on lots of other stuff inside InnoDB. This is however necessary, because you do not have any other means of find out out about those – e. g. when you are investigating a performance problem.

From what I learned from a consultant some time ago this is due to the limit interface specification through with MySQL itself (the upper layer) and the storage engines talk to each other. Because this protocol has to be somewhat generic messages from the bottom to the upper levels have to be somehow wrapped into some form of special result set which you then have to parse and understand on your own. Moreover if memory serves me right, there is a limitation on how much data can be transferred at a time (could be a limitation of the client as well). Because of this you will not even always get a full InnoDB status output, because it will be truncated if it gets bigger than 64k.

While this is not particularly nice it is a limitation I believe is acceptable, especially in the case of InnoDB, because the innodb_monitor feature allows you to get the full output into a log file.

What I consider much worse however, is that error messages from the underlying storage engine are often mapped to more generic MySQL messages in an unpredictable way.

Time and again I have run into problems that present you with an error message that has nothing to do with the actual problem. For example in a replication scenario you might get an error message 1236, claiming that there is something wrong with replication position counters, but it turns out that this message can also mean a full disk on the master. If you know enough about the implementation details you might see the way this message comes to pass, but if you are troubleshooting a production system this is not what you want to do. Moreover I tend to forget these peculiarities if they are just seldom enough. Just recently I found a machine spitting out strange errors on each and every query I issued (InnoDB):

Now, just having this message, what would you expect is wrong here? File corruption? Broken SCSI controller? Faulty memory?

When you use Google to search for “ERROR 1033 (HY000)” you will get all sorts of results, most of them suggesting to try myisam_check (not very useful for InnoDB) or the REPAIR statements. Often you will find someone who claims that restoring from the latest backup might be the only option.

While all of this is certainly true to solve the problems that this error message what originally intended to report, in my case they were all just leading in the wrong direction.

Turns out that something was wrong with the my.cnf configuration file. This was on a machine set up using the “Streaming Slave Deployment” mechanism I described in an earlier article. For some reason the script that usually adapts the config file automatically after downloading the data files had not been started, so a default my.cnf was still in place.

Unfortunately the InnoDB data file sizes did not match those downloaded from the master server. This is what my.cnf contained:

This is a listing of the data directory:

It may not be obvious, but 555745280 bytes for ibdata1 is not 512MB, but 530MB. Nevertheless the MySQL server started even with this wrong configuration. However every statement would fail with the message above.

Shutting down the server and correcting the line above to

restored everything to a working state:

While I really like MySQL and find it generally rather easy to configure and get very high performance, this is definitely a major weakness I would like to see improved in future versions. For the time being I will try to make a post about anything that strikes me odd enough for someone to be interested in, too :)

Posted by Daniel Schneller am 23:36 18 comments: Labels: database, mysql


Comfortable XPath accessor

Three weeks ago I blogged about a "Groovy way" to create XML documents in Java. This article now is about a convenient way to access XML without requiring more than a single class (downloadable here) on top of the JRE's own XML library functions.

In fact I wrote this class before I even looked for an easy way to create XML myself, because at the time I just had to parse some XML and extract the values in an easy to use fashion.

I believe it is easiest to show an example of how to use the class. Consider the following very simple Java class:

Usually the fields would not be public of course, but for the sake of the example, just imagine the getters and setters being there ;-)

Now consider getting an XML string back from an external system that you cannot change:

What's the easiest way to get this into the “Contact” class above, without using persistence frameworks, mapping tools and the like? What about this:

I find this rather straightforward. Of course the Strings should be declared as constants somewhere to prevent typos.

The example is really simple, because we just extract some plain values from the XML. However, you have the full power of XPath at your disposal. This means you could do something like this:

The XPathAccessor instance could (and should) be reused, of course, depending on how often you need to access the document. As it caches XPath expressions that have already been used, it saves some cycles to re-use it for all accesses to a particular document.

Getting the “historyNode” first is of course not really necessary in this simple case, however sometimes it can come in handy to keep the expressions readable when you need to access deeper parts of the XML.

As with the XElement class, feel free to use this one, too. I would be happy to get feedback and/or improvements.

Posted by Daniel Schneller am 00:49 No comments: Labels: groovy, java, xml


How the Vulcan greeting came about

Ever gave a thought on how the Vulcan greeting (you know, the V shaped hand gesture) came about in Star Trek? Turns out this is really a fun and interesting story, not just some script writer conceiving it out of thin air. Have a look at this very funny video in which Leonard Nimoy, Mr. Spock, explains how it entered the show.

Posted by Daniel Schneller am 23:45 No comments: 

Windows Date Created Timestamp strangeness

I have been using Windows since version 3.0 and thought  I had seen most of its subtleties. However today I found a new "gem" I had not encountered before.

We use two Perl scripts to do some FTP transfers regularly, scheduled by the Windows "Scheduled Tasks" to run several times a day. The scripts both use a common function to append to their respective daily log file. In case a file is older than 5 days, judging by its age in days based on the "Date Created" timestamp, it will be deleted and a new one created under the same name. This is intended to not have the files grow indefinitely.

While one of the jobs worked just fine, appending to its log files and rotating after 5 days, the other seemed to overwrite its log file on each run. Strangely enough  - as said before - they both used the same log function, just with different file names.

After some poking around in the scripts source we decided to make a more low level test to see whether this had anything to do with either Perl or the "Scheduled Tasks". We created a file from the command line by just redirecting an echo command. We then deleted the file and recreated it again by the same means. This is what we got (German Windows XP, but behavior observed was equivalent to Windows Server 2003 and Windows Professional 2000):

Notice the experiment starts at 22:46. A file is created and its creation timestamp is automatically set to 22:46 (dir /T:C shows the creation time). Then, at 22:47 the file is deleted and recreated with different content. Again the file creation date is 22:46 while the "Date Modified" timestamp is 22:47.

Looking at the timestamps of the log file mentioned above we noticed that the "Date Created" attribute was back several months. Apparently every time the file had been deleted and recreated the original timestamp had been restored as well.

Looking around the net for some explanation for this (we suspected a filesystem bug) after a fair amount of searching I finally came across "The Old New Thing: The apocryphal history of file system tunnelling" and through that Microsoft Knowledge Base article 172190, titled "Windows NT Contains File System Tunneling Capabilities".

Basically what happens is that Windows caches the timestamp of the deleted file for some time. In case a new file appears under the same name, it will get the cached value. The default maximum time period between deleting and re-creating the file is 15 seconds. However we could also see it happen with more than 8 minutes! I do not yet understand how this comes about...

Once I knew the keyword was "tunnelling" it was also easy to find this 2001 post from a discussion on Bugtraq. I completely agree with Ken Brown who wrote that post (especially concerning the 2nd paragraph):

"Tunnelling" is a long way from any keywords that I'd associate with file systems - and a search for "tunnelling and ntfs" turns up a great many references to VPNs and bits of networking. It now turns out that it isn't really a property of the file system at all, which obviously makes the search even harder.
Obviously not serious, but I bet that someone, somewhere, has an application that depends on file creation dates and wonders why it goes wrong every now and again. That is a *mild* potential security problem, if only because it could cause confusion. Documentation bugs can be security problems. Unexpected or unwanted behaviour from a machine is always a potential security problem.
The accumulation of seemed-like-a-good-idea-at-the-time backwards-compatible gotchas in the Windows file systems [...] all combine to introduce uncertainty and unpredictability, which leaves gaps for security errors.

Because one of our scripts took more than 15 seconds (the default time-to-live for cache entries) between deleting and recreating the file as it was scheduled during a high load time, it literally took advantage from the lack of resources.

The other script, running during far less busy periods had always been fast enough to trigger the tunnelling "feature" and get the cached creation time over and over again.  Of course the workaround for the scripts is simple: Consider the "Date Modified" attribute.

A more drastic approach would be to change the registry settings controlling this behavior. See the knowledge base article for details.

But seriously - I will never understand (and even less appreciate) - why Microsoft tends to always choose backwards compatibility over reliability. Maybe this was a sensible feature right when long filenames on FAT volumes came about, but seriously, is it necessary to carry this all the way to the 2003 server? I bet it is still present in Vista as well....

Posted by Daniel Schneller am 23:47 5 comments: Labels: bug, windows


Gnome Nautilus SSH fails when hostkey changed

Today I tried to upload some files to my server via Nautilus. Months ago I created an SSH connection to my home folder via the "Places - Connect to Server" option on the main menu. It allows you transparently use SSH via the graphical user interface.

However for some reason trying to double click the desktop connection just did not do anything at all. Selecting the entry in an open file manager windows led to a confusing error message:

Nautilus cannot display "ssh://shipdown.de".
Please select another viewer and try again.

Another connection, set up via WebDAV worked without problems. It occurred to me that this might have something to do with the recent crash of the server which made it necessary to set it up freshly. This of course included the generation of a new SSH host key. Trying to connect via the command line confirmed this:

Turns out that Nautilus fails for the same reason. Once I had edited the .ssh/known_hosts file and replaced the old key with the current one, the connection worked again.

There is a Ubuntu Bug (#41738) entry as well as a Gnome upstream report (#322501) describing this. However as this is rated as a low priority bug and has been known since Gnome 2.14 I do not expect it to be fixed very soon, so I thought it was worth noting here.

Posted by Daniel Schneller am 14:21 1 comment: Labels: gnome, linux, network, ssh, ubuntu


GMail advanced search operators

Several rules in my GMail account apply various labels to mails and mostly archive them right away. This is handy for newsletters, mailing lists and the like. However over time - if you do not read them all - there are unread mails, scattered around several labels which are too old to show on the first 50 items.

Up to now I sometimes went through the following pages, using the "Select unread" and "Mark as read" functions. Today I stumbled across a page that mentions a search term to show only unread mails. I had suspected something like this must exist, but there is no GUI feature I know of to use it.

So in addition to any other search criteria you might have (e. g. "label:newsletter") you can just add "is:unread" and only get those mails that you haven't looked at before. Other things I just tried out and found to work:

Only after that I found this page on the GMail Help Center. Sometimes life can be so easy, if you just know where to look... :-)
Posted by Daniel Schneller am 10:59 No comments: 

Building XML the Groovy way in Java

When working with XML - i. e. creating XML documents - the Java DOM API is a little cumbersome. You have to ask all sorts of factories for instances of themselves, those instances for documents, elements and so forth. It is usually a lot code to write, even if all you want is a little XML fragment with only a few elements, e. g. to be sent over the network to some server. One way to make your life easier is to resign to StringBuilder/StringBuffer and building the XML "by hand". However this is error-prone and not always easy to read.

Recently I had to implement a service that responded with XML over the net, building the document by collecting data from several sources and combining them together. The first version I wrote used the DOM API and once finished was hard to read even for me. I would have liked to use Groovy's MarkupBuilder for this, however company policy does not allow this (yet).

So I looked around for a similarly easy to read solution in plain Java. I found this entry in The Ancient Art Of Programming. It discusses the problem of the verbosity of the Java solution and compares it to the clarity of .NET's rather new Linq feature. In one of the comments Erik-Jan Blanksma posted an implementation that mimics the syntax using variable argument lists. I took the classes and built a second version of my code with it. Now the Java code looks almost like the Groovy MarkupBuilder code, except that I do not need any additional libraries. Take a look at this example:

This is really great to use. I added some tweaks to the code to make it more robust and suitable for my needs. Apart from adding some error checking (no empty element names, null values etc.) I also created an interface for classes that convert a java.util.Collection into a corresponding array of XElements. This was necessary, because the original version did not allow dynamic additions to the XML. This interface is intended to allow worker classes that somewhat mimic the Groovy syntax of just intermixing loops etc. A simple implementation could look like this:

Using it to add a list of Strings to an XML document can be done like this:

I have packed the necessary files into this ZIP archive. Feel free to use them, if they suit your needs. If you make any improvements, I would be happy to hear from you.

Posted by Daniel Schneller am 17:44 1 comment: 

MySQL Index Analyzer updated

After several months I have again spent a little work in the MySQL Index Analyzer I first published back in August of 2006.

I added a feature that will find duplicate columns inside an index, caused by the internal appending of the InnoDb primary key columns to each secondary index.

To get the code and read more about the new feature, including an example, go to the MySQL Index Analyzer Blog.

Posted by Daniel Schneller am 23:24 No comments: Labels: database, java, mia, software


Back online

Two weeks after we moved to our new appartement we got DSL back. I have to admit I was sceptical at first, because ususally I tend to not trust phone and internet providers too far when it comes to make changes to an existing setup. However apparently everything worked well. The ISDN line was switched over on the day we moved, keeping the same number without a glitch. The only thing that got lost on the way was the outgoing caller id. However after two calls to the hotline (hey, this is still well within acceptable limits, isn't it?) they assured me it was to be reactivated by tomorrow.

The DSL provider promised to get the line back up two weeks after the ISDN line had been established and they kept their promise. Today I plugged the Fritz!Box in and everything worked immediately. It is a real relief. I did not remember how slow ISDN was, even with two channels bundled to 128kbit/s. My girlfriend is relieved, too, not having to use my computer all the time, because she did not like to install the ISDN drivers.

From what I see the line should be good enough for 6MBit/s downstream, 3 at least. I will try and upgrade from the current 1MBit/s tomorrow. After all it's just 3€/month and includes a VOIP flat besides the internet flat. Let's hope this works smoothly as well :)

Posted by Daniel Schneller am 22:56 No comments: 
Thoughts on (Un)checked Exceptions

Posted by Daniel Schneller am 00:47 No comments: Labels: groovy, java, software, standards


Download Youtube Videos (and Google video)

Posted by Daniel Schneller am 23:54 6 comments: Labels: fun, software


MySQL server_errno=1236 when disk full

Posted by Daniel Schneller am 20:43 2 comments: Labels: database, mysql, windows


Visualize hard disk temperature with gnuplot

This data can now be plotted graphically using gnuplot: You need not care about the

Posted by Daniel Schneller am 16:06 2 comments: Labels: hardware, linux, ubuntu


Beware of Ubuntu Kernel 2.6.20-16!

Posted by Daniel Schneller am 01:42 3 comments: 

MySQL Optimizer Bug 28554

When we tried to clean up a rather large (4.500.000 rows, 20GB) InnoDB table some days ago, we were astonished by the time MySQL took to complete the task. We had already LIMITed the transaction size, but every single chunk still took minutes to execute. The table itself contains some number columns, including a numeric primary key, and a blob. The delete condition was mainly based on the primary key (being smaller than a predefined value) and status field. After some mails between the support crew and us an optimizer bug was identified: MySQL Bug #28554.

The problem is that in some cases the optimizer makes a bad choice concerning which index to use. It will pick a secondary index that can be used to cover a WHERE indexed_column=<constant> condition, even though it will cause way more data to be scanned than necessary. The primary key for the second condition pk_column<=<constant> would be a far better choice, visiting only a much smaller number of rows.

The bug has been verified in 4.1.12, 4.1.22 and 5.0.42-BK. Read the bug report for details and some example data.

Posted by Daniel Schneller am 00:02 No comments: 

USB scanner not working on VMwared XP

I recently changed my health-insurance company. Today I got a letter asking me to return my insurance card or notify them in case I destroyed it myself. So I thought I might just fax them an image of the destroyed card. Because Fritz!Fax works beautifully in my virtual XP box, I wanted to get my Canon LiDE Scan 70 installed in the VM (no Linux support...). I downloaded the most recent driver from Canon's website and installed it.

Unfortunately I only get blue screens as soon as I "connect" the scanner to the virtual machine using the VMware server console. Being somewhat disappointed I will now have to boot up the RealThing(TM) Windows partition again...

Posted by Daniel Schneller am 23:32 1 comment: Labels: hardware, linux, ubuntu, windows


Linux EventQueue Problem

As a part of an ongoing project I had to write a little on-screen keyboard. It is a very simple thing: Just a panel consisting of a series of JButtons that - when clicked - insert KeyEvents into the EventQueue to enable any Swing text component to be used. I do not have the original code at hand right now, but I reconstructed it at home:

Starting this will bring up a little window with two buttons "A" and "B". Clicking either of these will insert the appropriate letter into the text area. I tested this on Ubuntu Feisty Fawn with no problems at all. However on RH9 this (well, not exactly this, but very similar) code does not work reliably (that code works on Windows however, so it cannot be completely broken). On Red Hat 9 the button clicks just do not produce letters in the text area. It can be made to work, when I insert a Thread.sleep(1500) before putting the new KeyEvent on the queue. This will make the button go "down", remain there for 1,5s and then flip out again, usually (but not always) producing a letter in the text area. At the moment I have no idea what causes this strange behavior. On Thursday I will compare the code above with the original and try it out and Red Hat...

Any ideas are of course very welcome :)

Posted by Daniel Schneller am 21:20 4 comments: Labels: java, linux, windows


MySQL: Add primary key to table with duplicates

Maybe this is obvious, but I post it anyway, just to remind myself should I need it again.

Recently I had to change a table that I had not completely thought through when I first created it. The structure was so simple, I did not think I could do anything wrong with it:

There is a 1:0..* relationship between parent and child. Some sample data:

Clearly it is possible (and intended) that there can be several children with references to the same parent, even if the have equal values. Inserting data using my application was no problem. However when I tried to read back a parent including its children, I got an error from the persistence framework (proprietary 3rd party) that there was a probable inconsistency in the data, because there were several identical children. Turns out, the thing needs a primary key on every table to manage it in memory.

So how do you add a primary key to the children table when it already has data? First attempt:

Of course, adding a column fills it with either NULL or with the data types default value, in this case 0. Declaring it as a primary key then ought to fail, because all 9 rows got the default value.

So we need a some initial distinct values for the column (in the application the persistence layer will provide generated keys once there is primary key column). For this the auto increment feature comes in handy:

Because we do not need the auto incrementing anymore, we can remove it again:

Unfortunately this cannot be combined into a single ALTER TABLE statement, because the parser seems to first check if the statement's parts are valid, which is not the case for the last MODIFY COLUMN segment as the child_id column does not exist at that time.

Posted by Daniel Schneller am 12:28 18 comments: 

FindBugs - Writing custom detectors (Part 2)

This is the second part of the "Howto write a FindBugs Bug Detector" (see the first part here). To understand why one would write the kind of detector mentioned here, you should read the first part if you do not already know it.

Last time I presented a detector that is able to detect static fields of the types java.util.Calendar and java.text.DateFormat. While declaring fields like this may be suspicious, there is not necessarily something wrong with the code. The real danger comes from calling methods on suchlike fields, especially if they are not synchronized to protect them against concurrent access. So in this article we will extend and improve the existing detector to cope with this problem.

Something to chew on

This a simple class that uses a static Calendar instance. It does not really do much, but the shorter the program, the easier to understand its bytecode (this is what we need to do this time).

What I "like" most about concurrency programs (or multi-threading bugs) is that even the simplest and most harmless looking code can be a source of nasty and hard to find problems. To emphasize this I even marked the cal field final, because that's advice you hear often (e. g. Securing Java, Ch. 7, Rule 3 or Effective Java, Item 13, "Favor immutability" (Google Account required to view)) and for good reason. However in this particular case it will not help.

In order to write a detector that will alert the cal.clear(); line we need to take a look at the method's bytecode, because that is what the detector will have to look at:

The javap tool is part of the JDK installation, the -c option outputs bytecode instead of trying to recreate the Java source. Here you see the "machine code" instructions the compiler generated from the .java file. In the static intializer you can see the invocation of Calendar.getInstance() and the subsequent store (putstatic) of the result of that call to a static field which is denoted by #16. javap automatically adds a more human readable comment that allows you to identify the original variable name (cal) and type.

So a possible way to detect a call on a static Calendar is to look for a getstatic that accesses a Calendar or a subclass thereof, immediately followed by an invokevirtual opcode. For a list of all opcodes and their meanings, you can check The Java Virtual Machine Specification. This would work, at least for the simple cases, and in fact the first version of my detector worked liked this. There is a problem with this approach however (it stored the offset of the getstatic call and looked for a invokevirtual call less than 4 counters from there). Have a look at this Java/bytecode:

In this example the call to the static field is still there, however it is not as easy to detect, because a reference to it first gets stored in a local variable, then some other code follows. Only then is the Calendar accessed again (12: aload_1), this time from a register into which is was stored earlier (3: astore_1). In this situation the detector will not work.

The documentation on FindBugs is somewhat sketchy, but with some very kind help from the Findbugs-discuss mailing list, including some very helpful sample code, I managed to get it more reliable. These are the things that need to be done to generate a bug warning:

The first two were rather easy:

While the first two are rather easy, Bill Pugh provided valuable information for the third one: There is a class called the OpcodeStack which allows to identify the item a method was called on by filtering out the "superfluous" code in between 0: getstatic #16 and 13: invokevirtual #40 in the example above. In order to use it, your detector should subclass OpcodeStackDetector which in turn inherits from BytecodeScanningDetector. As this was the base class of our detector before, too, we can simply change our parent class. This enables allows for use of the OpcodeStack automatically. Have a look at the implementation for details. I am glad I did not have to figure that out myself :)

The code using it looks like this:

As for the synchronization check, David Hovemeyer provided me with the neccessary pointers and some pseudocode which resulted in the following:

I still need to try and find more information about the implementation of the so called CFG - the Control Flow Graph - to fully understand what the LockDataFlow, Location and LockSet classes do in full detail. What you can see however in the code above is, that it does not matter what the synchronisation block uses as its monitor object. This is something that might be worth some further improvements in the future, but for now it should suffice. Probably this is a case where JSR305 might come in handy.

Anyway, this is it. If all the above checks have not caused an early return from the method, we can generate a bug report:

The whole class is contained in the following listing. It especially contains some more housekeeping methods that take care of the CFG and LockDataFlow instances:

In order to use the new bug type, the findbugs.xml and messages.xml files have to be completed, too. Add the following lines to them:

When you download the current version of FindBugs either as a ZIP package or from the SVN trunk, it will contain an earlier version of this check, however disabled by default in the findbugs.xml file. I will submit the newer one as a patch to the tracker on SourceForge, so I guess it will be included in one of the later builds.

Some final words

While I am still far from really deeply understanding all the concepts and techniques in FindBugs, I found it relatively easy to get going, even though the primary source of "documentation" is the source code of existing detectors. I was also very happy to get quick, friendly and helpful responses on the mailing list. So I'd really like to encourage anyone who comes across a bug pattern that is not yet detected to give it a shot and try to implement a detector.

The future of FindBugs is open for ideas and discussions, Bill Pugh just yesterday started a FindBugs Google Group where you can take part in discussions and present any ideas you might have :)

Posted by Daniel Schneller am 00:53 2 comments: Labels: findbugs, java, software
