Q:

Why were the speed records set by seaplanes in the early thirties?

For nearly all of the 1930'ies, the fastest aeroplanes were all of the seaplane category (as opposed to landplanes, to use the terminology of FAI).
Presumably, the designers of the two categories had access to more or less the same technology (engines, materials, aerodynamics, etc.) given the large interest in the subject and funding available.
If we for a second rely on Wikipedia for the record listing, we see that the speed record for a seaplane set in 1931 is 655.8 km/h (Supermarine S.6B), while the record for a landplane set in 1935 (4 years later) is "only" 567.1 km/h (Hughes H-1 Racer). 
Seeing how fast aviation technology developed in that period, it is remarkable that four years after the Supermarine record, the fastest landplane was still significantly slower than the fastest seaplane.
I would expect the large pontoons on the seaplanes to provide more drag compared to a simple landing gear, and hence by logic the seaplane should be penalised and achieve lower maximum speeds. All things being equal. 
However, since the speed differences are so pronounced, all things obviously are not equal. So what are the differences between the two categories (or their designers and manufacturers) that can explain the gap in maximum speed?

A:

The answer: Fixed-pitch propellers.
Until the 1930s propellers were fixed pitch only. From Wikipedia:

The first practical controllable-pitch propeller for aircraft was
  introduced in 1932. French firm Ratier pioneered variable-pitch
  propellers of various designs from 1928 onwards, relying on a special
  ball bearing helicoïdal ramp at the root of the blades for easy
  operation.

This means all speed records were flown with fixed-pitch propellers, which needed to be optimised to the top design speed to coax as much thrust from the engines as possible. In turn, this means that most of the propeller blade was stalled at low speed, so take-off thrust was low. To achieve high speed, a high wing loading was needed and, in combination with low thrust, the take-off length of such an aircraft was beyond of the length of airfields back then.
On water, however, the maximum field length is close to infinity. Yes, drag with floats is especially high when transitioning from buoyancy to dynamic lift (the "hump"), but that can be overcome when that hump occurs at high enough speed so the propeller delivers enough thrust. 

A:

This may only be part of the story, but the Schneider Trophy was the most prestigious air trophy of that era, and it was initially reserved for seaplanes only. In I Could Never Be So Lucky Again, the famous aviator Jimmy Doolittle (writing about events in 1925) said:

The Schneider Cup race, to be held at Baltimore two weeks after the
  Pulitzer [air race], was reserved for seaplanes only. The race had first been run
  in 1913 and was sponsored by Jacques P. Schneider, pilot son of a
  wealthy French industrialist. It was considered the most important
  international air race at the time and received much press coverage in
  the flying nations of the world.

So at that time, the way to bring the most attention to your aircraft - or your country's aviation capabilities - was to enter the most famous air race, which happened to be restricted to seaplanes only:

The word came down from Washington that we could attempt to set or
  break records and make cross-country flights that were considered
  newsworthy and would project a favorable image of the Army Air
  Service.

I'm sure there were other factors involved, but publicity and prestige seem to be two big ones.

A:

No source on this, but my gut feeling is that it's a funding issue. The Air Force wasn't an independent branch and the navy was quicker to invest in aviation and didn't share with the army, and definitely had a larger budget then private citizens. The army definitely delayed emphasizing their Air Force (based on Rindlesbacher/rickenbacker's experiences). So again, no source other than books like The Aviators, but I would say funding in the Navy was higher and aircraft carriers weren't yet perceived by the navy as a whole as the dominant force they became in the early years of world war 2 (hence the pontoons).

