How can I reduce the size of iCal files? I've exported one of my Google Calendars – 336kb 
I then imported it into iCal and exported it without modification – 1.7mb 

Is there a problem that would the file size increase so much?

Is there any way to trim the fat? 

-- Mac OS 10.7.2
Not sure how you are measuring the size of the iCal information but when you exported the events from Google Calendars you got the bare event information in a transport form (.ics)

When this gets imported into iCal the data formats for that application must be created together with any linking and additional data. I would suspect if you were to be privy to the internal storage format of the Google Calendar it would also take up more than 336KB.  

It's a fact of life with software that the data storage formats will take more space than the bare contained data as the software needs to add more than just your data to provide functionality. 

As an example create yourself a TextEdit document and type a single space save then look at the size of the file.

If you save as plain text it's 1 byte, save as RTF and it's 328 bytes, and save as Word(docx)it's 3528 bytes.