Q:

What is the difference between 恥ずかしい and 気恥ずかしい?

the goo definition  of 気恥ずかしい: 

なんとなく恥ずかしい。きまりが悪い。

I don't really understand the difference.

A:

「気」 is a prefix in 「気恥{きは}ずかしい」.　It is read 「き」 or 「け」, depending on the word it is  attached to.
This prefix adds the meaning of "somewhat", "kind of", "to an extent", etc. to the verb or adjective that it attaches to.  Note that these are what 「なんとなく」 means at least roughly.
Thus, 「気恥ずかしい」 means "somewhat embarrassed", "a little bit ashamed", etc. whereas the plain 「恥ずかしい」 means "embarrassed", "ashamed", etc.
Common words using this prefix include:
「気{け}だるい」 = "somewhat dull", "kind of listless", etc.
「気忙{きぜわ}しい」 = "somewhat fidgety"
デジタル大辞泉 explains it this way.

動詞・形容詞に付いて、なんとなく、漠然としたなどの意を表す。「気おされる」「気だるい」

