Folding basket - is there such a thing? I'd like to be able to go to a grocery (or some other) store on my commute from work. I've got a rear rack (b-tween).
I know, that I can permanently mount a basket on it, but I'd prefer not to since it makes noise, can be too wide and so on (and looks uncool for me, yes).

So, are there any folding baskets or some alternatives to use?

edit
The accepted answer is not about baskets, that's true, but it really about a thing I needed.

To clarify: I want something that I can fold and put inside my backpack (not a big one) and unfold when needed.
A subset of panniers that are designed for shopping bags are often called "grocery panniers," the distinguishing features being an open top and dimensions similar to the base of a paper grocery bag (not that you can get those very much anymore). 

They are typically easily removable (just hooks), and in my experience do not bounce around alot, as was suggested in another answer. There is an elastic cord to the lower hooks that keeps them secure. 

If you want to leave them on your bike, they do fold compactly against the rack and usually have velcro or snap closures to hold them folded. I usually just remove them when I'm not using them. 

Here is an example: