Video transitions and filters - how to create custom ones? I am wondering if there is a possibility to do custom video transitions or filters. Like the video spiraling out, little squares appearing, circling out or something like that. Wiping the video out sideways, sliding it over like a page. Stuff like that. Adding a semi transparent image as filter over the video. Applying instagram style filters. Is there a tool that supports that at all?

Is there a tool with a manifold of transitions or filters?

best would be it works from command line in linux. But anything helps.
To get filters, you can use photoshop for that. iMovie can do all of the effects that you were talking about, and iMovie has a pretty nice collection of filters too. 
Hope that helps!