Question: Is it possible to create a custom AI in a Starcraft 2 mod? If so, how?

Answer: 
As the new Starcraft editor is very mighty and supports something like procedures and functions you possibly can enhance available AIs with own stuff. For an example create some triggers like "IF player1 has >x units of this and that type, THEN AIPlayer2 order to tech for this tech"

Maybe this style would be a bit too basic, but should be easy to do for non-programmers.
Well I know for the beta there were several AI mods released, so it might be possible.