I would like to start the conversation in blocking mode and later switch to non-blocking.
Is that a stupid idea?
The python docs are kind of ambiguous about it, there it says:

... You do this [setblocking(0)] after creating the socket, but before using it. (Actually, if you’re nuts, you can switch back and forth.)

I read this as 'please don't do that', so I was wondering if there are reasons as to why it is discouraged.
Is there some kind of undefined behavior, what problems could I run into?
Yes you can do this, but mostly people do multiple blocking sockets with threads or multiple non-blocking sockets with event loops. But it should be no problem in switching in-between, as long as you don't switch between buffered and unbuffered I/O.
switching between blocking and nonblocking on a python socket