pyspark bind issue cannot generate the SPARK UI
The warning "port 4040 is unaviable, attempting 4041" and forawrd..., is occures probably, because you open a a lot of SparkContext without close them.(SparkContext#close).
You can change the port if you want by the parameter spark.ui.port in configuration.
For your first issue, can you provide logs please of the running spark app? maybe you are running a fast program that close the web ui fast enough before you enter?
I am facing an issue with pyspark while running this in local mode. And the tricky thing is when I open cmd and run pyspark it creates a session and then if I open the localhost:4040 I can see the UI. But, if I run some program from pycharm I am not able to access the UI (localhost:4040) also the URL is not generated. 
And in other case when I have the pyspark shell open in parallel with pycharm program and if I run the same program in pycharm I will get a warning saying cannot bind port 4040 so attempting 4041. which is a bit understandable as shell has occupied port 4040 so it is attempting 4041. So, ideally 4041 should be allocated but that is not the case. As the documentation says it attempts 16 times, to get the next free port number available, I have not been allocated with any. So, then if I see the services running currently on my PC, I could see none of them are using those ports.
Please help me with this.