Using relative URL's
The page address needs to be http://1.2.3.4/a/page_first/ (with trailing slash).
ADDED:
You don't seem to understand relative URLs, so let me explain. When you reference an image like this <img src="my_image.png"/>, the image URL in the tag doesn't have any host/path info, so path is taken from the address of the HTML page that refers to the image. Since path is everything up to the last slash, in your case it is http://1.2.3.4/a/. So the full image URL that the browser will request becomes http://1.2.3.4/a/my_image.png.
You want it to be http://1.2.3.4/a/page_first/my_image.png, so the path part of the HTML page must be /a/page_first/.
Note that the browser will not assume page_first is "a directory" just because it doesn't have an "extension", and will not add the trailing slash automatically. When you access a server publishing static dirs and files and specify a directory name for the path and omit the trailing slash (e. g. http://www.example.com/some/path/here), the server is able to determine that you actually request a directory, and it adds the slash (and usually also a default/index file name) for you. It's not generally the case with dynamic web sites where URLs are programmed.
So basically you need to explicitly include the trailing slash in your page path: dispatcher.connect('page','/a/:number_of_page/', controller=self, action='page_method') and always refer to it with the trailing slash (http://1.2.3.4/a/page_first/), otherwise the route will not be matched.
As a side note, usually you put the images and other static files into a dedicated dir and serve them either with CherryPy's static dir tool, or, if it's a high load site, with a dedicated server.
I do a simple web application written in Python using cherrypy and Mako. So, my question is also simple. 
I have one page with URL http://1.2.3.4/a/page_first. Also there is an image that available on URL http://1.2.3.4/a/page_first/my_image.png. And I want to locate my_image.png on the page_first. 
I added a tag <img src="my_image.png"/>, but it is not shown. I looked at web developer tools->Network and saw that request URL for image was http://1.2.3.4/a/my_image.png, instead of http://1.2.3.4/a/page_first/my_image.png.
Why does it happen?
Thanks.