Red Hat Bugzilla – Bug 51792
Optimized byte-code should be provided also
Last modified: 2008-05-01 11:38:00 EDT
4Suite-0.11-2 contains only normal .pyc byte code but node the optimized
.pyo one:
| $ rpm -ql 4Suite | grep pyo
| $
This leads to significant slowdowns when used in an optimized project.
E.g.:
-----------
$ cat test.py
import xml.xslt.Processor
import xml.dom;
dt = xml.dom.DocumentType.DocumentType('foo', None, None, None, None)
dc = xml.dom.Document.Document(dt)
dc.createElement('bar')
proc = xml.xslt.Processor.Processor()
$ python test.py && time python test.py ## (1)
real 0m4.891s
user 0m0.970s
sys 0m0.110s
$ python -O test.py && time python -O test.py ## (2)
real 0m4.002s
user 0m3.930s
sys 0m0.070s
$ su -
# python -O test.py && time python -O test.py ## (3)
real 0m1.027s
user 0m0.910s
sys 0m0.110s
$ exit
$ python -O test.py && time python -O test.py ## (4)
real 0m1.027s
user 0m0.960s
sys 0m0.060s
-------------------------
[(1) & (2) were executed as normal user which can not generate files in the
4Suite directories. The first call of python in the examples was used to
generate the byte-code; then the plain execution-time can be measured at
the second one.]
You can see that (2) needs the most time; probably python generates the
optimized byte-code before it tests if it can be written. Because user can
not write into 4Suite-directories, this is a wasted effort.
When root executes (3) the .pyo files can be generated an a subsequent
repeating of (2) (#4) gives better times.
I suggest to generate optimized .pyo files and include them into the
package. This can be reached e.g. by calling
| python -O /usr/lib/python1.5/compileall.py \
| $RPM_BUILD_ROOT/usr/lib/python1.5/site-packages
in the %install stage.
Later, perhaps. Not now.
What shall I do between now and "later"?
Have I to omit the '-O' option manually or will it patched out in the
python-package?
How shall I handle rpm-packages providing .pyo files? When using them, the same
slowdown would happen. Or, will the .pyo-support completely removed from python?
BTW: it's not an enhancement; it's a bug impacting performance significantly.
They are built with their standard, included build procedures. Adding .pyo files
would require changing the standard buildprocedure, rebuild and retesting, which
we're avoiding as much as possible at this stage. Doing it for a little
performance gain isn't happening now - for the next release, it sounds like a
good suggestion. Not for this one.
BTW: The best way to solve this is to make the standard buildprocedure (from
distutils) create this file. I suggest you contact them on making it a feature
there.
Updating to python-2.2 has shown a new issue:
When root runs a python-program the compiled/optimized files will be created
automatically. Therefore, when updating python-1.5 to python-2.2 there will be
kept a lot of ghost files in the python-1.5/site-lisp directory.
E.g. when 4Suite was used by root, the file
| /usr/lib/python1.5/site-packages/xml/xpath/XPath.pyo
can be created. When updating to a python2.2 version of 4Suite, these files will
move into
| /usr/lib/python2.2/site-packages
XPath.pyo will not be deleted because it is not in the 4Suite-package manifest.
Therefore, the old python1.5 directory will contain a lot of orphaned .pyo files
and their directories.
Added in 4Suite 0.11-4