PyPy is an implementation of the Python programming language the language implementation itself. Eventually, dynamic optimization techniques - implemented as another translation aspect - should become robust against language changes. more...
Download one of the following release files::
python pypy/bin/py.py
This will give than the original CPython. Many extension modules are not enabled by default; to use them you need to pass --withmod-NAME arguments (for example, --withmod-_rawffi is required to import our version of ctypes).
This is probably not something you want to play with for too long, though, as it is really slow. Since the 0.7.0 release it is possible to.
If you want to play with the stable development PyPy version you can check it out from the repository using subversion. Download and install subversion if you don't already/doc..
For in-depth information about architecture and coding documentation head over to the documentation section where you'll find lots of interesting information. Additionally, in true hacker spirit, you may just start reading sources .
If you want to see if PyPy works on your machine/platform you can simply run PyPy's large test suite with:
cd pypy python test_all.py directory-or-files
test_all.py is just another name for py.test which is the testing tool that we are using and enhancing for PyPy. Note that running all the tests takes a very long time, and enormous amounts of memory if you are trying to run them all in the same process; test_all.py is only suitable to run a subset of them at a time. To run them all we have an autotest driver that executes the tests directory by directory and produces pages like the following one:
You may file bug reports on our issue tracker which is also accessible through the 'issues' top menu of the PyPy website. using the development tracker has more detailed information on specific features of the tracker.).
To start interpreting Python with PyPy, use Python 2.4 or greater:
cd pypy python bin)
The parameter is the number of loops to run through the test. The default is 50000, which is far too many to run in a non-translated PyPy version (i.e. when PyPy's interpreter itself is being interpreted by CPython)..4.1/test/pystone.py 10
See our configuration sections for details about what all the commandline options do.
There are quite) |- hash(W_StringObject('a')) -> W_IntObject(-468864544) |- int_w(W_IntObject(-468864544)) -> -468864544 |-10 LOAD_CONST 2 (<W_NoneObject()>) |-13 RETURN_VALUE |- <<<< leave <inline>a = 1 + 2 @ 1 >>>>
One of the original features provided by PyPy is the "thunk" object space, providing lazily-computed objects in a fully transparent manner:
cd pypy python bin/py.py -o thunk >>>> from __pypy__ import 200 lines of code. Since the 0.8.0 release you can translate PyPy with the thunk object space.
If you know more about pypy-exclusive features, choose one from the following, go to objspace proxies document.
The PyPy project uses test-driven-development. Right now, there are a couple of different categories of tests which you can run. To run all the unit tests:
cd pypy python test_all.py
(this is not recommended, since it takes hours and uses huge amounts of RAM). Alternatively, you may run subtests by going to the correct subdirectory and running them individually:
python test_all.py interpreter/test/test_pyframe (this will take hours and hours and hours):
cd lib-python/2.4.1/test python ../../../pypy/test_all.py
or if you have installed py.test then you simply say:
py.test -E
from the lib-python/2.4.1/test directory. Running one of the above commands tells you how to proceed.
The demo/ directory contains examples of various aspects of PyPy, ranging from running regular Python programs (that we used as compliance goals) over experimental distribution mechanisms to examples translating sufficiently static programs into low level code.
The translator is a tool based on the PyPy interpreter which can translate sufficiently static Python programs into low-level code. To be able to use it you need to (if you want to look at the flowgraphs, which you obviously should):
- Download and install Pygame.
- Download and install Dot Graphviz.
or:
- Have an internet connection. The flowgraph viewer'.
We have a type annotator that can completely infer types for functions like is_perfect_number (as well as for much larger examples):
>>> t.annotate([int]) >>> t.view()
Move the mouse over variable names (in red) to see their inferred types.
The LLVM or low level virtual machine project has, among other things, defined a statically typed portable assembly language and a set of tools that optimize and compile this assembly for a variety of platforms. As such, this assembly is a natural target for PyPy's translator.
To translate to LLVM assembly you must first have LLVM version 1.9 installed - the how to install LLVM page provides some helpful hints.
The LLVM backend is not as flexible as the C backend, and for example only supports one garbage collection strategy. Calling compiled LLVM code from CPython is more restrictive than the C backend - the return type and the arguments of the entry function must be ints, floats or bools - as the emphasis of the LLVM backend is to compile standalone executables.
Here is a simple example to try:
>>> t = Translation(snippet.my_gcd) >>> a = t.annotate([int, int]) >>> t.rtype() >>> f = t.compile_llvm() >>> f(15, 10) 5
The JavaScript backend is still experimental but was heavily improved during 2006 Google summer of code. It contains support for the document object model and a good integration with PyPy's unittesting framework. Code can be tested with the Spidermonkey commandline JavaScript interpreter in addition to a multitude of JavaScript capable browsers. The emphasis of the JavaScript backend is to compile RPython code into JavaScript snippets that can be used in a range of browsers. The goal is to make it more and more capable to produce full featured web applications. Please see the pypy/translator/js/test directory for example unittests.
Here is a simple example to try:
>>> t = Translation(snippet.my_gcd) >>> a = t.annotate([int, int]) >>> source = t.source_js()
If you want to know more about the JavaScript backend please refer to the JavaScript docs.
Use the CLI backend to translate the flow graphs into .NET executables: gencli is quite mature now and can also compile the whole interpreter. You can try out the CLI backend from the interactive translator shell:
>>> def myfunc(a, b): return a+b ... >>> t = Translation(myfunc) >>> t.annotate([int, int]) >>> f = t.compile_cli() >>> f(4, 5) 9
The object returned by compile_cli is a wrapper around the real executable: the parameters are passed as command line arguments, and the returned value is read from the standard output.
Once you have compiled the snippet, you can also try to launch the executable directly from the shell; you can find the executable in one of the /tmp/usession-* directories:
$ mono /tmp/usession-<username>/main.exe 4 5 9
To translate and run for CLI you must have the SDK installed: Windows users need the .NET Framework SDK 2.0, while Linux and Mac users can use Mono.
Using the interactive shell with the JVM is basically the same as any other backend. For example, you might enter:
>>> def myfunc(a, b): ... return a+b >>> t = Translation(myfunc) >>> t.annotate([int, int]) >>> f = t.compile_jvm() >>> f(4, 5) 9
As with the CLI, the object returned by compile_jvm is a wrapper around the real executable. When invoked, it actually starts a JVM process, passing the parameters as command line arguments and the result as a string over stdout. Therefore, the interface works best with simple types like integers and strings.
If you prefer to run the compiled code directly, you will find it in one of the /tmp/usession-* directories. You can run it like so:
$java -cp /tmp/usession-<username>/pypy/ pypy.Main 4 5 9
Note that the main entrypoint is always pypy.Main.
To successfully use the JVM you will need to have both a JDK installed (at least version 5), and the Jasmin assembler. Furthermore, you need a script on your path called jasmin which runs the Jasmin jar file, something like the following:
$ cat which jasmin #!/bin/bash java -jar $PATH_TO_JASMIN_JAR "$@".
(Note: for some hints on how to translate the Python interpreter under Windows, see the windows document)
Not for the faint of heart nor the owner of a very old machine: you can translate the PyPy's whole of Python interpreter to low level C code. This is the largest and ultimate example of source that our translation toolchain can process:
cd pypy/translator/goal python translate.py --run targetpypystandalone.py
By default the translation process will try to use the Boehm-Demers-Weiser garbage collector for the translated interpreter. Use --gc=generation to use our own exact generational implementation which is now faster and doesn't have external dependencies. Otherwise, be sure to install Boehm before starting the translation (e.g. by running apt-get install libgc-dev on Debian or Ubuntu).:
[translation:info] created: ./pypy-c [translation:info] Running compiled c source... debug: entry point starting debug: argv -> ./pypy-c debug: importing code debug: calling code.interact() Python 2.4.1 (pypy 0.7.1 build 18929) on linux2 Type "help", "copyright", "credits" or "license" for more information. (InteractiveConsole) >>>> 1 + 1 2 >>>>
With the default options, you can find the produced executable under the name pypy-c. Type pypy-c --help to see the options it supports -- mainly the same basic options as CPython. In addition, pypy-c --info prints the translation options that where used to produce this particular executable. This executable contains a lot of things that are hard-coded for your particular system, so it's not really meant to be installed or redistributed at the moment.
If you exit the interpreter you get a pygame window with all the flow graphs plus a pdb prompt. Moving around in the resulting flow graph.py script itself takes a number of options controlling what to translate and how. See translate.py -h. Some of the more interesting options are:
- --text: don't show the flow graph after the translation is done. This is useful if you don't have pygame installed.
- --stackless: this produces a pypy-c that includes features inspired by Stackless Python.
- --gc=boehm|ref|marknsweep|semispace|generation: choose between using the Boehm-Demers-Weiser garbage collector, our reference counting implementation or three of own collector implementations (as we have seen Boehm's collector is the default).
Find a more detailed description of the various options in our configuration sections.
You can also use the translate.py script to try out several smaller programs, e.g. a slightly changed version of Pystone:
cd pypy/translator/goal python translate.py targetrpystonedalone
This will produce the executable "targetrpystonedalone-c".
It is also possible to experimentally translate a PyPy version using the "thunk" object space:
cd pypy/translator/goal python translate.py targetpypystandalone.py --objspace=thunk
the examples in lazily computed objects should work in the translated result.
To create a standalone executable using the experimental LLVM compiler infrastructure:
./translate.py --text --batch --backend=llvm targetpypystandalone.py
To create a standalone .NET executable using the CLI backend:
./translate.py --text --batch --backend=cli targetpypystandalone.py
The executable and all its dependecies debug: entry point starting debug: argv -> debug: importing code debug: calling code.interact() Python 2.4.1 (pypy 0.9.0 build 38134) on linux2 Type "help", "copyright", "credits" or "license" for more information. (InteractiveConsole) >>>> 1 + 1 2 >>>>
Unfortunately the interactive interpreter will not work with Mono versions older than 1.1.17.2 due to a Mono bug. Everything else will work fine, but not the interactive shell. version.
You can also try the still very experimental clr module that enables integration with the surrounding .NET environment.
You can dynamically load .NET classes using the clr.load_cli_class method. After a class has been loaded, you can instantiate and use it as it were a normal Python class. Special methods such as indexers and properties are supported using the usual Python syntax:
>>>> import clr >>>> ArrayList = clr.load_cli_class('System.Collections', 'ArrayList') >>>> obj = ArrayList() >>>> obj.Add(1) 0 >>>> obj.Add(2) 1 >>>> obj.Add("foo") 2 >>>> print obj[0], obj[1], obj[2] 1 2 foo >>>> print obj.Count 3
At the moment the only way to load a .NET class is to explicitly use clr.load_cli_class; in the future they will be automatically loaded when accessing .NET namespaces as they were Python modules, as IronPython does.:
We use some optional tools for developing PyPy. They are not required to run the basic tests or to get an interactive PyPy prompt but they help to understand and debug PyPy especially for the ongoing translation work.
graphviz and pygame are both necessary if you want to look at generated flow graphs:
graphviz:
pygame:
ctypes (version 0.9.9.6 or later) is required if you want to run low-level tests. See the download page of ctypes.
The py library is used for supporting PyPy development and running our tests against code and documentation as well as compliance tests. You don't need to install the py library because it ships with PyPy and pypy/test_all.py is an alias for py.test but if you want to have the py.test tool generally in your path, you might like to visit:
PyPy employs an open development process. You are invited to join our pypy-dev mailing list or look at the other contact possibilities. We are also doing coding Sprints which are separately announced and often happen around Python conferences such as EuroPython or Pycon. Take a look at the list of upcoming events to plan where to meet with us.