[Technical Post] Running Python as fast as C code

8/5/2021 ● 3 minutes to read

As researchers, it is crucial for the job to quickly code up an idea to see if it works or not. While there are many ways to do so, Python is considered in the last few years the go-to tool enabling just that. It allows for focusing on the idea itself and not spending time on the technicalities of software development like other programming languages require.

That said, Python is well known for being a relatively slow language in general and especially compared to compiled languages like C or C++ [1]. So, after one tested an idea by developing a Python prototype and now wants to turn it into a fast and performant module? Most of the time, it would require manually converting the code from Python into another language such as Java, C#, C++, etc.

In the middle of 2021, where so much wonderful Python code has been developed - would it be great if Python just is fast by itself? Just imigine all the time spent on implementing everything twice - we could use this time for so much more meaningful stuff.

Meet PyPy

PyPy is a fast, compliant alternative implementation of Python. PyPy is a replacement for the famous CPython compiler. It is built using the RPython language that was co-developed with it.

The reason PyPy is so fast is thanks to a mix of compiler and interpreter logic called the just-in-time (JIT) compilation [2]. What is happening is that PyPy starts like an interpreter running the Python code directly from the source files (so far, equal to Python). Nevertheless, instead of running the code line by line, it compiles parts of the code to machine code just before it executes them. In a sense, JIT compilation is a combination between interpretation and the classical ahead-of-time compilation. As a result, JIT compilation enjoys both the performance boost of ahead-of-time compilation and the flexibility of interpreted languages.

Does PyPy holds to the promise?

To illustrate how much of a speed up PyPy can provide, we perform a small-size experiment. We ran both the standard Python interpreter (Spesificly, Python3.7) and PyPy compiler-interpreter on the following code: 


import math

run_size = math.pow(10, 9)
sum([i for i in range(run_size)])

While the code is not a meaningful one (just sum all numbers between 0 and 1,000,000,000) or optimal (as we generate a large list and only then calculating its sum) it is a good candidate for our test as it well represents casual, not very optimal, numerical-related code. Although not a scientific evaluation, this example well shows the amazing performance of PyPy as the standard Python interpreter needs around 80 seconds, the PyPy run is fully executed in just 1.89 seconds. This is more than 40 times improvment! Furthermore, when comparing this result to the corresponding C++ code that finishes after 1.72 seconds, it is easy to see that PyPy gives a fight to the master of speed.

It worth mentioning that PyPy works better in the cases that a large portion of the code is in Python and not using libraries that are implemented in a non-python language.

To summary things up, if you have a python project that needs a speed boost, just download the PyPy tool from pypy.org and install it on your device. On the website, you will be able to find plenty of tips and tricks on fine-tuning your python program to further increase the performance but even without any specific modifications, PyPy will do its magic.

Refferances

Continue Reading