How good is Cython

Python optimization: Cython

The Python code is optimized. All tasks are parallelized, the CPUs all run at 99%. The database is optimized. The hardware is exhausted. Nevertheless, a factor of 30 must still be gained in computing time.

There are now only a few options:

  • Different programming language / platform / framework
  • Outsourcing of computing processes to the GPU
  • Optimizing the Python code with Cython
None of these options should be chosen lightly without seeking external expertise - because all of them cost a lot of money. An external specialist can possibly point out a fourth way.

If there is no other way left open:

  • the first option is the worst. Your own development team either has to learn something completely new or new team members have to be laboriously integrated into the team.
  • outsourcing processes to the GPU can bring about a huge increase in computing power (factor 100/1000) if the problem can be transformed accordingly. It should also be remembered that the corresponding GPU hardware / licenses have to be bought / managed. No chance without external specialists. The adjustments can then be made by yourself with trained developers.
  • Optimization with Cython (factor 2-40) usually doesn't happen just like that. External specialists can quickly establish a knowledge transfer that enables in-house developers to optimize your existing code. The Cython optimization is also hardly dependent on the problem / hardware.
Cython enables C-like subroutines to be formulated in Python code. In the Python code, variables can be given C types. Loops and other Python language elements are then executed at C speed (factor 40). Interfacing with C libraries from Cython is also easier than from Python. Cython works well with Numpy / Scipy data structures.

Course content:

After a simple introduction to how Cython works, an algorithm given in Python (QuadTree) is ported piece by piece to Python.

With this port you will learn to find bottlenecks in your Python code. Because we show you which performance problems the Python / Numpy data structures suffer from.

Three points are specifically deepened:
  • Effective use of numpy arrays from within Cython with memory views
  • "Cython extension types": Object-oriented Cython and its limits
  • "Exception handling": How to route exceptions from Cython to Python.

Course objective:

After successfully completing this course you should be able to do the following yourself with Cython:

  • Create high-performance subroutines

  • Even writing object-oriented Cython with "Extension types".

  • Have a generally better understanding of optimization.