Performance Gains in Jython 2.5.2

Jython, the java implementation of the python language, has long been slower than it's cpython counterpart.  I've personally seen some simple benchmarks from years past such as this, where jython would actually run 3-10 times slower.  Benchmarks between the two implementations can be tricky, as I'm sure there are are some things one does better than the other depending on each developer's design decisions.  jython 2.5.2 was released this last March and has noted several performance improvements on its updated features.

I wanted to do a quick test just to compare the latest performance improvements implemented by jython's development team.  I had a simple ray tracer lying around (link) from when I was learning python at my current employer.  This is a very simple script that renders the scene at 512x512 resolution in about half a minute.  The scene has no acceleration structures to speak of and no texturing, so it's just a series of brute force calculations and pretty much zero to cache compared to a more complex scene.

Scene consisting of a procedurally-checkered sphere, a reflective sphere, and a Lambertian plane lit from above by a point light.  

Graphing

I repeatedly ran the script using each implementation and averaged each result.  I reran the cpython implementation a couple more times to make sure a jump for one run was a fluke.  Apparently it was.

• cpython 2.7.1 - 34s, 38s, 33s, 32s, 33s
• jython 2.5.1 - 37s, 37s, 36s
• jython 2.5.2 - 26s, 27s, 26s

Just a few thoughts and comments:

• jython 2.5.2 seems to definitely have some noticeable performance increases over 2.5.1
• jython may now be faster than cpython in certain situations (like the script above)
• jvm startup time was not taken into account as timing was done inside python
• for both jythons, JIT seemed to consistently kick in at around 10% of the run (I'm assuming it was JIT), where the script started to run faster

Although I use cpython for pretty much all my work, I'm pretty happy with the results and am excited to see jython make a bit a progess.  While one implementation may be faster than another in different circumstances, I think this shows jython's performance is at least similar to cpython for certain tasks.  Maybe future decisions to use one over the other will now depend on which libraries are more appropriate for a given project.  Of course, that may also change as I see importing cmodules on the jython TODO list.

update: 

In response to a handful of commenters, they asked for pypy to be included with the test.  I downloaded the latest x86 build for my system and reran the script three times--all three runs completing in eleven seconds.

Someone also mentioned if I had tried the new Java 7 build, which is the answer is yes.  I orginally downloaded java 7 to compare jython 2.5.2 against the newer version of java (which led to this benchmark).  I've heard a couple of features in particular slated for Java 7 would provide huge performance gains over Java 6 for the JVM scripting languages.  After running the two JVMs, however, I couldn't see much of a difference in performance.  Because of the newness of Java 7, I'm going to do some more investigating to make sure I had everything setup correctly to take advantage of these improvements.


Graphing