Putting programming language implementation performance into perspective.
Recently I voiced my concerns about the IronMonkey effort. It aims to allow programming language implementations like IronPython and IronRuby to run on the Tamarin VM.
My main concern is with regards to the runtime performance of such a stack. In short, running a program with an interpreter that is itself running on a bytecode virtual machine embedded in a Web browser raises many warning flags. When you stack one slow technology upon another, there’s a really good chance that what you’ll end up with with be, well, slow!
Today I noticed that The Voidspace Techie Blog has an article in response to the article I had written. The author seems to suggest that my reasoning was incorrect in nearly every way. I’m not so sure that this is the case.
YouTube and EVE Online are presented as examples of large-scale, Python-based software systems. I don’t doubt that they make very effective use of Python. It is a language that makes talented programmers very productive. And if you throw enough (powerful) hardware at it, I don’t doubt that it’s possible to run extremely popular sites with a back-end infrastructure written in Python.
But that’s not what we’re talking about with IronMonkey. Being browser-based, it would seem to me to be a technology that is targeting the client-side, rather than the server-side. Thus we often aren’t dealing with very powerful hardware, and in many cases are dealing with low-end desktop systems that are several years old. Performance problems that aren’t as obvious on modern server hardware can become readily apparent on typical desktop systems.
I don’t believe it’s incorrect to consider the existing implementations of languages like Python and Ruby to be “slow”. Although they’re not very scientific benchmarks, and of course not conclusive by any means, I think the results from The Computer Language Benchmarks Game are worth looking at. Based on the past experience I’ve had using a variety of languages in a variety of settings, the results seem to describe reality quite well.
We can look at the results (using the default weightings) for the benchmarks when run on Gentoo on a system with an Intel Pentium 4 CPU, or on Debian on a system with an AMD Sempron CPU. As would be expected, we see compiled languages near the top, with C being the fastest. But towards the last quarter of the list, we see Python. On the Pentium 4 system, those benchmarks are listed as running 17 times slower than C. For the Sempron system, they’re 21 times as slow.
The author of the response to my article is correct in stating that, “amongst the dynamic languages Python is not generally considered slow.” According to those benchmark results, Ruby is about three times slower than Python on both the Pentium 4 and Sempron systems. Based on my past experience with Ruby and Python applications of various sizes and tasks, those values seem quite reasonable.
So I think I’m quite correct in stating that Python can be considered a “slow” language, especially when compared to most of the popular compiled languages. Ruby appears to be even slower, in general. Now, that’s not to say that they’re bad languages, or should never be used. They do offer significant developer productivity gains, and are of course more than suitable for some applications.
But take such language implementations, which typically exhibit lower runtime performance, and inhibit them further by running them on a virtual machine which itself is embedded within a Web browser. It doesn’t strike me as a good situation to be in. For each unit of execution of the Python program, the overhead of the supporting levels could really start to add up. Now run this stack on a slower, low-end desktop system, and problems seen inevitable.
We’ll have to wait and see what they eventually produce. I’m not getting my hopes up too high, considering how bleak the situation currently looks. And there’s really nothing suggesting to me that such a stack has any hope of offering reasonable performance. It just seems too inherently flawed.