>I'm surprised by Firefox 4's performance. The Mozilla web pages claim to be matching or surpassing Chrome's speed but my measurements show them to be a lot slower.

Having recently done some benchmarking of my own code, and from simply running the browsers side-by-side a fair bit, this doesn't surprise me a bit. Firefox 4 consistently comes out 3x slower or worse than Chrome/Safari (I don't have Opera handy). I keep seeing claims, I keep seeing benchmarks, but I've never been able to replicate their results with any code I've written. Significantly faster than FF3, don't get me wrong, but they've still got a fair distance to go if they want to claim faster-than-WebKit.

I quite like this benchmark setup. Real-world results, split by display time. Nice writeup!

I’m not denying your results, but Webkit has nothing to do with the JS execution speed: Chrome and Safari use completely different JS engines and it doesn’t make sense to talk about ‘Webkit’ in this context.

Good point. I'll clarify: Safari, WebKit nightlies, and Chrome perform very similarly in many cases, and tend to be faster than anyone else.

He meant that, as far as the tests in the main article, Webkit has nothing to do with the results. Chrome uses the V8 javascript engine, Safari uses the Nitro javascript engine, while the webkit project are building their own javascript engine - Squirellfish.

Nitro is what Squirrelfish was renamed in Apple's marketing materials after they shipped it in Safari.

I know. And they all perform very similarly under most conditions (now. Not in the past.). And they're all near the top of all the charts all the time.

For what it's worth, this test has little to do with JS execution speed. See https://bugzilla.mozilla.org/show_bug.cgi?id=622358#c3 for an analysis of a profile of the test in Firefox. In brief, 80% is canvas operations due to the emulator making 1024 putImageData calls for every single frame. The time spent actually running JS is about 10% of the total time.

I updated the page with a graph from FF4.0b9 which is a bit improved over b8. The Javascript is now respectively similar.

You're right about the 80/10 split. FF4's rendering speed is an outlier compared to Safari/Chrome/Opera on the same hardware. To better compare the faster browsers I'd need to run multiple emulators per page.

EDIT: b9 improved over 8, not itself.

> Having recently done some benchmarking of my own code, and from simply running the browsers side-by-side a fair bit, this doesn't surprise me a bit. Firefox 4 consistently comes out 3x slower or worse than Chrome/Safari

It really depends on the benchmark. On the official ones FF4 does very well, on yours apparently it doesn't, ditto the one in the article.

There is little meaning to general statements at this point, when all the major JS engines are quite fast. It really depends on the benchmark. It would be easy, for example, to show a benchmark where FF4 is faster than Chrome, but again, it would prove nothing.

Which is why I like speed tests of real-world code. One isn't even remotely conclusive, no, but I'd argue that the results from many form a far more accurate measure than specific benchmarking code.

Benchmarks, especially the big ones, are targets for improving JS engines, while cases they don't cover may not get as much attention. The "official" ones should always be taken with a gigantic grain of salt. Especially as I believe I read that Mozilla's in-the-works engine is experimenting with making optimizing transformations of JS code to improve its speed, where none of the others do because it breaks from the spec, or some such argument.

I don't know where you heard that Mozilla is considering breaking the spec to improve JavaScript performance, but I can practically assure you that there are no plans to do that from what I've read.

I had the same experience with past versions of Firefox, but Firefox 4 is finally equivalent to Chrome on my machine, and in graphics-heavy cases, it's faster with both 2D (XRender) and 3D (WebGL) (in many cases, anyway).

What operating system are you on and what build of Firefox 4 are you running? I am using the nightlies on Arch Linux x64 with kernel 2.6.36.

OSX, following the betas. WebGL is a very different, very new environment, though. Its speed depends on a lot aside from the JS engine.

edit:

Most of my testing involves jQuery DOM manipulation, and benchmarking some DOM-free, stratght JS utility code / mini-libraries I've written, and one moderately-large one in the works. Because that's what I'm writing. Real benchmarks trump more scientifical benchmarks, for my purposes. YMMV, absolutely, these are just my outcomes.

For a specific case, changing a callback-based array iteration to a for loop, totalled after 10,000 empty operations (making it do something and accounting for that code showed almost no difference):

  Chrome 9: <1ms difference
            5ms average after change
  On FF4b7: 15ms difference
            15ms average after change (note that this means it was ~30ms before, a full 6x slower than Chrome)
  On FF3:   40ms difference
            51ms average after change

I'd be interested in two things (well, quite a few, but these two especially): any change in FF4b8, and a link to your code or (better) a bug report on the performance issue; we'll get to it after FF4 if it's not already better.

I like this because it clearly shows just how horribly bad web technologies really are. I used to say the web turned your computer into an IBM 3070 with a 386 under the hood.

Now I'll say the web turns your computer into an IBM 3070 with a TRS-80 under the hood.

That's also because it's an emulator. A TRS-80 surely couldn't emulate itself at full speed.

Or into a NES? http://benfirshman.com/projects/jsnes/

Slightly more impressive but yes, it's strange we all get so excited about performance improvements that can only be called less mediocre than the previous iteration.

I grew up on Galaxy Invasion and SubLogic's Flight Simulator. (Too bad they don't have that one available on the emulator page.)

I'm interested to see what changes with the 4.2 OS soon to be released.

I'd like to see that, too. I wrote Peter an email to ask if he'd gotten a chance to do that.

I'll update after I get my iPad upgraded. My guess is there will be little difference.

I didn't mention in the article, but I have run the emulator on the Wii's Opera browser and the PS3--both slow but the Wii is much faster. The XBOX360 hasn't heard about the WWW yet.

SEA DRAGON!!! WOOOO!!!!