Thursday, August 20, 2009

NaN != NaN

Let's talk about dividing by zero.

As you probably know, in Math the result of dividing a (real) number by zero is undefined. What is the result of division by zero in AS 3.0?

It depends. If you divide a number other than zero by zero, you get Infinity:

trace( 10 / 0 ); //Infinity

trace( 876 / 0 ); //Infinity

If you divide zero by zero, you get NaN (which stand for "not a number").

trace( 0 / 0 ); //NaN

(By the way, to test if the value is NaN, you can't ask "if ( myVar == NaN )." Instead, you have to ask "if ( isNaN( myVar ) )." )

I learned this nuance recently while trying to fix a weird bug. I was setting the scaleX property of a Sprite. The Sprite was a loader bar, and I was setting its scaleX equal to the percent loaded:

bar.scaleX = loaded / total;

When I tested it in Firefox, the bar was totally invisible. I actually expected that, because I hadn't coded the loading process yet. I thought I was correctly calculating the total. In other words, I expected loaded / total to be equivalent to something like 0 / 87.

What I didn't know is that total was also zero. So I was actually doing this...

bar.scaleX = 0 / 0;

Which is the same as...

bar.scaleX = NaN;

But, like I said, I didn't notice a problem, because the bar was doing what I expected -- showing me that zero percent had loaded by being invisible.

I noticed the problem when I checked out my SWF in Internet Explorer. (I had Flash Player 10 installed in both IE and Firefox.) In IE, the bar was not invisible. It was big. It was so big that it filled up the entire screen!

Eventually, I figured out that I was dividing zero by zero. That still didn't tell me why I was seeing a giant bar in IE and no bar in Firefox. But when I set the numbers to, say, 10 / 1 or 6 / 0, the bar looked the same in both browsers. Which is when I learned that, in AS 3.0, zero divided by zero is NaN. And it's when I learned that the IE and Firefox plugins treat NaN differently when it's used to set the scaleX value of a Sprite.

Why would you want to scale a Sprite to NaN? Hopefully, you wouldn't. I did it by accident.

In both browsers, you get fill-the-whole-screen Sprites if you set scaleX to Infinity (which I guess makes sense: the Sprite gets infinitely wide). So my guess is that in Firefox, sprites interpret a scaleX of NaN as zero, whereas in IE they interpret it the same way they interpret Infinity.

1 comment:

  1. A friend couldn't replicate my result. Here is some info that may help: I did my tests with...

    IE 7/Win and FF 3.5.2/Mac;

    Flash Player in IE
    Flash Player in FF