3 Numbers

We can write numeric constants with an optional decimal part plus an optional decimal exponent, like these examples:

      > 4                  --> 4
      > 0.4                --> 0.4
      > 4.57e-3            --> 0.00457
      > 0.3e12             --> 300000000000.0
      > 5E+20              --> 5e+20

Numerals with a decimal point or an exponent are considered floats; otherwise, they are treated as integers.

Both integer and float values have type "number":

      > type(3)    --> number
      > type(3.5)  --> number
      > type(3.0)  --> number

They have the same type because, more often than not, they are interchangeable. Moreover, integers and floats with the same value compare as equal in Lua:

      > 1 == 1.0      --> true
      > -3 == -3.0    --> true
      > 0.2e3 == 200  --> true

In the rare occasions when we need to distinguish between floats and integers, we can use math.type:

      > math.type(3)     --> integer
      > math.type(3.0)   --> float

Moreover, Lua 5.3 shows them differently:

      > 3          --> 3
      > 3.0        --> 3.0
      > 1000       --> 1000
      > 1e3        --> 1000.0

Like many other programming languages, Lua supports hexadecimal constants, by prefixing them with 0x. Unlike many other programming languages, Lua supports also floating-point hexadecimal constants, which can have a fractional part and a binary exponent, prefixed by p or P.^[3] The following examples illustrate this format:

      > 0xff                 --> 255
      > 0x1A3                --> 419
      > 0x0.2                --> 0.125
      > 0x1p-1               --> 0.5
      > 0xa.bp2              --> 42.75

Lua can write numbers in this format using string.format with the %a option:

      > string.format("%a", 419)       --> 0x1.a3p+8
      > string.format("%a", 0.1)       --> 0x1.999999999999ap-4

Although not very friendly to humans, this format preserves the full precision of any float value, and the conversion is faster than with decimals.

Lua presents the usual set of arithmetic operators: addition, subtraction, multiplication, division, and negation (unary minus). It also supports floor division, modulo, and exponentiation.

One of the main guidelines for the introduction of integers in Lua 5.3 was that “the programmer may choose to mostly ignore the difference between integers and floats or to assume complete control over the representation of each number.”^[4] Therefore, any arithmetic operator should give the same result when working on integers and when working on reals.

The addition of two integers is always an integer. The same is true for subtraction, multiplication, and negation. For those operations, it does not matter whether the operands are integers or floats with integral values (except in case of overflows, which we will discuss in the section called “Representation Limits”); the result is the same in both cases:

      > 13 + 15               --> 28
      > 13.0 + 15.0           --> 28.0

If both operands are integers, the operation gives an integer result; otherwise, the operation results in a float. In case of mixed operands, Lua converts the integer one to a float before the operation:

      > 13.0 + 25       --> 38.0
      > -(3 * 6.0)      --> -18.0

Division does not follow that rule, because the division of two integers does not need to be an integer. (In mathematical terms, we say that the integers are not closed under division.) To avoid different results between division of integers and divisions of floats, division always operates on floats and gives float results:

      > 3.0 / 2.0      --> 1.5
      > 3 / 2          --> 1.5

For integer division, Lua 5.3 introduced a new operator, called floor division and denoted by //. As its name implies, floor division always rounds the quotient towards minus infinity, ensuring an integral result for all operands. With this definition, this operation can follow the same rule of the other arithmetic operators: if both operands are integers, the result is an integer; otherwise, the result is a float (with an integral value):

      > 3 // 2        --> 1
      > 3.0 // 2      --> 1.0
      > 6 // 2        --> 3
      > 6.0 // 2.0    --> 3.0
      > -9 // 2       --> -5
      > 1.5 // 0.5    --> 3.0

The following equation defines the modulo operator:

      a % b == a - ((a // b) * b)

Integral operands ensure integral results, so this operator also follows the rule of other arithmetic operations: if both operands are integers, the result is an integer; otherwise, the result is a float.

For integer operands, modulo has the usual meaning, with the result always having the same sign as the second argument. In particular, for any given positive constant K, the result of the expression x % K is always in the range [0,K-1], even when x is negative. For instance, i % 2 always results in 0 or 1, for any integer i.

For real operands, modulo has some unexpected uses. For instance, x - x % 0.01 is x with exactly two decimal digits, and x - x % 0.001 is x with exactly three decimal digits:

      > x = math.pi
      > x - x%0.01        --> 3.14
      > x - x%0.001       --> 3.141

As another example of the use of the modulo operator, suppose we want to check whether a vehicle turning a given angle will start to backtrack. If the angle is in degrees, we can use the following formula:

      local tolerance = 10
      function isturnback (angle)
        angle = angle % 360
        return (math.abs(angle - 180) < tolerance)
      end

This definition works even for negative angles:

      print(isturnback(-180))       --> true

If we want to work with radians instead of degrees, we simply change the constants in our function:

      local tolerance = 0.17
      function isturnback (angle)
        angle = angle % (2*math.pi)
        return (math.abs(angle - math.pi) < tolerance)
      end

The operation angle % (2 * math.pi) is all we need to normalize any angle to a value in the interval [0, 2π).

Lua also offers an exponentiation operator, denoted by a caret (^). Like division, it always operates on floats. (Integers are not closed under exponentiation; for instance, 2^-2 is not an integer.) We can write x^0.5 to compute the square root of x and x^(1/3) to compute its cubic root.

Lua provides the following relational operators:

      <   >   <=  >=  ==  ~=

All these operators always produce a Boolean value.

The == operator tests for equality; the ~= operator is the negation of equality. We can apply these operators to any two values. If the values have different types, Lua considers them not equal. Otherwise, Lua compares them according to their types.

Comparison of numbers always disregards their subtypes; it makes no difference whether the number is represented as an integer or as a float. What matters is its mathematical value. (Nevertheless, it is slightly more efficient to compare numbers with the same subtypes.)

Lua provides a standard math library with a set of mathematical functions, including trigonometric functions (sin, cos, tan, asin, etc.), logarithms, rounding functions, max and min, a function for generating pseudo-random numbers (random), plus the constants pi and huge (the largest representable number, which is the special value inf on most platforms.)

      > math.sin(math.pi / 2)        --> 1.0
      > math.max(10.4, 7, -3, 20)    --> 20
      > math.huge                    --> inf

All trigonometric functions work in radians. We can use the functions deg and rad to convert between degrees and radians.

      > math.floor(3.3)            --> 3
      > math.floor(-3.3)           --> -4
      > math.ceil(3.3)             --> 4
      > math.ceil(-3.3)            --> -3
      > math.modf(3.3)             --> 3     0.3
      > math.modf(-3.3)            --> -3    -0.3
      > math.floor(2^70)           --> 1.1805916207174e+21

      x = 2^52 + 1
      print(string.format("%d %d", x, math.floor(x + 0.5)))
          --> 4503599627370497 4503599627370498

      function round (x)
        local f = math.floor(x)
        if x == f then return f
        else return math.floor(x + 0.5)
        end
      end

      > math.floor(3.5 + 0.5)    --> 4   (ok)
      > math.floor(2.5 + 0.5)    --> 3   (wrong)

      function round (x)
        local f = math.floor(x)
        if (x == f) or (x % 2.0 == 0.5) then
          return f
        else
          return math.floor(x + 0.5)
        end
      end
      
      print(round(2.5))         --> 2
      print(round(3.5))         --> 4
      print(round(-2.5))        --> -2
      print(round(-1.5))        --> -2

Most programming languages represent numbers with some fixed number of bits. Therefore, those representations have limits, both in range and in precision.

Standard Lua uses 64-bit integers. Integers with 64 bits can represent values up to 2⁶³ - 1, roughly 10¹⁹. (Small Lua uses 32-bit integers, which can count up to two billions, approximately.) The math library defines constants with the maximum (math.maxinteger) and the minimum (math.mininteger) values for an integer.

This maximum value for a 64-bit integer is a large number: it is thousands times the total wealth on earth counted in cents of dollars and one billion times the world population. Despite this large value, overflows occur. When we compute an integer operation that would result in a value smaller than mininteger or larger than maxinteger, the result wraps around.

In mathematical terms, to wrap around means that the computed result is the only number between mininteger and maxinteger that is equal modulo 2⁶⁴ to the mathematical result. In computational terms, it means that we throw away the last carry bit. (This last carry bit would increment a hypothetical 65th bit, which represents 2⁶⁴. Thus, to ignore this bit does not change the modulo 2⁶⁴ of the value.) This behavior is consistent and predictable in all arithmetic operations with integers in Lua:

      > math.maxinteger + 1 == math.mininteger      --> true
      > math.mininteger - 1 == math.maxinteger      --> true
      > -math.mininteger == math.mininteger         --> true
      > math.mininteger // -1 == math.mininteger    --> true

The maximum representable integer is 0x7ff...fff, that is, a number with all bits set to one except the highest bit, which is the signal bit (zero means a non-negative number). When we add one to that number, it becomes 0x800...000, which is the minimum representable integer. The minimum integer has a magnitude one larger than the magnitude of the maximum integer, as we can see here:

      > math.maxinteger        -->  9223372036854775807
      > 0x7fffffffffffffff     -->  9223372036854775807
      > math.mininteger        --> -9223372036854775808
      > 0x8000000000000000     --> -9223372036854775808

For floating-point numbers, Standard Lua uses double precision. It represents each number with 64 bits, 11 of which are used for the exponent. Double-precision floating-point numbers can represent numbers with roughly 16 significant decimal digits, in a range from -10³⁰⁸ to 10³⁰⁸. (Small Lua uses single-precision floats, with 32 bits. In this case, the range is from -10³⁸ to 10³⁸, with roughly seven significant decimal digits.)

The range of double-precision floats is large enough for most practical applications, but we must always acknowledge the limited precision. The situation here is not different from what happens with pen and paper. If we use ten digits to represent a number, 1/7 becomes rounded to 0.142857142. If we compute 1/7 * 7 using ten digits, the result will be 0.999999994, which is different from 1. Moreover, numbers that have a finite representation in decimal can have an infinite representation in binary. For instance, 12.7 - 20 + 7.3 is not exactly zero even when computed with double precision, because both 12.7 and 7.3 do not have an exact finite representation in binary (see Exercise 3.5).

Because integers and floats have different limits, we can expect that arithmetic operations will give different results for integers and floats when the results reach these limits:

      > math.maxinteger + 2            --> -9223372036854775807
      > math.maxinteger + 2.0          --> 9.2233720368548e+18

In this example, both results are mathematically incorrect, but in quite different ways. The first line makes an integer addition, so the result wraps around. The second line makes a float addition, so the result is rounded to an approximate value, as we can see in the following equality:

      > math.maxinteger + 2.0 == math.maxinteger + 1.0   --> true

Each representation has its own strengths. Of course, only floats can represent fractional numbers. Floats have a much larger range, but the range where they can represent integers exactly is restricted to [-2⁵³,2⁵³]. (Those are quite large numbers nevertheless.) Up to these limits, we can mostly ignore the differences between integers and floats. Outside these limits, we should think more carefully about the representations we are using.

To force a number to be a float, we can simply add 0.0 to it. An integer always can be converted to a float:

      > -3 + 0.0                      --> -3.0
      > 0x7fffffffffffffff + 0.0      --> 9.2233720368548e+18

Any integer up to 2⁵³ (which is 9007199254740992) has an exact representation as a double-precision floating-point number. Integers with larger absolute values may lose precision when converted to a float:

      > 9007199254740991 + 0.0 == 9007199254740991     --> true
      > 9007199254740992 + 0.0 == 9007199254740992     --> true
      > 9007199254740993 + 0.0 == 9007199254740993     --> false

In the last line, the conversion rounds the integer 2⁵³+1 to the float 2⁵³, breaking the equality.

To force a number to be an integer, we can OR it with zero:^[6]

      > 2^53               --> 9.007199254741e+15      (float)
      > 2^53 | 0           --> 9007199254740992        (integer)

Lua does this kind of conversion only when the number has an exact representation as an integer, that is, it has no fractional part and it is inside the range of integers. Otherwise, Lua raises an error:

      > 3.2 | 0       -- fractional part
      stdin:1: number has no integer representation
      > 2^64 | 0      -- out of range
      stdin:1: number has no integer representation
      > math.random(1, 3.5)
      stdin:1: bad argument #2 to 'random'
                            (number has no integer representation)

To round a fractional number, we must explicitly call a rounding function.

Another way to force a number into an integer is to use math.tointeger, which returns nil when the number cannot be converted:

      > math.tointeger(-258.0)   --> -258
      > math.tointeger(2^30)     --> 1073741824
      > math.tointeger(5.01)     --> nil      (not an integral value)
      > math.tointeger(2^64)     --> nil      (out of range)

This function is particularly useful when we need to check whether the number can be converted. As an example, the following function converts a number to integer when possible, leaving it unchanged otherwise:

      function cond2int (x)
        return math.tointeger(x) or x
      end

Operator precedence in Lua follows the table below, from the higher to the lower priority:

               ^
               unary operators  (-   #   ~   not)
               *    /    //   %
               +    -
               ..               (concatentation)
               <<   >>          (bitwise shifts)
               &                (bitwise AND)
               ~                (bitwise exclusive OR)
               |                (bitwise OR)
               <    >    <=   >=   ~=   ==
               and
               or

All binary operators are left associative, except for exponentiation and concatenation, which are right associative. Therefore, the following expressions on the left are equivalent to those on the right:

      a+i < b/2+1          <-->       (a+i) < ((b/2)+1)
      5+x^2*8              <-->       5+((x^2)*8)
      a < y and y <= z     <-->       (a < y) and (y <= z)
      -x^2                 <-->       -(x^2)
      x^y^z                <-->       x^(y^z)

When in doubt, always use explicit parentheses. It is easier than looking it up in the manual and others will probably have the same doubt when reading your code.

Not by chance, the introduction of integers in Lua 5.3 created few incompatibilities with previous Lua versions. As I said, programmers can mostly ignore the difference between integers and floats. When we ignore these differences, we also can ignore the differences between Lua 5.3 and Lua 5.2, where all numbers are floats. (Regarding numbers, Lua 5.0 and Lua 5.1 are exactly like Lua 5.2.)

Of course, the main incompatibility between Lua 5.3 and Lua 5.2 is the representation limits for integers. Lua 5.2 can represent exact integers only up to 2⁵³, while in Lua 5.3 the limit is 2⁶³. When counting things, this difference is seldom an issue. However, when the number represents some generic bit pattern (e.g., three 20-bit integers packed together), the difference can be crucial.

Although Lua 5.2 did not support integers, they sneaked into the language in several ways. For instance, library functions implemented in C often get integer arguments. Lua 5.2 does not specify how it converts floats to integers in these places: the manual says only that “[the number] is truncated in some non-specified way”. This is not a hypothetical issue; Lua 5.2 indeed can convert -3.2 to -3 or -4, depending on the platform. Lua 5.3, on the other hand, defines precisely these conversions, doing them only when the number has an exact integer representation.

Lua 5.2 does not offer the function math.type, as all numbers have the same subtype. Lua 5.2 does not offer the constants math.maxinteger and math.mininteger, as it has no integers. Lua 5.2 also does not offer floor division, although it could. (After all, its modulo operator is already defined in terms of floor division.)

Surprisingly, the main source of problems related to the introduction of integers was how Lua converts numbers to strings. Lua 5.2 formats any integral value as an integer, without a decimal point. Lua 5.3 formats all floats as floats, either with a decimal point or an exponent. So, Lua 5.2 formats 3.0 as "3", while Lua 5.3 formats it as "3.0". Although Lua has never specified how it formats numbers in conversions, many programs relied on the previous behavior. We can fix this kind of problem by using an explicit format when converting numbers to strings. However, more often than not, this problem indicates a deeper flaw somewhere else, where an integer becomes a float with no good reason. (In fact, this was the main motivation for the new format rules in version 5.3. Integral values being represented as floats usually is a bad smell in a program. The new format rule exposes these smells.)