Integer overflow

(Redirected from Arithmetic overflow)

Integer overflow can be demonstrated through an odometer overflowing, a mechanical version of the phenomenon. All digits are set to the maximum 9 and the next increment of the white digit causes a cascade of carry-over additions setting all digits to 0, but there is no higher digit to change to a 1, so the counter resets to zero. This is wrapping in contrast to saturating.

In computer programming, an integer overflow occurs when an arithmetic operation attempts to create a numeric value that is outside of the range that can be represented with a given number of digits – either larger than the maximum or lower than the minimum representable value.

The most common result of an overflow is that the least significant representable digits of the result are stored; the result is said to wrap around the maximum (i.e. modulo a power of the radix, usually two in modern computers, but sometimes ten or another radix).

An overflow condition may give results leading to unintended behavior. In particular, if the possibility has not been anticipated, overflow can compromise a program's reliability and security.

For some applications, such as timers and clocks, wrapping on overflow can be desirable. The C11 standard states that for unsigned integers modulo wrapping is the defined behavior and the term overflow never applies: "a computation involving unsigned operands can never overflow."[1]

On some processors like graphics processing units (GPUs) and digital signal processors (DSPs) which support saturation arithmetic, overflowed results would be "clamped", i.e. set to the minimum or the maximum value in the representable range, rather than wrapped around.

Origin

The register width of a processor determines the range of values that can be represented in its registers. Though the vast majority of computers can perform multiple-precision arithmetic on operands in memory, allowing numbers to be arbitrarily long and overflow to be avoided, the register width limits the sizes of numbers that can be operated on (e.g. added or subtracted) using a single instruction per operation. Typical binary register widths for unsigned integers include:

• 8 bits: maximum representable value 28 − 1 = 255
• 16 bits: maximum representable value 216 − 1 = 65,535
• 32 bits: maximum representable value 232 − 1 = 4,294,967,295 (the most common width for personal computers as of 2005),
• 64 bits: maximum representable value 264 − 1 = 18,446,744,073,709,551,615 (the most common width for personal computer CPUs, as of 2017),
• 128 bits: maximum representable value 2128 − 1 = 340,282,366,920,938,463,463,374,607,431,768,211,455

When an arithmetic operation produces a result larger than the maximum above for an N-bit integer, an overflow reduces the result to modulo N-th power of 2, retaining only the least significant bits of the result and effectively causing a wrap around.

In particular, multiplying or adding two integers may result in a value that is unexpectedly small, and subtracting from a small integer may cause a wrap to a large positive value (for example, 8-bit integer addition 255 + 2 results in 1, which is 257 mod 28, and similarly subtraction 0 − 1 results in 255, a two's complement representation of −1).

Such wraparound may cause security problems—if an overflowed value is used as the number of bytes to allocate for a buffer, the buffer will be allocated unexpectedly small, potentially leading to a buffer overflow which, depending on the usage of the buffer, might in turn cause arbitrary code execution.

If the variable has a signed integer type, a program may make the assumption that a variable always contains a positive value. An integer overflow can cause the value to wrap and become negative, which violates the program's assumption and may lead to unexpected behavior (for example, 8-bit integer addition of 127 + 1 results in −128, a two's complement of 128). (A solution for this particular problem is to use unsigned integer types for values that a program expects and assumes will never be negative.)

Flags

Most computers have two dedicated processor flags to check for overflow conditions.

The carry flag is set when the result of an addition or subtraction, considering the operands and result as unsigned numbers, does not fit in the given number of bits. This indicates an overflow with a carry or borrow from the most significant bit. An immediately following add with carry or subtract with borrow operation would use the contents of this flag to modify a register or a memory location that contains the higher part of a multi-word value.

The overflow flag is set when the result of an operation on signed numbers does not have the sign that one would predict from the signs of the operands, e.g. a negative result when adding two positive numbers. This indicates that an overflow has occurred and the signed result represented in two's complement form would not fit in the given number of bits.

Definition variations and ambiguity

For an unsigned type, when the ideal result of an operation is outside the types representable range and the returned result is obtained by wrapping, then this event is commonly defined as an overflow. In contrast, the C11 standard defines that this event is not an overflow and states "a computation involving unsigned operands can never overflow."[1]

When the ideal result of an integer operation is outside the types representable range and the returned result is obtained by clamping, then this event is commonly defined as a saturation. Usage varies as to whether a saturation is or is not an overflow. To eliminate ambiguity, the terms wrapping overflow[2] and saturating overflow[3] can be used.

The term underflow is most commonly used for floating-point math and not for integer math[4]. But, many references can be found to integer underflow.[5][6][7][8][9] When the term integer underflow is used, it means the ideal result was closer to minus infinity than the output type's representable value closest to minus infinity. When the term integer underflow is used, the definition of overflow may include all types of overflows or it may only include cases where the ideal result was closer to positive infinity than the output type's representable value closest to positive infinity.

When the ideal result of an operation is not an exact integer, the meaning of overflow can be ambiguous in edge cases. Consider the case where the ideal result has value 127.25 and the output type's maximum representable value is 127. If overflow is defined as the ideal value being outside the representable range of the output type, then this case would be classified as an overflow. For operations that have well defined rounding behavior, overflow classification may need to be postponed until after rounding is applied. The C11 standard [1] defines that conversions from floating point to integer must round toward zero. If C is used to convert the floating point value 127.25 to integer, then rounding should be applied first to give an ideal integer output of 127. Since the rounded integer is in the outputs range, the C standard would not classify this conversion as an overflow.

Methods to mitigate integer overflow problems

Integer overflow handling in various programming languages
Language Unsigned integer Signed integer
Ada modulo the type's modulus raise Constraint_Error
C/C++ modulo power of two undefined behavior
C# modulo power of 2 in unchecked context; `System.OverflowException` is raised in checked context[10]
Java N/A modulo power of two
JavaScript all numbers are double-precision floating-point
MATLAB Builtin integers saturate. Fixed-point integers configurable to wrap or saturate
Python 2 N/A convert to long type (bigint)
Seed7 N/A raise OVERFLOW_ERROR[11]
Scheme N/A convert to bigNum
Simulink configurable to wrap or saturate
Smalltalk N/A convert to LargeInteger
Swift Causes error unless using special overflow operators.[12]

There are several methods of handling overflow:

1. Avoidance: By allocating variables with data types that are large enough to contain all values that may possibly be computed and stored in them, it is always possible to avoid overflow. Even when the available space or the fixed data types provided by a programming language or environment are too limited to allow for variables to be defensively allocated with generous sizes, by carefully ordering operations and checking operands in advance, it is often possible to ensure a priori that the result will never be larger than can be stored. Static analysis tools, formal verification and design by contract techniques can be used to more confidently and robustly ensure that an overflow cannot accidentally result.
2. Handling: If it is anticipated that overflow may occur, then tests can be inserted into the program to detect when it happens and do other processing to mitigate it. For example, if an important result computed from user input overflows, the program can stop, reject the input, and perhaps prompt the user for different input, rather than the program proceeding with the invalid overflowed input and probably malfunctioning as a consequence. For another example, it is possible to add two numbers each two bytes wide using just a byte addition in steps: first add the low bytes then add the high bytes, but if it is necessary to carry out of the low bytes this is arithmetic overflow of the byte addition and it becomes necessary to detect and increment the sum of the high bytes. CPUs generally have a way of detecting this to support addition of numbers larger than their register size, typically using a status bit; the technique is called multiple-precision arithmetic.
3. Propagation: if a value is too large to be stored it can be assigned a special value indicating that overflow has occurred and then have all successive operation return this flag value. (Such values are sometimes called NaNs--Not A Number[s]). This is useful so that the problem can be checked for once at the end of a long calculation rather than after each step. This is often supported in Floating Point Hardware called FPUs.

Programming languages implement various mitigation methods against an accidental overflow: Ada, Seed7 (and certain variants of functional languages), trigger an exception condition on overflow, while Python (since 2.4) seamlessly converts internal representation of the number to match its growth, eventually representing it as `long` – whose ability is only limited by the available memory.[13]

Run-time overflow detection implementation `AddressSanitizer` is also available for C compilers.

In languages with native support for Arbitrary-precision arithmetic and type safety (such as Python or Common Lisp), numbers are promoted to a larger size automatically when overflows occur, or exceptions thrown (conditions signaled) when a range constraint exists. Using such languages may thus be helpful to mitigate this issue. However, in some such languages, situations are still possible where an integer overflow can occur. An example is explicit optimization of a code path which is considered a bottleneck by the profiler. In the case of Common Lisp, this is possible by using an explicit declaration to type-annotate a variable to a machine-size word (fixnum)[14] and lower the type safety level to zero[15] for a particular code block.[16][17][18][19]

In Java 8, there are overloaded methods, for example like `Math.addExact(int, int)`, which will throw `ArithmeticException` in case of overflow.

Computer emergency response team (CERT) developed the As-if Infinitely Ranged (AIR) integer model, a largely automated mechanism to eliminate integer overflow and truncation in C/C++ using run-time error handling.[20]

In computer graphics or signal processing, it is typical to work on data that ranges from 0 to 1 or from −1 to 1. An example of this is a grayscale image where 0 represents black, 1 represents white, and values in-between represent varying shades of gray. One operation that one may want to support is brightening the image by multiplying every pixel by a constant. Saturated arithmetic allows one to just blindly multiply every pixel by that constant without worrying about overflow by just sticking to a reasonable outcome that all these pixels larger than 1 (i.e. "brighter than white") just become white and all values "darker than black" just become black.

Examples

Unanticipated arithmetic overflow is a fairly common cause of program errors. Such overflow bugs may be hard to discover and diagnose because they may manifest themselves only for very large input data sets, which are less likely to be used in validation tests.

Taking the arithmetic mean of two numbers by adding them and dividing by two, as done in many search algorithms, causes error if the sum (although not the resulting mean) is too large to be represented and hence overflows.[21]

An unhandled arithmetic overflow in the engine steering software was the primary cause of the crash of the 1996 maiden flight of the Ariane 5 rocket.[22] The software had been considered bug-free since it had been used in many previous flights, but those used smaller rockets which generated lower acceleration than Ariane 5. Frustratingly, the part of the software in which the overflow error occurred was not even required to be running for the Ariane 5 at the time that it caused the rocket to fail—it was a launch-regime process for a smaller predecessor of the Ariane 5 that had remained in the software when it was adapted for the new rocket. Furthermore, the actual cause of the failure was a flaw in the engineering specification of how the software dealt with the overflow when it was detected: it did a diagnostic dump to its bus, which would have been connected to test equipment during software testing during development but was connected to the rocket steering motors during flight; the data dump drove the engine nozzle hard to one side which put the rocket out of aerodynamic control and precipitated its rapid breakup in the air.[23]

On 30 April 2015, the Federal Aviation Authority announced it will order Boeing 787 operators to reset its electrical system periodically, to avoid an integer overflow which could lead to loss of electrical power and ram air turbine deployment, and Boeing deployed a software update in the fourth quarter.[24] The European Aviation Safety Agency followed on 4 May 2015.[25] The error happens after 2³¹ centiseconds (248.55134814815 days), indicating a 32-bit signed integer.

Overflow bugs are evident in some computer games. In the arcade game Donkey Kong, it is impossible to advance past level 22 due to an integer overflow in its time/bonus. The game takes the level number a user is on, multiplies it by 10 and adds 40. When they reach level 22, the time/bonus number is 260, which is too large for its 8-bit 256 value register, so it resets itself to 0 and gives the remaining 4 as the time/bonus – too short to finish the level. In Donkey Kong Jr. Math, when trying to calculate a number over 10000, it shows only the first 4 digits. Overflow is the cause of the famous "split-screen" level in Pac-Man[26] and "Nuclear Gandhi" in Civilization.[27] It also caused the "Far Lands" in Minecraft which existed from the Infdev development period to Beta 1.7.3; however, it was later fixed in Beta 1.8 but still exists in the Pocket Edition and Windows 10 Edition versions of Minecraft.[28]

An integer signedness bug in the stack setup code emitted by the Pascal compiler prevented Microsoft / IBM MACRO Assembler Version 1.00 (MASM), a DOS program from 1981, and many other programs compiled with the same compiler, to run under some configurations with more than 512 KB of memory.

Microsoft / IBM MACRO Assembler (MASM) Version 1.00, and likely all other programs built by the same Pascal compiler, had an integer overflow and signedness error in the stack setup code, which prevented them from running on newer DOS machines or emulators under some common configurations with more than 512 KB of memory. The program either hangs or displays an error message and exits to DOS.[29]

In 2014, the music video for PSY's Gangnam Style received so many views on YouTube that it breached the maximum possible viewership number within a signed 32-bit integer. YouTube subsequently amended the maximum count to a 64-bit integer.

In August 2016, a Casino machine at Resorts World Casino printed a prize ticket of \$42,949,672.76 as a result of an overflow bug. The Casino refused to pay this amount calling it a malfunction, using in their defense that the machine clearly stated that the maximum payout was \$10,000, so any prize higher than that had to be the result of a programming bug. The Iowa Supreme Court ruled in favor of the Casino.[30]

References

1. ^ a b c ISO. "ISO/IEC 9899:2011 Information technology - Programming languages - C". webstore.ansi.org.
2. ^ "Wrap on overflow - MATLAB & Simulink". www.mathworks.com.
3. ^ "Saturate on overflow - MATLAB & Simulink". www.mathworks.com.
4. ^ Arithmetic underflow
5. ^ "CWE - CWE-191: Integer Underflow (Wrap or Wraparound) (3.1)". cwe.mitre.org.
6. ^
7. ^ Mir, Tabish (4 April 2017). "Integer Overflow/Underflow and Floating Point Imprecision". medium.com.
8. ^
9. ^ "Avoiding Buffer Overflows and Underflows". developer.apple.com.
10. ^ BillWagner. "Checked and Unchecked (C# Reference)". msdn.microsoft.com.
11. ^ Seed7 manual, section 15.2.3 OVERFLOW_ERROR.
12. ^ The Swift Programming Language. Swift 2.1 Edition. October 21, 2015.
13. ^ Python documentation, section 5.1 Arithmetic conversions.
14. ^ "Declaration TYPE". Common Lisp HyperSpec.
15. ^ "Declaration OPTIMIZE". Common Lisp HyperSpec.
16. ^ Reddy, Abhishek (2008-08-22). "Features of Common Lisp".
17. ^ Pierce, Benjamin C. (2002). Types and Programming Languages. MIT Press. ISBN 0-262-16209-1.
18. ^ Wright, Andrew K.; Matthias Felleisen (1994). "A Syntactic Approach to Type Soundness". Information and Computation. 115 (1): 38–94. doi:10.1006/inco.1994.1093.
19. ^ Macrakis, Stavros (April 1982). "Safety and power" (requires subscription). ACM SIGSOFT Software Engineering Notes. 7 (2): 25–26. doi:10.1145/1005937.1005941.
20. ^ As-if Infinitely Ranged Integer Model
21. ^ "Extra, Extra - Read All About It: Nearly All Binary Searches and Mergesorts are Broken". googleresearch.blogspot.co.uk.
22. ^ Gleick, James (1 December 1996). "A Bug and A Crash". The New York Times. Retrieved 17 January 2019.
23. ^ Official report of Ariane 5 launch failure incident.
24. ^ Mouawad, Jad (30 April 2015). "F.A.A. Orders Fix for Possible Power Loss in Boeing 787". New York Times.
25. ^ "US-2015-09-07 : Electrical Power – Deactivation". Airworthiness Directives. European Aviation Safety Agency. 4 May 2015.
26. ^ Pittman, Jamey. "The Pac-Man Dossier".
27. ^ Plunkett, Luke (2016-03-02). "Why Gandhi Is Such An Asshole In Civilization". Kotaku. Retrieved 2018-07-30.
28. ^ Minecraft Gamepedia. "Minecraft Gamepedia Page".
29. ^ Lenclud, Christophe. "Debugging IBM MACRO Assembler Version 1.00".
30. ^ Kravets, David (June 15, 2017). "Sorry ma'am you didn't win \$43M – there was a slot machine 'malfunction'". Ars Technica.