Davidv Posted February 25, 2020 Report Share Posted February 25, 2020 Hi, just putting together some math channel stuff where the numbers might get quite big. I am guessing that the largest number you can go to would be a signed 32 bit integer? 2,147,483,647 maximum? Or are they a 16 bit signed int? Or a float? something else. Also, what happens when you overflow this number? Will it stay at the max value, or bounce back to zero or a negative value? Thanks Quote Link to comment Share on other sites More sharing options...
TechDave Posted February 25, 2020 Report Share Posted February 25, 2020 It's a 32 bit single precision float. so there are a few answers to the question.https://en.wikipedia.org/wiki/Single-precision_floating-point_format Davidv 1 Quote Link to comment Share on other sites More sharing options...
Davidv Posted February 25, 2020 Author Report Share Posted February 25, 2020 Thank you Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.