This happens for integer arithmetic as well, as soon as you step off the happy path of trivial computations and onto the things we use floating-point for. You cannot exactly solve most differential equations, even if your underlying arithmetic is exact. These errors (“local truncation error”) then carry through subsequent steps of the solution, and may be magnified by the stability characteristics of the problem.
