Questions tagged [precision]
Issues related to the representation of numerical quantities in a finite representation in a given base differing from their exact mathematical value.
95 questions
4
votes
1
answer
128
views
How do large-scale simulations maintain numerical stability when chaotic systems amplify rounding errors?
Chaotic systems like weather models are extremely sensitive to initial conditions, and computers can only store finite precision. What algorithms or statistical techniques are used to keep results ...
0
votes
0
answers
105
views
High precision minimizer
I need a minimizer that works well with high precision. I am currently doing all calculations with mp math to handle the high precision, and am then using a minimizer in scipy where the objective ...
2
votes
1
answer
189
views
Looking for a numerical algorithm or library to calculate derivatives of degenerate eigenvalues
There is a well known result to easily calculate the derivative of non degenerate eigenvalues :
$$ v_p^T \frac{dA}{dx} v_p = \frac{de_p}{dx} \space\space \space(1)$$
But when they are degenerate, how ...
- 151
8
votes
2
answers
203
views
Computing $\frac{x - y}{x - z}$ when $x,y,z$ are close to each other
What is the most stable way to compute
$$\frac{x - y}{x - z}$$
when $x$, $y$, and $z$ are all close to each other? I would like to compute expressions of this form in low precision on a GPU, but when ...
- 3,235
7
votes
1
answer
307
views
Unexpected result when summing sorted (and unsorted) positive floating point numbers
I am exploring Higham's excellent Accuracy and Stability of Numerical Algorithms and chapter 4 is dedicated to summation.
So I decided to test the most basic thing. Summing positive random numbers ...
- 171
5
votes
0
answers
210
views
Single precision vs double precision conjugate gradients
I tested my conjugate gradients implementation with float and double precision and contrary to my guess the double code was twice faster than the single precision code. The reason is that I need many ...
- 3,021
2
votes
0
answers
128
views
Runge Kutta 4th order: unexpected result
My problem in brief: in some situations, the Runge Kutta 4th order method (RK4) doesn't seem to give 4th order improvement when using a smaller time step. I wonder how this worse-than-expected result ...
- 121
1
vote
1
answer
1k
views
Float equality tolerance for single and half precision
Suppose the metric is
abs(a-b) <= rtol * max(abs(a), abs(b))
i.e. math.isclose with ...
2
votes
1
answer
225
views
Numerically stable way to implement Cramer's rule analog
Problem statement
Let $A$ be an $n\times n$ matrix and $b$ an $n$-dimensional vector. For $j\in \{1, \dots, n \}$, let $A_j$ be the matrix where we take $A$ and replace the $j^{\rm th}$ column with $b$...
- 31
3
votes
1
answer
107
views
Dynamic tolerance in a conditional loop to obtain maximum precision allowed by machine floating point numbers
I have coded a simple program for a root finding problem using Halley's method. Here is the code:
...
4
votes
1
answer
154
views
Summation of trigonometric functions results in error with finite precision
Consider the following expression:
$$f(t) = B+\sum_{k=1}^{N} A_k\cos(\omega_kt)$$
where $A$ and $B$ are known. the frequencies are also known but are not multiples of a fundamental frequency. However, ...
0
votes
1
answer
169
views
Robust unit test for reciprocal approximation
Let $x$ and $y$ be representable floating point numbers. I'm looking for a unit test which can ensure that my user's compiler has not made the reciprocal approximation $\mathrm{fl}(x/y) \approx \...
- 2,255
1
vote
1
answer
108
views
What are the Exact Rules for Significant Figures, Precision, and Uncertainty?
In the physical sciences (which are physics, chemistry, astronomy, materials science, etc.), we learned that the uncertainty is +/- the smallest unit (which is 1) of the last significant figure if the ...
2
votes
1
answer
251
views
Accuracy loss in single-precision Euclidean norm computation
I do hydrodynamics simulations with Fortran and recently I met with this issue:
I have a single-precision array b of length ...
- 123
2
votes
1
answer
507
views
High precision numerical integration of discrete data with Matlab
I have discrete data of a function plotted below:
The "Y" values of the function near "X=1.57" are very close to each other and zero, like 9.25558265263186E-11 and 5....
- 23