It looks like you're new here. If you want to get involved, click one of these buttons!
Hello,
I have some problems when I have decimal third points value in the variable.
param(:nSource, TypeDouble, "nSource", :default => 3.75, :unit => "µm", :hidden => true)
when I declare the variable like this, nSource does not have 3.75 but 3.7500000000000000....2 like this.
Therefore, I always have to round function for getting 3.75.
However, as I calculate many values in the code, I cannot use round function for all of variable changes.
I mean, if I have to do, I should, but is there any way that I do not get that kinda infinit approximated number?
it happends when i pass the parameter from user input.
When I input the value 1.25, then the code has 1.25000000000000000002.
what is the problem?
And, if I can, I want to make the variable as a container which only can contain maximum 3 decimal point number.
for example, if the variable has 1.1, then 1.1, when it has 1.6467, then 1.647 like this.
Comments
Basically that's a common thing with standard floating-point arithmetics.
Read this first: https://floating-point-gui.de/
For avoiding this, use sprintf-like formatting of numbers before printing them, e.g.
s = "%.12g" % n
.Matthias