Wrong rounding coordinates

edited December 2017 in Layout
Hi,

I use KLayout 0.25-x64 on Windows-8, and I several times got a problem when changing the coordinates of a cell.
If I set the coordinates as 5 / 0.4, then I got the coordinates set at 4.999 / 0.399 which makes my wires off grid and miss the connections with the shapes drawn at X=5.0

The solution I found is to enter the coordinates with the full precision as : 5.000 x 0.400 and it works!

Best regards,
Laurent

Comments

  • edited November -1

    Hi Laurent,

    I tried to reproduce that issue, but I cannot confirm it. 0.4 stays 0.4 in my case. I'm using Windows 7, but I doubt there is a difference implied by the OS. It's weired that enter 5.000 makes a difference - computation wise there is no difference between 5.000 and integer 5.

    I'd like to ask you to check a few things:

    • What function are you using? Is it "instance properties"?
    • Please check the database unit of your file? Please check the value under File/Layout Properties. What is your DBU?
    • Is there any instance with magnification involved?

    If you can reproduce the issue, a sample layout would be helpful.

    Thanks,

    Matthias

  • edited November -1
    Hi Matthias,

    Yes, I use "instance properties", and my dbu = 0.001 u/µm.
    It happens whatever the magnification, most of the time it is simply 1.
    I will check if I can prepare a small test case.

    Thanks for answer, BRgds,
    Laurent
  • edited December 2017
    Hi,


    I'm not a programmer specialist in any way (never heard of Ruby before I started to use Klayout ;-) but I had similar problems (being 1 dbu off) when trying to make some code based on the scripts I found on the forum. When converting an input value s (in um) to number of dbu's the following expression is commonly used:

    s_dbu = (s / layout.dbu)

    I replaced it with:

    s_dbu = (s / layout.dbu).round

    to get rid of this wrong rounding coordinates.

    Hope this helps,

    Cheers,

    Tomas



    Matthias, thanks for providing such a great software tool and all the support, very much appreciated!
  • edited December 2017

    Hi Thomas,

    Floating point arithmetics is always subject to rounding issues. You can try that in plain Ruby:

    $ irb
    irb> dbu = 0.1*0.1*0.1
    => 0.0010000000000000002
    

    instead of the expected 0.001.

    Because Ruby's "to_i" integer cast is always rounding down, this leads to rounding issues when using this database unit for conversion:

    irb> (1.0/dbu).to_i
    => 999
    

    "round" will fix this issue by forcing "round to nearest".

    KLayout uses "round to nearest" whenever possible, so this issue should not happen. I could not reproduce
    the issue with a standard setup, but it's possible to force the database unit to something like

    0.00100000000001
    

    it's still displayed as "0.001" because of the limited number of digits in the display field, but the rounding issues can be reproduced. 5 becomes 4.999 because

    I suspect that Laurent's case is similar to this. You can check your precise DBU by using this command in the macro IDE:

    RBA::CellView::active.layout.dbu
    

    I have opened #32 on GitHub to track this issue.

    Matthias

  • edited November -1

    Hi Laurent,

    Using the odd DBU is was able to find the issue. Please see https://github.com/klayoutmatthias/klayout/issues/32.

    Thanks for reporting that.

    Kind regards,

    Matthias

  • edited November -1
    Matthias,
    That's exactly my problem, thank you for your time spent on it.
    Regards,
    Laurent
  • edited November -1
    Hi Matthias,

    In case you would like to reproduce:

    I've got a pcell library where I define a simple rectangle:

    # -------------------------------------------------------------------------------

    # The PCell declaration for the rectangle
    class Rectangle < PCellDeclarationHelper

    include RBA

    def initialize

    # Important: initialize the super class
    super

    # declare the parameters
    param(:l, TypeLayer, "Layer", :default => LayerInfo::new(1, 0))
    param(:w, TypeDouble, "Width", :default => 1.0)
    param(:h, TypeDouble, "Height", :default => 1.0)

    end

    def display_text_impl
    # Provide a descriptive text for the cell
    "Rectangle(L=#{l.to_s},W=#{'%.3f' % w.to_f},H=#{'%.3f' % h.to_f})"
    end

    def produce_impl

    # This is the main part of the implementation: create the layout

    # fetch the parameters
    w_dbu = (w / layout.dbu).round
    h_dbu = (h / layout.dbu).round

    # create the rectangle
    p1=Point.new(-w_dbu / 2,-h_dbu / 2)
    p2=Point.new(-w_dbu / 2,h_dbu / 2)
    p3=Point.new(w_dbu / 2,h_dbu / 2)
    p4=Point.new(w_dbu / 2,-h_dbu / 2)

    pts=[p1,p2,p3,p4]

    # create the shape
    cell.shapes(l_layer).insert(Polygon.new(pts))

    end

    end

    # -------------------------------------------------------------------------------


    I've also put

    layout.dbu = 0.0001

    at the end of the pcell library as it is needed for my designs.


    When I, for instance, specify 0.15 for the height, I end up with 0.1498 when I don't use the ".round".


    Cheers,

    Tomas
  • edited November -1

    Hi Tomas,

    thanks for the code - but this problem is inherent to Ruby (and most other languages). So whenever float to integer conversion is involved, you should use "round". I cannot do much against this - except recommending to directly use "D" types as available in 0.25:

    # create the rectangle
    p1=DPoint.new(-w / 2,-h / 2)
    p2=DPoint.new(-w / 2,h / 2)
    p3=DPoint.new(w / 2,h / 2)
    p4=DPoint.new(w / 2,-h / 2)
    
    pts=[p1,p2,p3,p4]
    
    # create the shape
    cell.shapes(l_layer).insert(DPolygon.new(pts))
    

    Matthias

  • edited December 2017
    Unfortunatly for that layout, I don't work with script, but only edit the instance to move it by entering new calculated coordinates.

    BRgds,
    Laurent
  • edited November -1

    Yes ... I just wanted to clarify that these are two different cases. The one from the properties dialog is solved (see GitGub master branch here) and will be release with the next minor release.

    Regards,

    Matthias

Sign In or Register to comment.