KLayout for Mac Yosemite 10.10.1?

2»

Comments

  • edited 7:08AM

    Hi Kazzz,

    Thank you very much for your help.

    By following your instructions I was able to build KLayout such that I could view layout correctly.

    However I still have the DRC issue - even the most basic functionality seems broken.

    Example 1:

    • Start KLayout
    • New Layout
    • New Layer (e.g. 1/0)
    • Create Rectangle
    • Start a new DRC script:

      • p=input(1)
      • p.output(2)
    • Run - error!

    The error is as follows. A dialog box pops up with the message:

    Caught the following exception:
    std::bad_alloc in Shapes::insert (Class RuntimeError)
    
    Press 'Ok' to continue and 'Cancel' to stop in the debugger
    

    Any idea where to start looking to help debug this problem?

    Simon

  • edited July 2015

    Hi Simon,

    Glad to see that my comments partially solved your problems.
    I tried your DRC "Example 1" on three different platforms below:

    (1) Windows 7 32-bit,  binary package  
    (2) Linux Mint Rebecca 64-bit, source build  
    (3) Mac OSX Yosemite 64-bit, source build (our concern)  
    

    I could reproduce your DRC problem on (3); Windows and Linux
    versions run normally. So, this is Yosemite-specific issue.

    I started "klayout" from command line under the main directory

    $ cd {somewhere}/klayout-0.23.11/build.mac-yosemite-gcc-release/main  
    $ ./klayout  -e  
    

    Then tried "Example 1" and got:

    klayout(56675,0x7fff759d5300) malloc: *** mach_vm_map(size=140729813475328) failed (error code=3)  
    *** error: can't allocate region  
    

    As per info on websites, this means "ran out of memory."
    If "size" is required memory size in byte, the figure looks too big as my Mac has
    only 16GB of main memory.

    I feel a good start point for debugging is to rebuild klayout with "-g" option and
    run with "valgrind" for memory issues.

    Kazzz

  • edited 7:08AM

    Hi,

    Thanks for your input.

    To start with I'm trying to follow the code in Ruby - specifically in "drc.lym".

    I can't seem to change klayout's behaviour with any changes to this file.

    How does this file get attached to klayout?

    Is it integrated during the build process or is it dynamically loaded each time klayout starts?

    Is it loaded from the src/builtin_in_macros or from another location?

    Simon

  • edited 7:08AM

    I figured out the above problem. It's the dodgy strm2gds and other utilities failing and crashing make before it can recompile the Ruby scripts.

    Now that is sorted I think I have found where the problem may be.

    In the "p=input(1)" line the Ruby eventually creates a new "Region" object.

    I inserted some code into the constructor of the object in dbRegion.cc to print out what's going on.

    The new code is:

    Region::Region (const RecursiveShapeIterator &si)
        : m_polygons (false), m_merged_polygons (false), m_iter (si)
    {
    printf("Region::Region 1\n");
    printf("  this=%08X\n", this);
        init ();
        m_bbox_valid = false;
        m_is_merged = false;
        printf("size()=%d\n", size());
    }
    

    The above code hangs upon the call to "size()".

    The size function looks like it iterates through a linked list to count members. I'm guessing there should only be one member (the rectangle) however the fact that the code hangs indicates that the linked list (or whatever structure is used) is not correctly terminated.

    This is where I'm finding it a little difficult to follow the code. Can anyone help me to figure out where to look to determine why this problem is occurring?

    Thanks,

    Simon

  • edited 7:08AM

    Hello Again,

    I think I've chased this rabbit about as far as I can without some advice from the original author or at least someone who understands how this works.

    The problem seems to be in RegionIterator::at_end() which is used in for loops to determine when the end of the loop has been reached.

    The code is:

    bool at_end () const
    {
        return m_from == m_to && m_rec_iter.at_end ();
    }
    

    m_rec_iter.at_end() indicates that the set of shapes or objects in the current "region" has been completed.

    I think m_from and m_to are used to cycle through various regions in the layout. However in usual circumstances these are initialized to blank entries.

    The constructor for the RegionIterator - which is what we're talking about - is as follows:

    RegionIterator::RegionIterator (const db::RecursiveShapeIterator &iter, const db::ICplxTrans &trans)
        : m_rec_iter (iter), m_iter_trans (trans), m_from (), m_to ()
    {
        set ();
    }
    

    Note that m_from & m_to are constructed with no arguments. What does this mean?

    After a lengthy chase through many files I found the relevant constructor in tlReuseVector.h. It is as follows:

    reuse_vector_const_iterator ()
        : mp_v (0), m_n (0)
    { }
    

    m_n looks like some sort of index into a vector. mp_v is a vector of something or other. I have no idea what initialising it to 0 does.

    However I'm guessing that m_from and m_to when initialised with empty constructors will test as equal on Linux and Windows but unequal on Yosemite.

    When I hack the at_end() function to just check m_rec_iter.at_end () then the DRC starts working again. However this is not a long term solution - as the m_from==m_to test must be important in some contexts (otherwise why is it there?).

    I have followed the code but I don't really understand the overall structure of it. I don't really know what m_from and m_to are for or why they test as unequal when they should be equal. Can anyone help me to figure out how to fix this?

    Thanks

    Simon

  • edited 7:08AM

    Hi,

    I've arrived at a workable solution - but I'm guessing that the authors could improve upon it.

    My solution is based upon the assumption that m_from and m_to are essentially garbage unless the appropriate RegionIterator constructor is called.

    I record this by adding a boolean flag to RegionIterator called mode_fromto which is set to true in one constructor and false in the other one.

    The at_end function only checks the m_from==m_to condition if mode_fromto is true.

    I've looked through the code and it seems that the only place m_to is set is in the constructor - so I think it's fairly safe to assume that it can be ignored if that constructor is not used.

    The new code in dbRegion.h is as follows (only changes shown here)

    class KLAYOUT_DLL RegionIterator
    {
        public:
            typedef db::Polygon value_type; 
            typedef const db::Polygon &reference;
            typedef const db::Polygon *pointer;
            typedef std::forward_iterator_tag iterator_category;
            typedef void difference_type;
    
            /**
             *  @Returns true, if the iterator is at the end
             */
            bool at_end () const
            {
                if(mode_fromto)
                    return m_from == m_to && m_rec_iter.at_end ();
                else
                    return m_rec_iter.at_end();
            }
            .
            .
            .
        private:
            friend class Region;
    
            typedef db::layer<db::Polygon, db::unstable_layer_tag> polygon_layer_type;
            typedef polygon_layer_type::iterator iterator_type;
    
            db::RecursiveShapeIterator m_rec_iter;
            db::ICplxTrans m_iter_trans;
            db::Polygon m_polygon;
            iterator_type m_from, m_to;
            bool mode_fromto;
    
            /**
             *  @brief ctor from a recursive shape iterator
             */
            RegionIterator (const db::RecursiveShapeIterator &iter, const db::ICplxTrans &trans)
                : m_rec_iter (iter), m_iter_trans (trans), m_from (), m_to ()
            {
                set ();
                mode_fromto=false;
            }
    
            /**
             *  @brief ctor from a range of polygons inside a vector
             */
            RegionIterator (iterator_type from, iterator_type to)
                : m_from (from), m_to (to)
            { 
                //  no required yet: set ();
                mode_fromto=true;
            }
    

    Simon

  • edited July 2015

    Hello Simon,

    Thank you for your intensive work to address this issue.
    I've confirmed that the suggested modification is a workable solution.
    The simple DRC "Example 1" works as expected on Yosemite, too.


    Hello Matthias,

    I've incorporated Simon's patch and prepared a new binary package for Yosemite:
    "klayout-0.23.11-MacOSX-Yosemite-2-Qt487mp.dmg.bz2"
    The file is uploaded to the Dropbox directory as before.

    Kazzz

  • edited 7:08AM

    Hi all,

    I'm deeply impressed by your analysis!

    However, I'm a bit scared. Basically the "reuse_vector_const_iterator" default constructor clearly says it will initialize the members to 0, so the content must be well initialized and m_from == m_to. Provided there is no compiler bug.

    That's what's scary. I tested the code in a huge variety of compilers: a broad selection of gcc's, clang and MS VC++. No issues seen so far. Maybe there is a bug in the compiler. And if there is, what else will it do to the code?

    I see one point: maybe -O3 is far too aggressive. Could you try less optimization (-O2 and maybe -O1) and see if the problem is gone then? I would feel much better with that option.

    Thanks and best regards,

    Matthias

  • edited July 2015

    Hello Matthias,

    Thank you for your comments.
    Above discussions are with "-O2" flag.

    I've reverted "dbRegion.h" to the original one and
    changed "-O2" to "-O1" for rebuilding KLayout.
    Unfortunately, the problem reproduced.

    Just for your info.

    Kazzz

  • edited 7:08AM

    Hi Kazz,

    thanks for the test ... if the optimization does not make a difference, then I think we can rule out a compiler bug.

    Just one more hypothesis: something writes the wrong memory and overwrites the contents of the iterators. Even more scary ...

    The problem seems easy to reproduce, so that should be possible to track down in the debugger. I'd set a breakpoint in the RegionIterator's constructor before "set()". I'm very, very sure that then "m_from==m_to". Then step over set() and watch if that still is the case. Maybe at some point we get "m_from!=m_to" which could give an hint what is going on.

    Matthias

  • edited 7:08AM

    Hi Matthias, Kazz,

    I'm trying to compile version 0.24.4 for Yosemite and I've run into a problem. Any ideas?

    My build command is:

    ./build.sh -with-qtbinding -qtbin /opt/local/libexec/qt4/bin -qtinc /opt/local/libexec/qt4/include -qtlib /opt/local/libexec/qt4/lib -rbinc /opt/local/include/ruby-1.9.1 -rblib /opt/local/lib/libruby.1.9.1.dylib 2>&1 | tee building-klayout.log

    I had to hack /usr/include/dispatch/object.h as per http://hamelot.co.uk/programming/osx-gcc-dispatch_block_t-has-not-been-declared-invalid-typedef/

    It now all compiles but fails at link with 2888 duplicate symbols.

    Example of one of the 2888:

    duplicate symbol tl::RegisteredClass<db::user_object_factory_base >::RegisteredClass(db::user_object_factory_base*, int, char const*, bool) in:
    antConfig.o
    dbLibraryProxy.o

    Any ideas what's going wrong?

    Simon

  • edited January 2016

    Hi Simon,

    I've tried the steps described below.

    My initial guess is that your "gcc" is not "clang" from Xcode but an "ordinary gcc" from Mac Ports or alikeness.

    --> Please refer to (4), (6), and (7) below.

    Hope this will help you resolve the issue your are facing.

    Kazzz

    (1) My KLayout is the latest; i.e. Version 0.24.4

    No change is made to the tar-ball source code.

    (2) I have Qt4, Ruby and Python from Mac Ports that are installed under the standard directory.

    Please note that the binary package for our community is NOT using these

    Ruby and Python but those of OSX-bundled. From Mac Ports, only Qt4 is used/linked.

    MacBookPro{sekigawa}(21)$ port installed qt4*
    The following ports are currently installed:
      qt4-mac @4.8.7_2 (active)
      qt4-mac-mariadb-plugin @4.8.7_1 (active)
    
    MacBookPro{sekigawa}(22)$ port installed ruby*
    The following ports are currently installed:
      ruby19 @1.9.3-p551_2+doc (active)
      ruby21 @2.1.8_0 (active)
      ruby_select @1.0_0 (active)
    
    MacBookPro{sekigawa}(23)$ port installed python*
    The following ports are currently installed:
      python2_select @0.0_1 (active)
      python3_select @0.0_1 (active)
      python27 @2.7.11_0+universal (active)
      python34 @3.4.4_0+universal (active)
      python_select @0.3_6 (active)
    

    (3) My Yosemite is the latest; i.e. Version 10.10.5

    MacBookPro{sekigawa}(8)$ uname -a
    Darwin MacBookPro.local 14.5.0 Darwin Kernel Version 14.5.0: Tue Sep  1 21:23:09 PDT 2015;
    root:xnu-2782.50.1~1/RELEASE_X86_64 x86_64
    

    (4) My Xlang is the latest; i.e. Version 7.2 (7C68)

    MacBookPro{sekigawa}(9)$ gcc --version
    Configured with: --prefix=/Applications/Xcode.app/Contents/Developer//usr --with-gxx-include-dir=/usr/include/c++/4.2.1
    Apple LLVM version 7.0.2 (clang-700.1.81)
    Target: x86_64-apple-darwin14.5.0
    Thread model: posix
    

    (5) I have run your build script without hacking "/usr/include/dispatch/object.h" and got...

    MacBookPro{sekigawa}(5)$ ./simon.sh
    Scanning installation ..
    
    Using Ruby interpreter: ruby1.9
        Ruby version code is 10901
        Ruby headers found: /opt/local/include/ruby-1.9.1 and /opt/local/include/ruby-1.9.1/x86_64-darwin14
        Ruby installation is in:
        - /opt/local/lib/libruby.1.9.1.dylib (lib)
        - /opt/local/include/ruby-1.9.1 (includes)
        - /opt/local/include/ruby-1.9.1/x86_64-darwin14 (config for 1.9.x)
    
    Using Python interpreter: python
    *** WARNING: Python library not in default path, trying to use MULTIARCH
        Python library found: /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib//Python.framework/    Versions/2.7/Python
        Python headers found: /opt/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7
        Python installation is in:
        - /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib//Python.framework/Versions/2.7/Python (    lib)
        - /opt/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 (includes)
    
    Qt installation directory is /opt/local/libexec/qt4/lib (lib), /opt/local/libexec/qt4/bin (tools) and /opt/    local/libexec/qt4/include (includes)
        Qt bindings enabled
    
    Platform is mac-yosemite-gcc-release
    *** WARNING: no check done of Qt-headers.
    *** ERROR: wrong -pylib path: /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib//Python.    framework/Versions/2.7/Python does not exist or not a file
    

    (6) To make the binary packages for our community, so far, I've made five build scripts for different

    combinations of compilers and libraries. Please get this from my Dropbox.

    Out of these 5, "Mac-Xcode-Ruby20Python27-BuildMyKlayout.sh" is the one used for building the binary package published.

    On the other hand, "Mac-Xcode-Ruby19Python27-BuildMyKlayout.sh" uses all, Qt4, Ruby, and Python, from

    Mac Ports. This must be closest to your development environment.

    One important point is that all these build scripts premise "LLVM (clang)" as the compiler; not "GNU gcc".

    (7) I have rerun "Mac-Xcode-Ruby19Python27-BuildMyKlayout.sh" for confirmation.

    As usual, the build process did not fully run through but aborted while making "str*" tool sets,

    which is a known issue.

    In such a case, move to "build.mac-yosemite-gcc-release" and issue "make install" command.

    Then the main application will be made normally.

    MacBookPro{sekigawa}(42)$ pwd
    ${HOME}/SVNWork/klayout-0.24.4/bin.mac-yosemite-gcc-release/klayout.app/Contents/Frameworks
    MacBookPro{sekigawa}(43)$ ll
    合計 6448
    drwxr-xr-x 15 sekigawa staff     510  1  9 09:11 .
    drwxr-xr-x  6 sekigawa staff     204  1  9 09:11 ..
    drwxr-xr-x  4 sekigawa staff     136  1  9 09:11 Python.framework
    drwxr-xr-x  4 sekigawa staff     136  1  9 09:11 QtCore.framework
    drwxr-xr-x  4 sekigawa staff     136  1  9 09:11 QtDesigner.framework
    drwxr-xr-x  4 sekigawa staff     136  1  9 09:11 QtGui.framework
    drwxr-xr-x  4 sekigawa staff     136  1  9 09:11 QtNetwork.framework
    drwxr-xr-x  4 sekigawa staff     136  1  9 09:11 QtScript.framework
    drwxr-xr-x  4 sekigawa staff     136  1  9 09:11 QtSql.framework
    drwxr-xr-x  4 sekigawa staff     136  1  9 09:11 QtXml.framework
    -rwxr-xr-x  1 sekigawa staff 3571220  1  9 09:11 libcrypto.1.0.0.dylib
    -rwxr-xr-x  1 sekigawa staff  342988  1  9 09:11 libpng16.16.dylib
    -rwxr-xr-x  1 sekigawa staff 1708676  1  9 09:11 libruby.1.9.1.dylib
    -rwxr-xr-x  1 sekigawa staff  809856  1  9 09:11 libssl.1.0.0.dylib
    -rwxr-xr-x  1 sekigawa staff  160244  1  9 09:11 libz.1.dylib
    
    MacBookPro{sekigawa}(47)$ pwd
    ${HOME}/SVNWork/klayout-0.24.4/bin.mac-yosemite-gcc-release/klayout.app/Contents/MacOS
    MacBookPro{sekigawa}(48)$ ll
    合計 162704
    drwxr-xr-x 3 sekigawa staff       102  1  9 09:11 .
    drwxr-xr-x 6 sekigawa staff       204  1  9 09:11 ..
    -rwxr-xr-x 1 sekigawa staff 166606108  1  9 09:11 klayout
    MacBookPro{sekigawa}(49)$ otool -L klayout
    klayout:
        @executable_path/../Frameworks/libruby.1.9.1.dylib (compatibility version 1.9.1, current version 1.9.1)
        @executable_path/../Frameworks/Python.framework/Versions/2.7/Python (compatibility version 2.7.0, current     version 2.7.0)
        @executable_path/../Frameworks/QtGui.framework/Versions/4/QtGui (compatibility version 4.8.0, current     version 4.8.7)
        @executable_path/../Frameworks/QtCore.framework/Versions/4/QtCore (compatibility version 4.8.0, current     version 4.8.7)
        @executable_path/../Frameworks/QtXml.framework/Versions/4/QtXml (compatibility version 4.8.0, current     version 4.8.7)
        @executable_path/../Frameworks/QtNetwork.framework/Versions/4/QtNetwork (compatibility version 4.8.0,     current version 4.8.7)
        @executable_path/../Frameworks/QtSql.framework/Versions/4/QtSql (compatibility version 4.8.0, current     version 4.8.7)
        @executable_path/../Frameworks/QtDesigner.framework/Versions/4/QtDesigner (compatibility version 4.8.0,     current version 4.8.7)
        /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0    .0, current version 1153.18.0)
        /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.0.0)
        /usr/lib/libc++abi.dylib (compatibility version 1.0.0, current version 125.0.0)
        /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.5)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)
    

    This is About KLayout built with "Mac-Xcode-Ruby19Python27-BuildMyKlayout.sh"

    cf. This is About KLayout of the binary package published

  • edited 7:08AM

    Hi Kazz,

    thanks for all your efforts supporting the community.

    I wonder whether I should upload the binaries for others to the server.

    What's your opinion?

    Thanks,

    Matthias

  • edited January 2016

    Hi Matthias,

    In the Mac (OS X) world, there are several ways to setup a programming environment.
    A good starting point is here.

    Depending on programmer's preferences, the two major package managers listed below can be used to
    setup non-OSX-bundled programming environments including Qt4, Ruby2.x and Python2.7.

    Moreover, different programmers may need different programming tool kits for their specific purposes like:
    Python version on OSX where not only KLayout but also
    other tools sharing the same programming environment must go together, I guess.

    On the other hand, as far as the C++ compiler is concerned, LLVM (clang) is the official compiler.
    Since LLVM is very actively maintained, the one distributed with "Xcode" is always bit obsolete.
    So one, like me, can build it from "HEAD of trunk" for testing purpose.

    If we see the above mentioned circumstances, the number of possible combinations to build KLayout executable
    on OS X can be many. So I'm afraid of creating unpredictable runtime issues by publishing different binary
    packages built in different programming environments.

    Instead, I would like to propose the followings:
    [1] A binary package distributed via the community site is to be arranged to work on an OS X machine
    where no programming environment (including "Xcode") is installed.
    This means that such a binary package assumes use in a pure end-user environment.
    This also implies that OSX-bundled Ruby and Python should be used.
    The latest binary package I published is following this policy.

    [2] KLayout community members who successfully built KLayout for Mac from its source are encouraged to post
    his/her script (if it is a new one) used for building the binary.

    I'll write a separate mail with suggested directory structure with initial files.

    Please give me some more time ;-)

    Regards,

    Kazzz

Sign In or Register to comment.