Deep deleting everything in the layout as well as the Layout itself. (Python)

Hi Matthias,

I would like to learn the proper way of deep deleting everything inside the all layouts (pcells, static cells etc.) as well as the layouts itself. So I would like to create a condition just the same as when I first open the KLayout Application. Can you help me about how to do it programmatically?

Thank you!

Best Regards,
Ege

Comments

  • Hi Ege,

    you mean from a script?

    Did you create the layout objects yourself or do those live inside the main window?

    Closing all views should basically release all layout objects. That's halfway true for libraries, but I don't assume that's your problem.

    A problem with objects you created yourself (e.g. RBA::Layout::new) is that cleanup does not necessarily happen when you expect it to. This is in particular true for Ruby which has a garbage collection scheme. It is notoriously difficult to trigger when you need it. "GC.start" is a good way in Ruby, but not necessarily reliable in my experience.

    That's why the "big" objects have a method "_destroy" which you can call to explicitly release the object's resources. Note that also Region or Shape objects or arrays of such objects can become "big" and need to be released.

    Python in general is somewhat more reliable in terms of resource release as it uses reference counting mainly.

    But of all ways, the most efficient way it to shut down the process. In general, the object space of KLayout has some corners which are difficult to clean up. Libraries are one of these. The idea is to register libraries but not to unregister them anymore. This frees then system from the task of handling loss of library references at runtime.

    Matthias

  • With Ege We are using PCells created in Pyhton. Sometimes we change the PCell generation code and reload the python module. If KLayout has references to these PCells it crashes. To avoid crash we can use "Deep delete" in the Cell list in the GUI or turn the Cells static. This removes the references to the library and we can safely reload the python module. At the moment we do not know how to implement deep delete in python. Knowing this we can make a macro which does cleanup, reloads the library and regenerates the test cells.

    Perhaps you can advise how to achieve recursive "deep delete"? Our first attempt was to use layout_view.clear_layers(), but this likely keeps all the PCells in the cache.

  • Hi,

    I'm fairly sure that "pya.MainWindow.instance().close_all()" should basically remove all layouts and references.

    I'm a bit worried to hear you're getting crashes. Maybe it's worth debugging that. Are you able to reproduce one an tell me how?

    Kind regards,

    Matthias

  • Hi Matthias,
    This is the error log that happens when we try to use pya.MainWindow.instance().close_all() .

    Best,
    Ege

    Exception code: 0xc0000005
    Program Version: KLayout 0.25.9 (2019-06-19 r20c910b) AMD64

    Backtrace:
    0x13b2949 - (klayout_db.dll) +10569
    0x146180c - (klayout_db.dll) +727052
    0x1461c4e - (klayout_db.dll) +728142
    0x145f50a - (klayout_db.dll) +718090
    0x1460c3a - (klayout_db.dll) +724026
    0x1ac3ea7 - (klayout_db.dll) +7421607
    0x6f5cd345 - (klayout_pya.dll) +54085
    0x6d64e9f9 - (libpython3.5m.dll) +322041
    0x6d6c5531 - (libpython3.5m.dll) +808241
    0x6d6c6fc5 - (libpython3.5m.dll) +815045
    0x6d6c70f0 - (libpython3.5m.dll) +815344
    0x6d6331a3 - (libpython3.5m.dll) +209315
    0x6d60abd3 - (libpython3.5m.dll) +43987
    0x6d61f6ba - (libpython3.5m.dll) +128698
    0x6d60abd3 - (libpython3.5m.dll) +43987
    0x6d6631ef - (libpython3.5m.dll) +405999
    0x6d661687 - (libpython3.5m.dll) +398983
    0x6d60abd3 - (libpython3.5m.dll) +43987
    0x6d6c03cb - (libpython3.5m.dll) +787403
    0x6d6c6fc5 - (libpython3.5m.dll) +815045
    0x6d6c715a - (libpython3.5m.dll) +815450
    0x6f5ceb51 - (klayout_pya.dll) +60241
    0x9b6ceef - (klayout_lym.dll) +52975
    0x6a591bab - (klayout_lay.dll) +334763
    0x6a591fc2 - (klayout_lay.dll) +335810
    0x6a673b34 - (klayout_lay.dll) +1260340
    0x668e82bb - (Qt5Core.dll) +1737403
    0x2fb4af1 - (Qt5Widgets.dll) +1002225
    0x2fb5d25 - (Qt5Widgets.dll) +1006885
    0x2fb5eb5 - (Qt5Widgets.dll) +1007285
    0x307262e - (Qt5Widgets.dll) +1779246
    0x2f0e0e8 - (Qt5Widgets.dll) +319720
    0x307270c - (Qt5Widgets.dll) +1779468
    0x2eca3bc - (Qt5Widgets.dll) +41916
    0x2ed05c3 - (Qt5Widgets.dll) +67011
    0x6a54b9fb - (klayout_lay.dll) +47611
    0x668c7812 - (Qt5Core.dll) +1603602
    0x2ececa9 - (Qt5Widgets.dll) +60585
    0x2f24667 - (Qt5Widgets.dll) +411239
    0x2f26edb - (Qt5Widgets.dll) +421595
    0x2eca3bc - (Qt5Widgets.dll) +41916
    0x2ecf746 - (Qt5Widgets.dll) +63302
    0x6a54b9fb - (klayout_lay.dll) +47611
    0x668c7812 - (Qt5Core.dll) +1603602
    0x2a764fc - (Qt5Gui.dll) +156924
    0x2a77c25 - (Qt5Gui.dll) +162853
    0x2a5cdde - (Qt5Gui.dll) +52702
    0x6690effa - (Qt5Core.dll) +1896442
    0x7ffc474c63ed - (USER32.dll) +91117
    0x7ffc474c5de2 - (USER32.dll) +89570
    0x6690e683 - (Qt5Core.dll) +1894019
    0x6dd32cd5 - (qwindows.dll) +470229
    0x668c6a5d - (Qt5Core.dll) +1600093
    0x668cd827 - (Qt5Core.dll) +1628199
    0x6a54ea5e - (klayout_lay.dll) +59998
    0x6a553049 - (klayout_lay.dll) +77897
    0x401f28 - (klayout_app.exe) +7976
    0xd01702 - (klayout_rba.dll) +5890
    0x67ae6cc6 - (x64-msvcrt-ruby240.dll) +1469638
    0x67af43e0 - (x64-msvcrt-ruby240.dll) +1524704
    0x67aed16c - (x64-msvcrt-ruby240.dll) +1495404
    0x67af23f4 - (x64-msvcrt-ruby240.dll) +1516532
    0x679dff4e - (x64-msvcrt-ruby240.dll) +393038
    0x679e2b2f - (x64-msvcrt-ruby240.dll) +404271
    0xd0836f - (klayout_rba.dll) +33647
    0x401a7a - (klayout_app.exe) +6778
    0x4013f8 - (klayout_app.exe) +5112
    0x4014eb - (klayout_app.exe) +5355
    0x7ffc478a7bd4 - (KERNEL32.DLL) +97236
    0x7ffc48eeced1 - (ntdll.dll) +446161

  • Hi Ege,

    Thanks for reporting this. As with a quick check I could not reproduce it. I loaded two files and then used this Python line. No crash.

    One question: have you tried with the latest version (which is 0.26.1)?

    Thanks,

    Matthias

  • Hi,

    I'm working on the same project as jheinsoo and egedursun. After some changes to our pcell library reloading code, the crash problem disappeared. Thus there is no need to deep delete all cells, and everything works fine now.

    For completeness, here is a minimal example of the kind of code that was causing KLayout to crash previously:


    import pya class PCell1(pya.PCellDeclarationHelper): def produce_impl(self): pass def display_text_impl(self): return "PCell1" class PCell2(pya.PCellDeclarationHelper): def produce_impl(self): cell = self.layout.create_cell("PCell1", "PCell1Library", {}) self.cell.insert(pya.DCellInstArray(cell.cell_index(), pya.DTrans())) def display_text_impl(self): return "PCell2" def create_libraries(): library1 = pya.Library() library1.description = "Test library" library1.layout().register_pcell("PCell1", PCell1()) library1.register("PCell1Library") library2 = pya.Library() library2.description = "Test library" library2.layout().register_pcell("PCell2", PCell2()) library2.register("PCell2Library") def reload_libraries(): mw = pya.MainWindow.instance() #mw.close_all() # uncommenting this fixes problem in this case, but not in more complex situations library1 = pya.Library.library_by_name("PCell1Library") library1.delete() library2 = pya.Library.library_by_name("PCell2Library") library2.delete() create_libraries() mw.create_layout(1) def create_pcell(): layout = pya.CellView.active().layout() cell = layout.create_cell("PCell2", "PCell2Library", {}) cell_view.cell = cell pya.MainWindow.instance().create_layout(1) create_libraries() create_pcell() reload_libraries()

    Running this macro always crashes KLayout with the exception code 0xc0000005. It looks like the crash happens when there is a PCell from one library inside a PCell from another library, and the libraries are deleted/created. It is not completely clear why the problem was fixed by the changes we did to our real code (which is much more complex than the example).

  • edited September 2020

    @jannek Works perfectly (I mean: crashes) :)

    Thanks a lot for spending your time on providing this sample code! This helps a lot for debugging the issue (and providing a solution I hope).

    I had to apply a small modification as "cell_view" wasn't available, but that's probably because you had this variable defined somewhere else:

    def create_pcell():
      cell_view = pya.CellView.active()
      layout = cell_view.layout()
      cell = layout.create_cell("PCell2", "PCell2Library", {})
      cell_view.cell = cell
    

    I created a ticket for this issue here: https://github.com/KLayout/klayout/issues/646

    Best regards,

    Matthias

Sign In or Register to comment.