Using the KLayout functionality from within a different GUI application

Hi Matthias,

I need to provide the "overlaps to void" functionality for the GUI image processing application which generates polygons internally and needs to output a legitimate GDS file. I have converted the ruby script you provided on this forum to python and it works fine, albeit within KLayout GIU app. I am unfortunately stuck with Windows. I read on this forum that running KLayout headless on Windows is problematic (cygwin is not an option for me), otherwise I would have run it as a separate process to do the polygons->GDS conversion I need (with communication via files). Could you suggest an approach that does not include picking KLayout apart to extract the needed functionality? I understand, that your new python library project would be an ideal solution, but this is apparently in the not-so-near future.
Thanks in advance for any hints.
BTW, I am programming since times immemorial (early 1970s), and I must say that KLayout is among the very best of the serious SW projects I have come across in my lifetime. Big thanks for that!

Comments

  • Thanks a lot for this encouraging feedback :-)

    Actually the Python library is not far from being realized. There already are RPM's and DEB packages available for Linux for trying out here: http://www.klayout.org/downloads/pymod/. Please also refer to the Wiki page for details https://github.com/klayoutmatthias/klayout/wiki/klayout---Standalone-KLayout-Python-Module.

    For Windows, I am preparing an installer that includes a Python package. However a specific problem on Windows is the binary ABI compatibility and I have not explored ways to provide Python packages for any Python installation (sepcifically those built with Visual Studio). The Python module I will provide with this installation is based on MSYS2 Python 3 (taken from ArchLinux). If that's enough, you'll be set up soon.

    Kind regards,

    Matthias

  • edited June 2018

    Hi Matthias,

    Thanks for the prompt and detailed reply. This gives me reasons for optimism! I am currently using Anaconda, but I hope I shall be able to get away with MSYS2 (which I never tried before).

    On a different subject: When I use your "overlaps to void" script, all works fine, but only on the top level - that is the outermost hull gets holes, but if holes contained polygons (containing holes, containing polygons,...etc.) - they of course disappear, since there is no recursion into containment. I guess I can sort this out on my own after some careful reading of the docs, but if you have a hint/shortcut, it would be much appreciated.

    Thanks again and good luck with Python module - I am sure lots of people like myself are keen to give it a try.

    Cheers,

    Mikhail

    PS

    I thought I'd formulate my problem better:

    1. Thousands of polygons as input coordinates (counter-clockwise-polygons, clockwise-holes,
      which is, probably, not essential)
    2. I read them and add to the layer, then save as GDS.

    That's it, basically.

    When I tried to do this manually in GUI, I set 'Select background combination mode' to 'Diff',
    and then started drawing polygons and polygons within polygons and ...
    This worked fine - correct holes with correct connectors, dark and clear regions where they shold be.

    It seems that I do not have to sort the input in the order of containment
    (e.g. outermost to innermost) - holes and connectors come out correctly automagically.

    Now I am thinking how to turn this into a macro.

  • Hi Mikhail,

    maybe you can use the EdgeProcessor for your job. It has a fancy method called "simple_merge_e2p" https://www.klayout.de/doc-qt4/code/class_EdgeProcessor.html#method43.

    This method takes edges (point-to-point connections) and assumes that "right side" of the edge is "inside". So a polygon with counterclockwise orientation is a hull and a polygon with clockwise orientation is a hole. You just need to create edges from your polygons, lump them together into an array of edges and run "simple_merge_e2p" on this set. The result will be polyons with holes and unlimited levels of containment.

    Matthias

  • Hi Matthias,

    Thanks a lot - this worked splendidly! Now I am even more keen to see the Python package. I hate this kind of questions, but can't help asking: do you have a rough idea (months, weeks, etc.) of the time of release with Python package?

    Cheers,

    Mikhail

  • I'm working on this. A prototype is there already (see the link above). But Windows is a hard thing to support as there are manifold deployment paths.

    My current challenge is to enable Qt-less lightweight builds so the whole thing can go into a distutils-based package. However, I need to say that it's hard work to pull an application apart that way.

    Kind regards,

    Matthias

  • Hi Mikhails, you can run the new version with the Python scripting on Windows today, if you have Windows 10. (I admit this is cheating, but ...) Following:

    https://seanthegeek.net/234/graphical-linux-applications-bash-ubuntu-windows/

    I have downloaded the Ubuntu experimental binary and installed it on Ubuntu on Windows (needs Windows 10). With the open source web server it works. I have yet to find an issue with that approach. For laughing out loud, I even start a Ubuntu script inside Windows Explorer like this:

    https://www.howtogeek.com/270810/how-to-quickly-launch-a-bash-shell-from-windows-10s-file-explorer/

    With my version of Windows I am stuck at 16.04 but 18.04 works with up to date Windows 10. Developer mode is no longer needed.
    Satya Nadella loves Linux.
    Basically, this approach has so far replaced Cygwin for me. It is more tightly integrated. Microsoft is paying Canonical for Ubuntu support AFAIK.

  • Thanks a lot for the nice hint! I am trying it now, and it does look nice. However, my ultimate goal is to have an .exe file runnable as windows process, and besides, requiring the user to enable Ubuntu on Windows appears to be asking for too much. All the same - this is fun stuff, and I am eager to explore it further.
    Thanks again,

    Mikhail

  • Hi,

    I have built the pymod under Windows10, WSL-Ubuntu16, Python 3.5.2, gcc 5.4.0, and now trying to use it. Got stuck with the following:

    import klayout.db as db
    
    output_file = 'test.gds'
    
    layout = db.Layout()
    top_cell = layout.create_cell("TOP")
    print('Writing output to {}'.format(output_file))
    layout.write(output_file)
    

    Result:

    $ python3 case.py
    Writing output to test.gds
    Traceback (most recent call last):
      File "case.py", line 9, in <module>
        layout.write(output_file)
    RuntimeError: Cannot determine format from filename in Layout.write
    

    Writing GDS from within KLayout pya macro works fine, of course.

    Any ideas?

    Thanks,

    Mikhail

  • Hi Mikhail,

    I think there is something wrong with the detection of streamer plugins.

    Basically the file readers and writers are provided through separate DLLs which are should be installed separately in a folder called "db_plugins". This folder is looked up relative to the "db.so" core library. If this folder is missing or

    I have not tested and debugged under Windows myself yet. But I'm currently preparing a "setup.py" for distutils. This should make deployment much easier. Windows debugging and enabling is on my list next.

    Thanks,

    Matthias

  • Thanks, Matthias - this is encouraging news! Something to look forward to.

  • For what it's worth, I generated the Python module "klayout" using the setup.py file on RHEL6, generated a 40MB test vehicle (compressed gds), and did XOR against the same Python script run within KLayout. The XOR was clean. So this is pretty amazing, since I can now use PyCharm with the built-in default Python and use the klayout Python module (and any other module). Great job !!

  • setup.py worked for me too! As mentioned above, my configuration is
    Windows10, WSL-Ubuntu16, Python 3.5.2, gcc 5.4.0

    Thanks, Matthias and Erwin!

    Native Windows + Anaconda Python is still a Holy Grail for me...

  • Thanks everybody. That's good news.

    Windows + Anaconda is on my roadmap. I have a few issues to solve - like getting the dependency libraries. I don't think this is a basic issue, but setup.py still is kind of mysterious to me.

    Matthias

  • distutils, setuptools and setup.py will only deal with python dependencies. If we need other platform-dependent dependencies, unfortunately we have to code it ourselves. ubuntu has apt-get, fedora-like has yum, mac has brew, but windows it's more complicated -- you basically give an instruction manual asking the person to "install that software first, otherwise this doesn't work".

    I think it is fair to copy-paste third-party code and build/ship with the main project for situations like this, provided the licensing is compatible.

  • Here is an update: as Anaconda is MSVC-based on Windows, I started to make the code build on MSVC again. But that a major effort as MSVC is deviating from gcc/clang considerably in some areas. It will take a little.

    Stay patient :-)

    Matthias

Sign In or Register to comment.