Object callbacks, connected objects following each other when moving

lukasc

Hi Matthias,

I am wondering if it possible to associate a function as a callback for objects (cells, PCells, polygons, etc), triggered by object moves? I know that a PCell has them for parameter changes, but I don't see anything for moving.

What I am looking into implementing is having objects follow each other. Namely, if I move one object (say a PCell), the path connecting to it would follow.

If you have a chance, I'd love your feedback on what I've been working on the for the past week. These are add-on functions specific for silicon photonics / integrated optics, using Python. I'm starting to create a library of PCells, and scripts for layout with waveguide functionality. I'm also implementing a simple LVS functionality, except with a focus on Verification and Schematic / Netlist extraction.

https://github.com/lukasc-ubc/SiEPIC_EBeam_PDK

Thanks

Lukas

Matthias

Hi Lukas,

there is not really a "following" callback, but your idea is the one that is behind the "handles" of the circle: these are dummy shapes (in fact: points) which you can drag and the layout of the cell will follow. In that case it's a simple circle, but you could have complex paths following one or more handles too.

Regarding the other question: right now I'm not working on a lot. I'm quite busy and the spare time which is left is entirely taken by bug fixing and support. So no big news, I'm afraid.

Matthias

davidnhutch

Lukas,

... though in such a case as Matthias describes, the "follower" and "followee" both have to be inside the same PCell, which may not be what you are ideally looking for. I presume you want to move some component (let's say it's an MMI) and have a waveguide follow it when you move it?

A callback system would be nice but I'm sure a major work in the source code. But you know of the Partial tool right? You can select the MMI and certain path points, and drag to both move the MMI and stretch the path in one fell swoop, and path stays connected to the MMI.

It only works under certain circumstances: (i) the waveguide has to be path or polygon, ROUND_PATH won't work because its guiding shape points aren't selected using that action*, and (ii) only works for 90deg wg bends (as opposed to sinusoidal curve) and for horizontal or vertical moves of the MMI -- basically, moves where you are stretching parts of the waveguide that are already straight horizontal or vertical, but just making those segments longer or shorter).

Anyway, not exactly what you wanted but I thought I'd point it out in case you haven't fully explored the Partial tool yet or hadn't considered it as a combo move-a-cell and stretch-a-waveguide tool.

======================

Matthias,

One single point of the guiding shape of ROUND_PATH can be selected using Partial tool if you click on the point and you can move it. But that point and in fact several points can't be "Partial" selected by dragging a box across them. Is this the intended behavior? I would expect that ROUND_PATH should instead behave like a regular path, where you can indeed select multiple points by dragging a box using Partial tool, and shift-click to select more, before finally dragging those selected points.

If that were rectified, then it would help make the above waveguide-routing system I describe above much better.

David

lukasc

Thanks Matthias and David.

I think I figured out a way to implement the follow functionality. (updated on GitHub)

I have two hotkeys:

• W: snaps the path to the nearest pins, and uses the ROUND_PATH to make it a waveguide. Keep track of which pin and waveguide are connected.

• Shift+W: put the ROUND_PATH back to a path

Next, to do:

• H: Heal the waveguide if an object was moved. Re-connect.

Matthias

Hi David (and Lukas),

thanks for pointing out the partial selection issue. Basically PCell guiding shapes should behave like normal shapes and hence the selection rectangle should apply here as well.

I'll try to fix this in the next minor release.

Best regards,

Matthias

davidnhutch

Lukas,

Your github code looks great. However I am having trouble running it. Two questions: (I am new to Python, these are probably simple.)

1) I notice from your video you are using the same KL version as me (0.24.3). (I am running Windows however.) I get SyntaxErrors on the 'print' commands. A quick search showed that Python 2 uses print command while python 3 uses print() function. So, it appears you are using python 2 while I have python 3. (I didn't install python separately, I'm just using whatever came in the KLayout windows installer.) Changing these print commands to print() functions fixes the SyntaxErrors, however I am still puzzled why you are using python 2 -- did you compile KL 0.23.4 from scratch, intentionally using python 2.x? Furthermore would you consider using the built-in Python 3.x instead for your github code, since that's what comes with the latest KLayout?

2) This is a similar question to one someone just asked on a separate post, but it throws an error on the line "import numpy as numpy", complaining that "No module named 'numpy'" -- how to include this? Did you compile KL with numpy included, and if so, how? Preferably I'd like a way to include it in to an already-compiled KL build though.

Thanks,

David

lukasc

Hi David,

On the Mac, python is pre-installed, and is version 2.7. And on both Linux and OSX (apple), KLayout isn't built with Python -- KLayout just uses the system-installed Python. And on the Mac, we have "numpy" (numerical python, arrays, etc).

In Windows, for some reason, Python gets bundled into KLayout, and is using version 3. And KLayout's build of Python doesn't include numpy for some reason. (separate discussion started by Jonas on this).

I just changed the two things you mentioned:

• changed print "", to be print ("")
• removed numpy commands

So hopefully it will now run on Windows with Python 3.

Let me know if it works!

thanks

davidnhutch

Thanks Lukas,

You missed a few print commands - for example lines 73 and 79 in SiEPIC_EBeam_Path_to_Waveguide.lym

After fixing that and running that .lym file while a path is selected, it says:

Caught the following exception: 'pya._Iterator' object has not attribute 'next' (Class AttributeError)

on line 394 of SiEPIC_EBeam_functions.lym.

An internet search revealed (link):

If you need code that runs in both Python 2 and Python 3 without 2to3 conversion you can make a function that under Python 2 calls iterator.next() and under Python 3 calls next(iterator). The six module contains such a function, called advance_iterator().

So I changed it from p_iter.next() to next(p_iter)

Then it has an error on line 179 of ..._functions.lym:

name 'MODULE_NUMPY' is not defined

Commenting that out gives another error on line 357 of ..._functions.lym:

unsupported operand type(s) for %: 'NoneType' and 'tuple' (Class TypeError)

I stopped there as I'm not sure how deep this rabbit hole goes. But anyway it is clear that there must be a better solution for Python 2 (KL on linux) - to - Python 3 (KL on windows) interoperability. I'll wait for Matthias to comment, he usually has a clever idea I haven't thought of yet ;-)

David

lukasc

Hi David,

I think we need to make it work for both 2 and 3. I'd rather make the code work on any installation and not have to ask people to modify their setups...

The "six" package is a good idea. It is there by default on my OSX KLayout. However, it's not there by default on my two Linux machines (old debian, new RedHat). Could you please check if you have it in your KLayout? Just type in "import six" in the Python console and see if you get an error.

I can also test this here, with one computer running Python 2, and the other 3...

I'll fix these...

Matthias

Uh ... you're expecting a lot from me :-)

The Python 2 to 3 transition was a major break, so out of the box, only simple code will run unmodified on both versions. Using the necessary if's you can always include switches without having to use a 3rd-party module.

This is basically what six.py does (code copied from six.py):

# Useful for very coarse version differentiation.
PY2 = sys.version_info[0] == 2
PY3 = sys.version_info[0] == 3    

...

if PY3:
    ... Python 3 code
else:
    ... Python 2 code

The advance_iterator() implementation itself is amazingly simple and doesn't even use a switch:

try:
    advance_iterator = next
except NameError:
    def advance_iterator(it):
        return it.next()

There are more pitfalls of this kind, but there should be ways to work around them.

Best regards,

Matthias

davidnhutch

@Matthias,

I think (hope :-) Lukas was saying "we need to make [his SiEPIC PDK] work for both 2 and 3" rather than we need to make core KL work such that its scripts work for both 2 and 3. So, neither he nor I am asking that of you.. which would be a major undertaking!

But, still there will forever be a fundamental problem here, because Windows comes packaged with 3, and other installations use existing 2 (at least, it is my understanding that Python 2 is widely deployed in Linux systems while Python 3 has so far had more limited adoption in Linux/OSX -- correct me if that is wrong).

So, Linux/OSX users have the freedom to choose whatever Python they want (because it's not co-packaged with KL) while Windows users have to use Python 3.

Perhaps package Python 3 with all three (Linux, OSX, Windows) rather than having Python packaged with Windows installer and using built-in Python on Linux/OSX?

[EDITED:]

Or how about this as an imperfect but perhaps relatively straightforward fix. Option B:

Would it be possible to include both Python 2 and 3 in the Windows installer, and have a toggle switch in the File > Setup as to which one the interpreter uses? Then, a Windows user could have the same flexibility that a Linux user has, and could choose whatever makes sense for them. But of course the user couldn't have, for example, one script using Python 2 calling another script using Python 3.

Or, perhaps easier: Option C: Build one windows installer with Python 2 and one windows installer with python 3, then the user can decide at download time which one to grab.

Out of options B and C, I think B is better.

David

davidnhutch

@Lukas,

import six

in the console with the Python radio button pressed just gives:

No module named 'six'

David

lukasc

Thanks Matthias and David,

So it is interesting how many things the Windows build is missing (numpy, six).

I just installed Python 3 on my OSX computer, hoping that KLayout would start using this. But alas. It sounds like it needs to be recompiled.

Matthias – would it make sense to move all 3 KLayouts to Python 3, as David suggests? I'm sure I won't be the only one who will want his/her Python-KLayout scripts to work on all three platforms! However, similar to David, I'm new to Python so don't have a good understanding of why there are two versions, and why they didn't maintain backwards compatibility over simple things like print and .next().

David – I fixed the errors you've identified, but have no way to test them.

lukasc

Can you also check if this works in Windows KLayout?

 from time import gmtime, strftime
 strftime("%Y-%m-%d %H:%M:%S", gmtime()) 

thanks

davidnhutch

Lukas,

David – I fixed the errors you've identified, but have no way to test them.

I'll test it again, but will report errors to you via email rather than by this thread, because there could be a lot more going back-and-forth and I don't want to clog up this thread.

Can you also check if this works in Windows KLayout?

It works and returns the date and time.

David

lukasc

Matthias,

David found something I can't figure out. Namely that in Windows, "RecursiveShapeIterator object is not an iterator"

Here's the sample code:

layout = pya.Application.instance().main_window().current_view().active_cellview().layout()
LayerSi = pya.LayerInfo(1, 0)
LayerSiN = layout.layer(LayerSi)
cell = pya.Application.instance().main_window().current_view().active_cellview().cell
iter2 = cell.begin_shapes_rec(LayerSiN)
while not(iter2.at_end()):
    print(iter2)
    next(iter2)

It works on OSX/Linux, in Python 2.7, except with the change:

    iter2.next()

And it must also work in Ruby. Is there something about the different implementation in Python 3.4 that prevents this from working?

Thanks
Lukas

lukasc

Check this out:

davidnhutch

Thanks Lukas,

This may be obvious, but I'll add that the error is thrown on the final line. (It creates RecursiveShapeIterator just fine, it just fails on next(iter2).)

David

jfluecki

Matthias,
A bit more input on the above error: (running on Windows, Python 3.4.2 (default, Jul 25 2015, 23:50:22) [MSC v.1600 64 bit (AMD64)
iter2.next() does work for the RecursiveShapeIterator but not on a 'regular' iterator (e.g. p_iter = path.each_point) where next(p_iter) works. Is that because the RecursiveShapeIterator is a Klayout specific class?

Jonas

Matthias

Hi Jonas,

yes, you're right. The RecursiveShapeIterator is not a Python iterator - it's called an iterator because it implements the pattern, but it is no related to Python iterators in any way. next is simply a method of this class, so you cannot use next(iter) on this object. The similarity is confusing, I admit.

Regarding the choice of Python: there is no reason not to use to Python 3, but it will be difficult to drop Python 2 support entirely. CentOS 6 which is old, but quite common in enterprise EDA infrastructures, comes with Python 2.6 only and the EDA infrastructure maintainers are notoriously reluctant to add new package. So I even had to extend the compatibility range downwards. For more recent Linux distributions, Windows and probably MacOS as well, there is no reason not to use Python 3. For the binary distributions I support I can try to make sure to use Python 3 where possible.

In general I think it makes sense to specify the Python requirements in the scripts and check for the correct version before running into hard-to-debug errors.

Regards,

Matthias

lukasc

Hi Matthias,

In the latest KLayout 0.25, is there any functionality that could enable callbacks, as per the original question? Namely, when I move a fixed cell instance, or pcell instance, I would like it to trigger the execution of some code..

Thanks!
Lukas

Matthias

Hi Lukas,

not really, no. The API's are still the same - more or less.

I guess you can basically watch the selection by connecting to the LayoutView#selection_changed event. This gives you a callback when the selection has changed (which also happens on move). But I think the difficulty will be to detect whether an object has moved.

Matthias