Storing custom data in GDS/OASIS files?

Anyone knows if there is some way to store custom data in GDS/OASIS files from the Python interface? I my application, I would like/need to read and write strings and floats.

Thanks,
Robert

Comments

  • edited April 2022

    Hi Robert,

    GDS in general lacks a standardized way to store attributes in a generic way.

    But you can use "user properties", which are essentially key/values pairs. GDS supports numeric keys and string values.

    User properties need to be attached to something. In GDS these "somethings" are shapes or instances. If you happen to have instances or shapes you can attach properties to, you're lucky. Otherwise you can generate your own objects. For example, KLayout stores PCell data in a hidden top cell which is populated with dummy objects to keep track of key/value pairs. When KLayout restores the data, it needs to be aware that the order might have changed (order of objects is not defined in GDS). So the keys and values need to be designed in a way their order does not matter.

    If you need more than that, you may consider switching to OASIS. OASIS is extended beyond GDS in two ways: first, OASIS also offers properties that can be attached to cells and globally to the file (= Layout object in KLayout). Furthermore, property keys and values are typed and can be integers, strings, floats or even lists in OASIS.

    KLayout implements file and cell-level user properties for GDS as well, but formally such an extension is not compatible with mainstream GDS.

    OASIS also has special records for storing custom objects (e.g. XGEOMETRY). KLayout does not use them as I'd not rely on other tools preserving them in a data processing chain and the interpretation is application dependent. User properties however are standard features and are supposed to be understood and processed by all tools.

    Matthias

  • Thanks. I was trying this but did not quite manage. Please tell me if I misunderstand. I did:
    pya.Application.instance().main_window().current_view().layout.setProperty("Foo", "Bar")
    and then
    pya.Application.instance().main_window().current_view().layout.property("Foo")
    which did return
    'Bar'
    However, saving the layout as .gds and ticking the "Write cell/layout properties", closing and then opening the file and running
    pya.Application.instance().main_window().current_view().layout.property("Foo")
    again, 'Bar' is not returned.

    Related to this, I noticed that if I save a gds-file and tick that it should be in ascii mode, the next save still writes it in binary mode. I don't know if this is a desired behavior, but it seems more natural to continue to save the file in the format that was set.

    Finally, if the above way is the way to store parameters, would it be possible to have some way to have the "Write cell/layout properties" boxes ticked by default? Either by script or in general KLayout settings?

    Thanks!

  • Hi Robert,

    As I mentioned, GDS cannot store name keys. It can only store numeric keys. So ```setProperty(1, "Something")` should work.

    I do not know if the GDS file/cell level properties are understood by any other program than KLayout. This is quite a risky option and I'd rather not like to make it default. You may not be able to read the GDS files with other tools. When you write by script you can set this option using a SaveLayoutOptions object and setting SaveLayoutOptions#gds_write_cell_properties and SaveLayoutOptions#gds_write_file_properties to true.

    Thanks about reporting the issue with saving GDS text files. I have created a ticket for this: https://github.com/KLayout/klayout/issues/1071

    Matthias

  • Hi again,

    Seems it does not unfortunately:
    > pya.Application.instance().main_window().current_view().layout.setProperty(1, "Bar") Argument cannot be converted to a string in QObject_Native.setProperty (eval):1

    Putting the 1 in quotes does not work either, then it is the same as in the previous post.

    The reason I want this is this is that I need to store external information from some parametric cells as a backup. My PCells have external dependency, and I want to check if the external dependency has changed from last time the file was opened. So I would like to save a backup of the external data in the GDS-file. Another way I was thinking to do this is to simply use a hidden parameter of the PCell. I think that would be a very nice solution, but then I would need to know when a GDS-file is about to be saved so that I can update the "backup" parameter of my PCells just before saving so that I do not have to write to the backup parameter all that time, which will be time consuming. Is there some way to do this? Inform a PCell that it is about to be saved in a particular GDS-file?

  • edited May 2022

    Hi @ravn,

    the method is "set_property" (https://www.klayout.de/doc-qt5/code/class_Layout.html#k_127), not "setProperty". Honestly I don't know what "setProperty" does. Is that a Python thing maybe?

    So this works for me:

    ly = ... some layout ..
    ly.set_property(1, "value")
    options = pya.SaveLayoutOptions()
    options.gds_write_file_properties = True
    ly.write("test.gds", options)
    

    But regarding your application idea: this will not work. Any library cell is agnostic about it's use (or rather: encapsulated). So it is not informed where is it instantiated and it will not be told when it is saved. You cannot access the layout where the PCell is placed from within the PCell if the PCell lives inside a library.

    I recommend to use a hidden parameter - i.e. "version" - and store a version code. For the first release of your PCell you store 1 for example. With the next (upgraded) release you make the version code 2. Inside the "coerce_parameters" callback you can check the version code and take measures to align the parameters - e.g. translate from version 1 to 2 if you read a 1 in the version field.

    Matthias

  • Hi again @Matthias , thanks for the feedback. I will try to understand properly the mechanism behind how a library is instantiated in a layout since I now have bugged you with questions regarding this several times. Sorry about that.

    About the code, I found the setProperty-method from dir(ly). I assumed it was some typo going from set_property as it is stated in the documentation. setProperty works in the sense that I can add stuff but it is not stored in the file.

    Your proposal set_property does not work from Python for me unfortunately, or I am doing something wrong. This is the result:

    pya.Application.instance().main_window().current_view().layout.set_property(1, "Waaa!")
    Attribute set_property not found (tried lookup as a child widget)
      :/built-in-pymacros/qt_helper.lym:24
      (eval):1
    
  • No worries about bugging me :)

    Regarding the problem, here is the proper solution:

    pya.CellView.active().layout().set_property(1, "Waaa!")``
    

    The explanation is that: pya.Application.instance().main_window().current_view() gives you the current layout view (= tab in the main window). It has a "layout" method, but it does not do what you want. Instead it gives you the QLayout which is a concept of the Qt widget that the layout view also acts as. QLayout is entirely unrelated to pya.Layout.

    I admit that is confusing.

    Actually the LayoutView can hold multiple GDS/OASIS/CIF... layouts (pya.Layout). So there is not a single layout object, but multiple ones, each kept inside a container called "CellView". So a "LayoutView" has multiple cell views. You could now use "pya.Application.instance().main_window().current_view().cellview(0).layout()" to get the layout object of the first cell view (BTW: the shorter version is pya.LayoutView.current().cellview(0).layout()).

    As one of the layouts is the "active" one (i.e. the one you will draw on in edit mode), the easiest is to take the active cell view (pya.CellView.active()) and use it's layout object.

    For completeness, you could also check if there is any active cell view at all:

    if pya.CellView.active() is None:
        # error: no layout loaded
    ...
    

    Matthias

  • Ah, thanks. Clear.

    Given the constraints you describe above, this is probably not the way to do what I want. I continued to think about this and realized that hidden parameters in the PCell class looks like a better solutions. However, when playing with that, it seems those parameters can only be modified from the coerce_parameters_impl-method for some to me unknown reason.

    My PCells have an external dependence on their parameters. So, what I want to do is to store the last evaluated value for some parameters in the GDS-file so that when it is opened at a later time, the PCell can check if any of the external dependencies have changed and raise a warning if they have.

    But for this to work, I need to be able to write to the parameters in several places, so is that possible?

  • @ravn You're right, "coerce_parameters_impl" is the only place to manipulate parameters. That is an architectural decision and it required as parameters for example are essential for caching layouts. So if the system was allowed to change parameters in all possible cases, cache inconsistencies for example are hard to avoid.

    "coerce_parameters_impl" is basically called whenever the parameters changed, but not necessarily "frequently". You can basically include external sources at this point, but your should not do so in "produce_impl" for example, as this will create the mentioned cache inconsistencies. Hidden parameters participate in the cache lookup, so they are the perfect way to establish the state of a PCell.

    So if you're able to detect the mentioned external dependencies within "coerce_parameters_impl", you can store this last evaluated value in a hidden parameter and it will be available when you read the PCell again. This is essentially very similar to keeping file or cell parameters.

    Best regards,

    Matthias

    P.S. I replied to your mail. Did you receive my response?

  • I see. It would probably be ok to update parameters in coerce_parameters_impl, but from what I can see that function is not called when cell.refresh is called. I am using cell.refresh to update the rendering of a cell from a file-system trigger when my external dependency has changed. But since coerce_parameters_impl is not called, I cannot update my parameters then. Would it be possible to have the cell.refresh-function call the coerce_parameters_impl?

    Kind regards,
    Robert

  • Yes it is and this is noted: https://github.com/KLayout/klayout/issues/1079

    :)

    Best regards,

    Matthias

Sign In or Register to comment.