Importing .py files from technology directory

kevinz

I'm using KLayout 0.30.4 on FreeBSD and Windows.

I have a technology file for which I've set a custom base path (outside of ~/.klayout). It follows the standard KLayout technology structure with a pymacros/ subdirectory containing both a .lym (set to auto-run) that imports .py files.

KLayout does not seem to have trouble finding this pymacros/ directory since they show up in the per-technology Python scripts in the Macro Editor. However, if I have e.g. Disk.py and try to import Disk from the .lym file, I get an error "No module named 'Disk' (Class ModuleNotFoundError)".

I have to add:

import sys
sys.path.append("/path/to/my/tech/pymacros")

Before this will work.

The macro editor documentation suggests that "Technology folders: each technology folder can carry a "macros" or "pymacros" subfolder where technology-specific macros are kept" which seems to imply that the technology folder should automatically be added to the import path.

Is this the intended behavior? Would you consider automatically adding the technology path pymacros/ folder so that it is importable?

Matthias

Hi @kevinz,

There are actually two paths for Python and Ruby code:

• pymacros (Python) and macros (Ruby) is supposed to contain "rich" macros. Such macros are not plain Python or Ruby files, but XML files. These files contain the code, but additional information such as menu binding, auto-execution etc. Naturally, such need to be top-level entry points as they cannot be imported into Python.
• python and ruby are the folders where plain Python or Ruby code needs to go to. These folders should be in the search path.

A note regarding technology-specific macros and code: "technology-specific" does not mean that this code is executed only when that specific technology is needed. This is just a way to contribute additional code that may, but does not need to be related to the technology. You can bind libraries to technologies and that is usually what this code does. But you need to be careful with potential name collisions. Specifically the top level macros share the same namespace. Also the search path is unspecific and not relative to the top level macro.

So you should encode the technology in the module names, like

Folder structure

tech/
  sky130/
    pymacros/  <-- not added to sys.path
      auto_exec.py   <-- does "import sky130.something"
    python/  <-- added to sys.path
      sky130/
        something.py

If you don't include "sky130" in the Python structure, another technology could define "something.py" and modules would clash.

Matthias

kevinz

I see now, thanks! What I was missing was that plain Python files should go in python/ and not pymacros/.

kevinz

Actually, it doesn't seem like import found the macros.

I'm in the rich Python macro (.lym) DiskRingLib which has import Cap. Cap.py is in the technology directory python/ (and thus shows up under "python branch"). Running the rich macro gives:

Manually adding the path works around the issue, as before.

Let me try with an official build on Windows and see if that improves things...

Matthias

Hi @kevinz,

Seems like that scheme is not supported for technologies. I think that's a bug, but I need to check first if there is some issue preventing that path enabling in general.

My proposed workaround is this:

import pya

import sys
import os

py_path = os.path.join(os.path.dirname(__file__), "..", "python")
if not py_path in sys.path:
  sys.path.append(py_path)

import Cap

Matthias

Matthias

I created a GitHub issue for tracking that topic: https://github.com/KLayout/klayout/issues/2169

Matthias

kevinz

The work around works, thanks!

On Windows 0.30.4, the Macro Editor does not even recognize the presence of the python/ directory. On FreeBSD (built from source), the python/ subdirectory at least shows up in the Macro Editor as the "python branch", despite not being automatically added to the path.

Thanks for creating the issue and perhaps this will be available in a future release. For now, the workaround is fine for me.