need compilation advice: python3 cannot "import klayout.db" with version 0.28.15 [solved]

bobo38

Hello here !

I'm playing around with openPDK gf180mcu right now and was very happy to reuse KLayout after so many years for layout edition. It's also awesome that KLayout is first class layout tool for this PDK :-) with DRC and LVS setup. I could not test those last week-end because I ran into some issue:

$ python3 -V
Python 3.11.7
$ python3 $PDK_ROOT/gf180mcuA/libs.tech/klayout/drc/run_drc.py -h
Traceback (most recent call last):
  File "/stock/sky130de/share/pdk/gf180mcuA/libs.tech/klayout/drc/run_drc.py", line 59, in <module>
    import klayout.db
ModuleNotFoundError: No module named 'klayout'

I downloaded the PKGBUILD from https://archlinux.org/packages/extra/x86_64/klayout and tried to compile changing the ./build.sh options with no success so far. I guess I'm doing something stupid during the compilation if I read the changelog in the sources correctly: it should work if "installed properly".

Now that I write this post I realize that it might be related with this bugfix https://github.com/KLayout/klayout/pull/1618 . Submitting this post anyway in case somebody runs into the same issue: I will try to compile from latest source and come back if it's solving the issue.

$ grep KLAYOUT_VERSION= version.sh 
KLAYOUT_VERSION="0.28.15"
$ grep -C 3 -s klayout.db Changelog
* Bugfix/enhancement: some LEF/DEF parser issues solved with the help of a new complete sample case
* Bugfix: Avoid a segfault while editing a ruler. This happens is both a selection and a transient selection is active.
* Enhancement: Some enhancements for image editing (e.g. selection remains after moving handles)
* Enhancement: klayout.db Python module is auto-loaded for providing stream readers
* Enhancement: Spice reader now supports parametric subcircuits
* Enhancement: Build issue fixed for Qt 5.15.2 bindings
* Enhancement: Including Python's matplotlib into Windows binaries
--
      KLayout supports production of standalone Python modules 
      which provide the same features than the "pya" module but 
      for use without the KLayout binary. If installed properly,
      "import klayout" will pull in all modules. "import klayout.db"
      will pull in the database part.
      Deployment on PyPI is supported.
* Enhancement: Modal help dialog from modal windows

bobo38

Compilation from 0.28.17 source (with " Fixed issue #1610: leaking reference in Python debugger #1618 ") is running :-)

bobo38

Compilation Finished. Retesting. Issue is not solved with last version of KLayout source.

$ klayout -v
KLayout 0.28.17
$ python3 $PDK_ROOT/gf180mcuA/libs.tech/klayout/drc/run_drc.py -h
Traceback (most recent call last):
  File "/stock/sky130de/share/pdk/gf180mcuA/libs.tech/klayout/drc/run_drc.py", line 59, in <module>
    import klayout.db
ModuleNotFoundError: No module named 'klayout'

Here is to give an impression about the package content and installed dependencies (in case something should miss). Please ask me in case you need more information for debugging, I'm out of ideas atm.

$ tree pkg/klayout/usr
pkg/klayout/usr/
├── bin
│   └── klayout
├── lib
│   ├── db_plugins
│   │   ├── libcif.so -> libcif.so.0.28.17
[...]
│   │   ├── libpcb.so.0.28 -> libpcb.so.0.28.17
│   │   └── libpcb.so.0.28.17
│   ├── lay_plugins
│   │   ├── libbool_ui.so -> libbool_ui.so.0.28.17
[...]
│   │   └── libxor_ui.so.0.28.17
│   ├── libklayout_ant.so -> libklayout_ant.so.0.28.17
[...]
│   ├── libklayout_tl.so.0.28.17
│   └── pymod
│       └── klayout
│           ├── dbcore.cpython-311-x86_64-linux-gnu.so
│           ├── laycore.cpython-311-x86_64-linux-gnu.so
│           ├── libcore.cpython-311-x86_64-linux-gnu.so
│           ├── QtCore.cpython-311-x86_64-linux-gnu.so
│           ├── QtDesigner.cpython-311-x86_64-linux-gnu.so
│           ├── QtGui.cpython-311-x86_64-linux-gnu.so
│           ├── QtMultimedia.cpython-311-x86_64-linux-gnu.so
│           ├── QtNetwork.cpython-311-x86_64-linux-gnu.so
│           ├── QtPrintSupport.cpython-311-x86_64-linux-gnu.so
│           ├── QtSql.cpython-311-x86_64-linux-gnu.so
│           ├── QtSvg.cpython-311-x86_64-linux-gnu.so
│           ├── QtUiTools.cpython-311-x86_64-linux-gnu.so
│           ├── QtWidgets.cpython-311-x86_64-linux-gnu.so
│           ├── QtXml.cpython-311-x86_64-linux-gnu.so
│           ├── QtXmlPatterns.cpython-311-x86_64-linux-gnu.so
│           ├── rdbcore.cpython-311-x86_64-linux-gnu.so
│           └── tlcore.cpython-311-x86_64-linux-gnu.so
└── share
    ├── applications
    │   ├── klayoutEditor.desktop
    │   └── klayoutViewer.desktop
    └── icons
        └── hicolor
            └── 32x32
                └── apps
                    └── klayout.png

13 directories, 241 files

$ pacman -Qi klayout
Name            : klayout
Version         : 0.28.17-1
Description     : High Performance Layout Viewer And Editor. Support of GDS and OASIS files.
Architecture    : x86_64
URL             : https://www.klayout.de
Licenses        : GPL
Groups          : None
Provides        : None
Depends On      : libgit2  qt5-base  qt5-tools  qt5-multimedia  qt5-xmlpatterns  qt5-svg  python  ruby
Optional Deps   : None
Required By     : None
Optional For    : None
Conflicts With  : None
Replaces        : None
Installed Size  : 172.96 MiB
Packager        : Unknown Packager
Build Date      : Mon 26 Feb 2024 01:10:15 PM CET
Install Date    : Mon 26 Feb 2024 02:39:46 PM CET
Install Reason  : Explicitly installed
Install Script  : No
Validated By    : None

bobo38

Archlinux "klayout" package maintainer contacted by email. I've checked dependencies on KLayout GitHub everything looks there… (and next time I will use more cores for compilation)

bobo38

Some more investigations content of "pymod" directory should not be stored in "/usr/lib/pymod" but likely in such a directory: "/usr/lib/python3.11/site-packages". I will try to figure out what to change.

It's an Archlinux PKGBUILD bug not a KLayout bug. Sorry for bothering here

bobo38

Now it's working \o/

$ python3 /stock/sky130de/share/pdk/gf180mcuA/libs.tech/klayout/drc/run_drc.py -h
Run GlobalFoundries 180nm MCU DRC.

Usage:
    run_drc.py (--help| -h)
    run_drc.py (--path=<file_path>) (--variant=<combined_options>) [--verbose] [--table=<table_name>]... [--mp=<num_cores>] [--run_dir=<run_dir_path>] [--topcell=<topcell_name>] [--thr=<thr>] [--run_mode=<run_mode>] [--no_feol] [--no_beol] [--no_connectivity] [--density] [--density_only] [--antenna] [--antenna_only] [--no_offgrid] [--split_deep] [--macro_gen] [--slow_via]

Options:
    --help -h                           Print this help message.
    --path=<file_path>                  The input GDS file path.
    --variant=<combined_options>        Select combined options of metal_top, mim_option, and metal_level. Allowed values (A, B, C, D, E, F).
    [...]       

Root cause was that pymod content wasn't installed at the right place. Now it is:

$ tree usr
usr
├── bin
│   └── klayout
├── lib
│   ├── db_plugins
│   │   ├── libcif.so -> libcif.so.0.28.17
│   │   ├── libcif.so.0 -> libcif.so.0.28.17
[...]
│   ├── libklayout_tl.so.0.28 -> libklayout_tl.so.0.28.17
│   ├── libklayout_tl.so.0.28.17
│   └── python3.11
│       └── site-packages
│           ├── klayout
│           │   ├── db
│           │   │   ├── __init__.py
│           │   │   └── pcell_declaration_helper.py
│           │   ├── dbcore.cpython-311-x86_64-linux-gnu.so
│           │   ├── __init__.py
│           │   ├── lay
│           │   │   └── __init__.py
│           │   ├── laycore.cpython-311-x86_64-linux-gnu.so
│           │   ├── lib
│           │   │   └── __init__.py
│           │   ├── libcore.cpython-311-x86_64-linux-gnu.so
│           │   ├── QtCore.cpython-311-x86_64-linux-gnu.so
│           │   ├── QtDesigner.cpython-311-x86_64-linux-gnu.so
│           │   ├── QtGui.cpython-311-x86_64-linux-gnu.so
│           │   ├── QtMultimedia.cpython-311-x86_64-linux-gnu.so
│           │   ├── QtNetwork.cpython-311-x86_64-linux-gnu.so
│           │   ├── QtPrintSupport.cpython-311-x86_64-linux-gnu.so
│           │   ├── QtSql.cpython-311-x86_64-linux-gnu.so
│           │   ├── QtSvg.cpython-311-x86_64-linux-gnu.so
│           │   ├── QtUiTools.cpython-311-x86_64-linux-gnu.so
│           │   ├── QtWidgets.cpython-311-x86_64-linux-gnu.so
│           │   ├── QtXml.cpython-311-x86_64-linux-gnu.so
│           │   ├── QtXmlPatterns.cpython-311-x86_64-linux-gnu.so
│           │   ├── rdb
│           │   │   └── __init__.py
│           │   ├── rdbcore.cpython-311-x86_64-linux-gnu.so
│           │   ├── tl
│           │   │   └── __init__.py
│           │   └── tlcore.cpython-311-x86_64-linux-gnu.so
│           └── pya
│               └── __init__.py
[...]

Relevant PKGBUILD section, I've added the 4 lines below "### deal with python modules":

$ grep -A 20 -e "package()" PKGBUILD 
package() {
  install -D -m 644 klayoutEditor.desktop "$pkgdir"/usr/share/applications/klayoutEditor.desktop
  install -D -m 644 klayoutViewer.desktop "$pkgdir"/usr/share/applications/klayoutViewer.desktop
  cd klayout-$pkgver
  install -D -m 644 etc/logo.png "$pkgdir"/usr/share/icons/hicolor/32x32/apps/klayout.png
  install -D -m 755 build-release/klayout "$pkgdir"/usr/bin/klayout
  cd bin-release
  for lib in `find . -type f | grep -e so`; do
    install -D -m 755 $lib "$pkgdir"/usr/lib/$lib
  done
  for lib in `find . -type l | grep -e so`; do
    cp -a $lib "$pkgdir"/usr/lib/$lib
  done
  ### deal with python modules
  rm -rf "$pkgdir"/usr/lib/pymod
  local site_packages=$(python -c "import site; print(site.getsitepackages()[0])")
  mkdir -p "$pkgdir$site_packages"
  cp -r pymod/* "$pkgdir$site_packages"
}

Sorry again for bothering here for a packaging issue. Writing here forced me to explain the problem, hence find the root cause and solution without real help. Have a great week !

Matthias

No worries. Thanks for sharing this experience ...