›

import numpy

ashkanashkan
edited November 2023 in Python scripting

Hi,

I recently tried to rerun some of my old codes (maybe from a year ago), and surprisingly, they didn’t compile! In some of the codes, I imported the numpy library, but Klayout returned this error:

ImportError: Error importing numpy: you should not try to import numpy from
        its source directory; please exit the numpy source tree, and relaunch
        your python interpreter from there.

Did Klayout change anything that prevents numpy from being imported?
My Klayout version is 0.28.13.

To add to this problem, this issue appear only in Windows version and not in Ubuntu version.

Comments

  • MatthiasMatthias

    Hi @ashkan,

    I am regularly upgrading the build environment as including new components forces me to do so. This means that also numpy versions change.

    In 0.28.12 numpy was 1.21.0. In 0.28.13, numpy is 1.26.1.

    I am not really familiar with numpy and I have no idea what this message means. But plain "import numpy" on the console works, so I don't think there is a general issue. Maybe you did set a sys.path that numpy does not like? Or maybe the version makes a difference?

    Regards,

    Matthias

  • sekigawasekigawa

    I'm having the same problem in both the Windows binary distribution and the MINGW64 source build.

    1. Comparison of Different Builds





    2. Initial Survey
    https://github.com/numpy/numpy/issues/24965#issuecomment-1772872708

    Somehow you've ended up with an installation of numpy where from numpy.__config__ import show fails.
Normally that only happens if numpy is imported from inside the numpy source tree.
It sounds like you installed numpy via pip, so that shouldn't be happening, which makes me think
you have a broken install somehow.

I see some minGW strings in your debug output
 - minGW isn't supported by us and isn't tested in our CI so there may be issues we aren't aware of.
   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
I doubt anyone will spend a lot of time working on this so you'll likely have to dig in a little
bit more into what's going wrong.

    Despite the above comments,

    3. Reinstallation
    I uninstalled then reinstalled numpy and the packages that depend on it, but the problem remained unchanged.

    4. Values of sys.path
    I also checked the sys.path and found that the source build was weird.


    5. KLayout 0.28.13 on different MacOS
    No problems. :)

    Best regards,
    Kazzz-S

  • sekigawasekigawa

    6. Values of sys.path (again)

    I edited the contents of bin-release/.python-paths.txt and got the expected path settings.

  • sekigawasekigawa
    edited December 2023

    7. Further investigation(1)

    The original exception message seems inappropriate, so I have modified numpy/__init__.py and got different messages.


    To be continued...

  • MatthiasMatthias

    Hi @sekigawa,

    thanks for these investigations.

    I could not reproduce the problems myself yet, but I will try with different machines. The ones I tested had numpy installed on them.

    I actually did not change anything between 0.28.12 and 0.28.13 except adding libgit2 and upgrading to latest MSYS. I did not a few times before and never ran in trouble with numpy.

    Matthias

  • MatthiasMatthias

    I could reproduce the problem after wiping my system entirely. So I am suspecting some essential DLL is missing.

    sys.path looks normal to me.

    I will try to find the missing pieces ...

    Matthias

  • MatthiasMatthias

    Looks like for some reason the essential DLLs are missing in the installation. I assume the packaging script has some trouble with the new MSYS version. I will try to fix that and provide new Windows packages.

    Matthias

  • MatthiasMatthias

    I stand corrected: the DLLs are now called ".pyd" - so they are there. It's a different problem then.

    Matthias

  • MatthiasMatthias

    Okay, found the issue: some DLLs that the .pyd depend on are not installed. I got it working by copying

    /mingw64/bin/libopenblas.dll
/mingw64/bin/libgomp-1.dll
/mingw64/bin/libgfortran-5.dll
/mingw64/bin/libquadmath-0.dll

    to the same place than "_multiarray_umath.cp311-mingw_x86_64.pyd".

    The reason for the missing DLLs is that the build script collects dependent DLLs, but only analyzes ".dll" files, not ".pyd".

    I think that is easy to fix. I'll provide a new Windows installer.

    Matthias

  • MatthiasMatthias

    I have created a ticket for this here: https://github.com/KLayout/klayout/issues/1553

    Matthias

  • sekigawasekigawa
    edited December 2023

    Hi @Matthias,

    Thanks for your effort :)

    8. Further investigation(2)
    After manually copying the four DLLs (in the source build env), I can import numpy normally.
    I hope the other cases have the same root cause.

    Best regards,
    Kazzz-S

  • MatthiasMatthias

    I fixed the deployment script and uploaded new installers. Version is still 0.28.13 because only DLLs got added. Please re-install using the Windows installers from https://www.klayout.de/build.html.

    I can import numpy, scipy, pandas, matplotlib and other packages now.

    Thanks for these investigations.

    Matthias

  • sekigawasekigawa

    Hi @Matthias,

    Thanks for the quick fix.
    I've tested the new Windows ZIP package and found it OK!

    Kazzz-S

  • MatthiasMatthias

    Very good. I'm happy to hear it works on your side too! :)

    Best regards,

    Matthias

  • ashkanashkan

    @Matthias It works on my device too.
    Thanks.

  • ashkanashkan
    edited December 2023

    @Matthias I'd like to report a possible bug in KLayout 0.28.13. I am not sure if it is related to my version, but I tried it on version 0.28.5 and it works fine. The problem occurs when I try to move an object with the Movements setting in Diagonal mode. I cannot move the object to any angle between 90 and 180 degrees or between 270 and 360 degrees. If I try to do so, the object’s movement will be mirrored on the X-axis. Could you please check on this issue and let me know if there is a solution?

    Edit1:
    Another issue that I have noticed is that when I try to move an array of objects by clicking on the desired location, they will suddenly jump to a different place.

    Edit2:
    I have found another strange behavior when I use a Pcell and then flatten it. If I modify anything on that specific Pcell in its layout, all other Pcells also change accordingly!

  • MatthiasMatthias

    Please use the issue tracker on GitHub for bug reports (https://github.com/KLayout/klayout). This discussion is about a different topic.

    But I guess you're not facing a bug. Instead you are editing objects inside the hierarchy and that will make all other places, where this object is used, change as well. This is the magic of hierarchical layout.

    Turn on "Select Top Level Objects" in the "View" menu, so you can only select objects from the top cell. This will avoid these issues, but you have to enter subcells if you want to do edits there. "Descend in hierarchy" (Ctrl+D) is a way to do so and it is comparable with "Edit in Place" in Cadence.

    Matthias

Sign In or Register to comment.
Powered by Vanilla