›

How do I run KLayout's testsuite?

yearningmareyearningmare
in KLayout Development

So I have changed the code in a few places, and now I want to make sure that I didn't inadvertently break something.

Is there a manual on how to run tests? As far as I understand, KLayout officially supports all three major OSes, so I presume that I would have to run a copy of the KLayout reference Linux in a docker (based on python's "manylinux" quasi-distro?), as well as VMs for Windows and MacOS?

As far as I can see, the files are quite scattered over the source directory, with Jenkins, travis, azure, and maybe more ways to run the testsuite, so I would like to ask if there is a "safe choice" way of running tests locally, without signing up for all those "cloud providers" just to run the tests.

Something like "destroy 1 container and two VMs, sping new instances, built, test, print results" in a single script.

Comments

  • MatthiasMatthias

    Hi @yearningmare,

    Yes, there is such a way. You can run the tests locally, preferably on Linux. I'd recommend Docker and an old distro such as CentOS7. Your code changes need to be compatible with the baseline configuration of that distro in terms of C++ standard and Qt version.

    PRs are build locally on my own private Jenkins instance which also runs a number of private test cases I cannot publish. This includes MSVC/Windows and some selected Linux distros. Master is also tested in some selected configurations (MSYS/MinGW, Qt-less, 64bit coordinates, Anaconda-based etc.). But that happens only after merge. My resources are limited. Typically, code changes that work on Linux also work on the other platforms.

    For the Linux-based tests: When you build KLayout there is a binary called "ut_runner" in the build folder (not "bin" as it is not installed).

    To run it, you need to set two environment variables:

    • "TESTTMP" to point to a temporary folder holding test data
    • "TESTSRC" to point to the source directory

    On Linux, typically you also need to set LD_LIBRARY_PATH to the "build" folder.

    For example:

    $ ./build.sh ...  (creates 'build-release' for example)
$ export LD_LIBRARY_PATH=$(pwd)/build-release
$ export TESTSRC=$(pwd)
$ export TESTTMP=build-release/testtmp
$ build-release/ut_runner

    "ut_runner" has some options (use "-h"). You can select certain tests or run in editable or non-editable mode only. "-s" enables "slow" tests, which roughly doubles the execution time.

    The same concept holds for other platforms, including MacOS and MSVC. On Windows however, things are much more difficult to configure as Python intentionally blocks DLL load paths.

    A general remark: I am not prepared for accepting contributions. There are no contributor guidelines, nor do I have the bandwidth to maintain a modern-standard CI/CD environment. Also, contributions need to be aligned with roadmap or architecture and governance is also a matter of resources.

    There have been some attempts to sneak in unsafe code in the past and apparent attempts to social engineer a way into the project. I tend to accept contributions only from reliable sources known to me. This is not personal.

    Matthias

  • yearningmareyearningmare

    Well, so far I have just built klayout with ASAN (it builds and even runs), and the issues I am finding are mostly memory issues, so I don't expect the patches that I might suggest (or might not) to exceed a few lines.

    Concerning Centos7, are you happy with using "Software Collections" additional repositories for Rhel/Centos? Devtoolset-11 provides GCC-11, which fully supports C++-17, and provides some niceties from C++-20.

  • yearningmareyearningmare

    Where can I find the roadmap?

    (I guess the architecture is not likely to be documented?)

  • MatthiasMatthias
    edited June 25

    Hi @yearningmare,

    Regarding C++ and CentOS 7: distro-provided C++ and Qt only. No additional repos or dev packages. That means C++ 11, but not later. I am sorry. EDA folks are conservative.

    ASAN is probably a good idea. Thanks. I have been doing sanitizing with Coverity, SonarQube and Valgrind occasionally, but it is time for a new sweep. But I will review the PR closely. I don't know if I can trust you. Times are weird.

    Roadmap-wise I told you there is no governance. After all, this is a single-person, spare-time project. The roadmap is in my head, undocumented. Please talk to me about your plans before you start something and take care of the license. It is copyleft. I assume you know what the means.

    Matthias

  • HarutiHaruti

    Thank you @Matthias for provide this insight :)

Sign In or Register to comment.
Powered by Vanilla