Bug with boolean operations

lemaitfl

Hi all,

When making boolean operations between 2 regions, I ran into the following error: "ERROR: src\tl\tl\tlThreadedWorkers.cc,259,! m_running"

I made a minimum example after quite some time troubleshooting, and here is what I observe:

• The error comes randomly, sometimes it works, sometimes it doesn't.
• The shapes don't even have to touch each other (not related to weird shapes or positions).
• It occurs when some shapes are in a child cell (2 levels down in my case).
• It occurs when I use the DeepShapeStore() to keep the cell hierarchy.
• It only occurs for regionB - regionA, but not for regionA - regionB.
• Apparently running on Mac or Linux doesn't give an error.

I attach the script and gds input file leading to an issue. In my case, all the iterations without hierarchy work (100), then all operations A-B work (100) and then it gives the error somewhere before succeeding 100 times B-A operations.

Can someone check if it occurs on another windows computer?
Do you have any idea of what can cause this?

Thanks in advance!
Florian

Matthias

Hi Florian,

thanks for this well-prepared test case.

As expected, I cannot reproduce it on Linux, but I will try on Windows.

Could you indicate which is the KLayout version you're using?

The error indicates some race condition, although there are enough Mutexes to avoid this problem (problem with pthreads library?).

Maybe you can try entirely disabling threads by using

dss.threads = 0

Matthias

sebastian

Hi Matthias,

I think I can somewhat confirm. I don't have the same exact error as Florian,

but I am running on Debian 11 with

Python 3.9.2 (default, Feb 28 2021, 17:03:44) 
[GCC 10.2.1 20210110] on linux

with both klayout==0.27.10 and the newest version (I think 0.27.11?) I get the Seg fault when substracting regions in fast succession (but not always, so I think race condition is reasonable). I can try with python3.10 tomorrow if that helps.

Edit: I will also happily test Florian's test case and report back when I have run it.

Also, this is in the normal python module, not in the Qt application.

Edit2:

Maybe ignore my post. I think these might not be related. But, I encountered this error when using the region_multi_fill for the top cell. When calculating the region for the fill it would randomly segfault (I basically used manual tiling). Unfortunately cannot provide code to replicate (yet)

StefanThiede

on linux I found valgrind —tool=helgrind useful. It pointed to issues even though the app was working.

Matthias

I'm afraid I do not see the issue also on Windows (version 0.27.12, Windows 10 Pro, 21H2). I increased to 10000 iterations and wasn't able to reproduce the problem a single time.

I don't think the original problem is a memory corruption issue. The error message hints to a threading issue and failing locks or wait semaphores. The standard KLayout build uses pthread.dll and atomics for locking. If for some reason the wrong pthread DLL is picked up, such an error may happen.

@lemaitfl Are we talking about the GUI application or maybe you're referring to the Python module? Are you using the standard binaries?

Thanks,

Matthias

lemaitfl

Hi all,
First of all, thanks a lot for your efforts in helping out!
Somehow I don't have the issue anymore today... (last time I tried restarting my computer and it didn't fix the issue).
I am running python scripts in PyCharm (but I also got the same issue by running the script in Windows PowerShell last time).
If of any help, I am using KLayout version 0.27.7, and python 3.10.2, on Windows 11 Pro 21H2.

If it appears again, I'll try your dss.threads = 0 suggestion.

I don't understand much from the rest of your posts
But for now I'm back on track
Thanks again for your help!

lemaitfl

I got the error again from running another script, and then I get it when running my minimum example...
I don't know where it comes from, but the dss.threads = 0 seems to get rid of it
Is there any consequence of disabling the threads?

Matthias

@lemaitfl So I assume you are using the klayout module from PyPI - maybe there is an issue with the pthread library or the atomics implementation (from https://github.com/mbitsnbites/atomic). I'm not really an expert on this - maybe there are restrictions on the CPU type.

If you set dss.threads to zero, all threading will be disabled and execution is synchronous. No mutexes are wait conditions are involved and races cannot happen. That means you cannot utilize multiple cores, but the deep engine doesn't scale well with number of cores anyway. Beside that there is no disadvantage.

Matthias