Slow loading on large GDS2 files with many layers

Hi everyone, reaching out on behalf of my team here so please excuse if some details are missing!

We are experiencing very long load times when loading in GDS2 files with >100 layers, up to 15 minutes sometimes. The files themselves are ~3.5GB. Our current system specs are:

Intel i7 8700 @ 3.7GHz
48 GB 2666Mhz RAM
2 TB SSD

The biggest limiting factor appears to be the CPU, but any sort of upgrade would probably mean replacing the whole PC! Any suggestions to reduce load time with the limited amount of information I'm able to provide?

Thanks in advance!

Comments

  • Is the data already local on that SSD? Or coming from some other
    source? I imagine that the load-rate could be dependent on other things
    such as media data rate (transferring from an older USB2.0 thumb
    drive at 10MBPS, or over a 10Mbps Ethernet branch due to a lousy
    chain of local Ethernet switches, could take a while).

  • Hi Dick, everything is running off a local SSD. I've watched Performance Monitor while the file loads, and while drive utilization is pretty high, from what I can tell the bottleneck is always a single core on the CPU.

  • @JamesH46 >100 layers is nothing unusual. You can experience some drawing performance degradation with a huge (>1000) number of layers, but not for loading. Also 3.5G is a fairly small GDS file. Specifically given you have 48G memory at your proposal.

    With GDS there are few reasons something takes so long. If you load from a local disk, I don't expect such files to load longer than 1 Minute.

    There are a few things you can check:

    • Maybe the file is gzip'd and much larger in reality?
    • Maybe the file is called GDS and is something else? Like OASIS?
    • What is the memory history? A 3.5G GDS file should take something like ~10G of memory and viewer mode and a little more in editor mode.
    • Is the time spent in parsing (while the progress bars counts MB) or sorting (the stage afterwards)? You can also increase the verbosity in the log viewer ("File/Log viewer") to get numbers.
    • If the time is spent on sorting: maybe your GDS file has an unusual hierarchy like a huge number of cells?

    Matthias

  • Hi Matthias, thanks for your response! After viewing the log (thanks for the tip!), it appears the either the large number of cells or drawing process itself is the issue. Parsing and Sorting each take ~15 seconds, but during the drawing phase each of the 127 layers individual takes 5-10 seconds to draw, which adds up significantly! Is there any way to optimize the drawing process further? We can load in as top level hierarchy only, but at some point are still going to need to draw the entire layout.

    I've attached a screenshot of the portion of the log covering sorting and the first part of drawing in the event it's useful! Do you know if there's a way to add timestamps to the log?

    screenshot

  • @JamesH46 From the log I don't see much about the drawing times.

    So you're saying that you wait until the full picture is built? Drawing happens in a background thread, so usually one does not need to wait. A full layout image of all layers does not tell a lot anyway, so I wonder why to wait for that.

    You can also utilize multiple cores for drawing if you need to in "File/Setup -> Display/Optimization", 'Number of threads used for drawing'.

    Apart from that I don't think there is much room for optimization on the drawing performance. I squeezed out performance as much as I could already. Even hardware acceleration . There are some (rare) conditions under which drawing performance needs to be traded off for drawing quality. I prefer quality as I think there is little more annoying than seeing different images on different zoom levels.

    But I cannot analyse the problem in detail unless you provide me with a sample.

    Kind regards,

    Matthias

  • @Matthias Thank you for the recommendations! I don't think the team was aware they could load only the specific layers they need rather than waiting for the full layout to draw, and increasing the thread count has significantly reduced drawing times.

  • Very good - but actually all layers are loaded initially. This step is finished once the progress bar disappears and the canvas shows. After that the layers are drawn one by one (or in parallel with multiple threads) - but you do not need to wait. While the layers are drawn you can already start working with the tool.

    Matthias

Sign In or Register to comment.