Huge oasis processing

edited December 2016 in General
Hi all,

I am trying to use KLayout to process huge oasis file. I got some questions here:

1. After KLayout read in oasis file,
1) is all the layout data kept in memory?
2) Or some of the data written to hard disk as tmp? If yes, where is it?
Leading edge design layout oasis file size may be larger than 100G

2. I used following ruby script to flatten a hierarchical oasis layout with size of 4.7G (the same hierarchical layout saved in GDSII format is 247G), on a Linux server with 196G memory. KLayout process was killed due to out of memory.
--------------------------------------------------------
#input_file
#output_file

ly=RBA::Layout.new

ly.read($input_file)
topCell=ly.top_cell
topCell.flatten(true)

ly.write($output_file)
---------------------------------------------------------

Is there any work around to flatten such an oasis file?

3. Is there any API to generate a huge oasis file? Say I want an oasis file with size of 200G, but I don't care the content, random polygons/layers/cells would be OK.


Any comments will be appreciated.

Thanks,
Benny

Comments

  • edited November -1

    Hi Benny,

    You probably mean mask data, do you? Design data (hierarchical, pre-OPC, pre-fracture) is hardly 100G OASIS. Maybe you're expecting too much of an open source tool. There are commercial solutions that allow you deal with that kind of data. If you're really doing leading-edge design, I think you should not depend on open source projects.

    And yes, KLayout does it in-memory. But give it enough swap and you're basically back to file caching.

    However, flattening a layout is giving me the creeps. Flattening usually explodes your data. The GDS/OASIS file sizes don't tell anything about the actual number of flat shapes this operation needs to produce. Maybe your layout contains regular SRAM areas where a single cell array can easily explode into hundreds of millions of shapes. In that case, flattening a layout is bound to produce a wealth of shapes. I think that is the issue you face here.

    I sense that to you it's just about producing big test data. There is no API, but the OASIS spec is open. So you can basically create a writer that produces a binary OASIS stream serially. That gives you a way to create any amount of OASIS data.

    Matthias

  • edited December 2016
    Hi Matthias,

    Thanks a lot for reply, very helpful. Yes, I am trying to produce big oasis file for testing.

    Benny
  • edited November -1
    Hi Matthias,

    If I got a layout with multiple layers, and use following script to load one single layer "1.0"
    ----------------------------------------------------------------------------------------------
    ly=RBA::Layout.new

    spec=RBA::LayerInfo::new(1, 0)
    layer_map.map(spec, ly.insert_layer(spec))

    opt=RBA::LoadLayoutOptions::new
    opt.set_layer_map(layer_map, false)
    ly.read($input_file, opt)
    ----------------------------------------------------------------------------------------------

    Will KLayout read only Layer-1.0's data into memory and skip other layers, resulting less memory consumption compared to "ly.read($input_file)" ?

    Thanks,
    Benny
  • edited November -1

    Hi Benny,

    yes, that's right. The other layers will be skipped, provided you pass "false" as the second parameter to "set_layer_map".

    Regards,

    Matthias

Sign In or Register to comment.