Undefined Material Data

Hi Matthias,

I can't create an account on GitHub for some reason, so I am posting the question here for now.

I have modified the xSection 1.4 to allow blocks to be executed from the XS file. This is so that I can get a xsection from a portion of the processing if needed. I can do FEOL alone, or BEOL alone, or various modules. However, if .grow() or .etch() is given material to :into or :on or :through that hasn't been created yet (because it was in a module that I de-selected), the code just gives up. I tried very hard to insert something like if

    Object.const_defined?(:GATE_OX).to_s.include? "true"

in your code in the MaskData class methods grow and etch. But the calling of the class during runtime already fails when I try to pass a non-existing variable into .grow().

I would like to create empty materialdata at the beginning of the xs file for every planned .grow(), .etch(), .deposit(), etc. and I have tried something like

    GATE_OX = MaterialData.new([], 0)

but it creates runtime errors about no layout being created. Can you help ?

Thanks.

Thomas.

Comments

  • Come on ... you can't be the only person on earth not getting a GitHub account ... :)

    To solve your problem, you can either use (note "self")

    gate_ox = MaterialData.new([], self)
    

    or simply:

    gate_ox = nil
    

    In the second case, you can use "gate_ox" on "output", but it won't be output at all. While in the first case, it will become an empty material layer.

    BTW: please try to use lower case material names (at least for the first letter). Upper case variables are constants in Ruby, so whenever you redefine a material you'll receive "Redefining a constant" warnings.

    Matthias

  • nil still creates the same error "expected material or array of material" when given to :through in .grow()

    MaterialData.new([], self) works to prevent the hang-up, but this new empty layer blocks .grow() if I provide it as :through, although it is a bit non-sensical.

    I am getting close to giving up on this sectioning of the x-section module.

  • Hi Matthias
    in mask_data, I replaced:

              if !v.kind_of?(Array)
                through = [v]
              else
                through = v
              end
             through.each do |i|
                if !i.kind_of?(MaterialData) 
                  raise "'#{method}' method: 'through' ..."
                end
              end
    

    with:

              if !v.kind_of?(Array)
                through_temp = [v]
              else
                through_temp = v
              end
             through = []
             through_empty = true
             through_temp.each do |i|
               if !i.kind_of?(MaterialData) 
                 raise "'#{method}' method: 'through' ..."
               end
               if i.get_polygons.size> 0
                  through.push(i)
                  through_empty = false
               end
             end    
             if through_empty
               through = nil
             end
    

    and I added the get_polygon method to the class material_data. Now, it properly disregards empty materials in :through of .grow()

  • Thanks for the patch ... but that would mean - in case of multiple specifications in "through" - if one "through" component is empty, none of them is considered, right?

    I wonder how you can split your script if your backend step requires a "through" specification from the frontend part. My feeling is that in this case the separation isn't happening at the right level.

    Or maybe you can replace the frontend by a dummy one (e.g. a "deposit") which provides the "through" target. Or you consider the frontend masks to be empty in still execute the frontend stack. This would render a uniform but valid material stack.

    Matthias

  • Actually, I think it is the opposite. I assume that all provided :through materials are empty by setting

       through_empty = true
    

    as I loop through all through_temp: if one material with more than zero polygons is found then

       through_empty = false
    

    In the end, I only set the real through materials to nil if through_empty was still true. So this means that as soon as at least one :through material is not empty, the whole "though" object is not nil. The only issue I can possibly see is that the section ruler could run through an area where :though calls for a an empty material in one location, and some other valid material in another location. In that case, it would not nil the through array, but the part of the array that represents the empty layer would still block the .grow(), because it its still empty.

    I think you are right. This should not be attempted in parse_grow_etch_args(args, method), but at the actual grow() step. Maybe I can figure out to test for empty materials there.

    Thanks.

    Thomas.

  • I have to ask one last question. I made a progressbar by counting the number of "output" statements in the xs file, and then incrementing it with every "output" command during the eval(). all works fine, except when I click on the cancel button on the bar. I don't know how I could properly end things in the section script when that happens. I tried many things, but it is not working, as you can see in the code that tries to capture the event exception. It just ends up leaving the progressbar on the screen and hanging up everything. the program then continues to use a lot of CPU processing and can only be task-manager-ended. also, I noticed that there are more than 1 progress bars overlaid at times, at runtime. At times, the progressbar is twice as wide and has the label centered, at times there are 2 bars with the same counter in it, yet only one label and one cancel button. I meddled with the code for an eternity, and I don't know why it would be there more than once. I only created it once, and I make sure at runtime, that it is only created once. Could there be something of a bug in RBA:RelativeProgress ?
    Here is the modified output function, and the ones that set up the progress bar. Pretty much everything else still has the original xs skeleton in it. Any idea how to implement this correctly ? I could not find much in the forum, neither in the online documentation.

    Thanks.

    Thomas.

    def output(layer_data, layer_num, data_type, layer_name = "", stipple = nil, color = nil, screenShotString = nil)
        if layer_data==nil
            puts "no Layer data"
            return
        end
        view = RBA::LayoutView::current
        length = 50
        layer_name_padded = (layer_name.upcase + ' ' * length)[0,length]
        newLayerName = "#{format('%03d', layer_num)} / #{format('%04d', data_type)} : " + layer_name_padded
        ls    = RBA::LayerInfo.new(layer_num, data_type, newLayerName)
        li    = @target_layout.layer(ls)
        lInfo = @target_layout.get_info(li)
        @target_layout.set_info(li, lInfo)
        shapes = @target_layout.cell(@target_cell).shapes(li)
        shapes.clear # new feature
        set_layer_prop(ls, li, stipple, color)
        @ep.boolean_to_polygon([ RBA::Polygon.new(@roi) ], layer_data.data, RBA::EdgeProcessor::mode_and, true, true).each do |polygon|
            shapes.insert(polygon)
        end
        if instance_variable_defined?("@p1") && @p1!=nil
            if @p1.kind_of?(RBA::RelativeProgress) && !@p1.destroyed
                puts " incrementing @p1"
                begin 
                    @p1.inc
                    puts "INCREMENTING #{$xsNum}"
                rescue => ex
                    puts "CANCEL in pinc ?: " + ex.to_s
                    while !@p1.destroyed
                        @p1.destroy
                    end
                    sleep 2
                    puts "destroyed: " + ex.to_s
                    @target_layout.update
                    view.update_content
                    @target_view.update_content
                    RBA::Application.instance.main_window.redraw
                    view.zoom_fit
                    @target_view.zoom_fit 
                    view.max_hier
                    @target_view.max_hier
                    finish_output_view
                    @target_view
                    puts "restore to normal ?"
                    throw :stop
                ensure 
                    puts "@p1 dealt with in #{$stepNum}"
                end
            end
        end 
    
        if screenShotString!=nil
            fName = "X" + format('%02d', $xsNum) + "_S" + format('%02d', $stepNum) + "_" + screenShotString
            $stepNum = $stepNum + 1
            if $saveImages
                saveScreenShot(fName)
            end
            if $waitForClick==true
                wait (fName)
            end
        end
        puts "new output done"
    end
    #######################################################################
    #######################################################################
    def make_rel_prog_bar(eval_text)
        if !$waitForClick
            if !instance_variable_defined?("@p1")
                puts " creating @p1"
                @p1 = RBA::RelativeProgress::new("Building xSection #{$xsNum} ...", count_em(eval_text, "output"))
                puts " @p1 created: #{count_em(eval_text, "output")} outputs"
                $cancel_pressed = false
            else
                if @p1.kind_of?(RBA::RelativeProgress)
                    @p1.destroy
                    @p1 = RBA::RelativeProgress::new("Building xSection #{$xsNum} ...", count_em(eval_text, "output"))
                    puts " @p1 created: #{count_em(eval_text, "output")} outputs"
                end
            end
        end     
    end
    #######################################################################
    #######################################################################
    def evalSection(callingClass, eval_text, points, cv)
        puts "evalling"
        callingClass.destroy     # closes QDialog
        if points.size>0
            $xsNum = 1
                points.each do |pp|
                puts "new xsection"
                $stepNum = 0            
                @target_view = nil
                setup(pp[0], pp[1], cv)
                update_basic_regions
                make_rel_prog_bar(eval_text)
                if eval_text.size>0
                    eval(eval_text, binding, @file_path)
                end
                if instance_variable_defined?("@p1")
                    if @p1.kind_of?(RBA::RelativeProgress)
                        if !@p1.destroyed? 
                            puts " destroying  @p1"
                            @p1.destroy
                        end
                    end
                end             
                finish_output_view
                    @target_view    
                $xsNum = $xsNum + 1 
            end
        end
    end
    #######################################################################
    def run_after_click(callingClass, points, cv)
        $eval_text = ""
        text_comm =  loadSection($file_lines, $common_section[0], $common_section[1])
        text_secs = loadSections($file_lines)
        if $sections!=nil && $selected_sections!=nil
            numSelSections = countSelectedSections()
            puts "numSelSections=#{numSelSections}"
            if numSelSections>0
                $eval_text = text_comm + text_secs
            else
                $eval_text = ""
            end
        else
            $eval_text = combineLines($file_lines)
        end
    
        if !$eval_text
            raise("Error reading file #{@file_path}")
        else
            if $eval_text.size>0
                # adding stop condition 
                $shouldStop   = false
                catch :stop do
                    evalSection(callingClass, $eval_text, points, cv)
                end
                combineCells()
            else
                raise("Error reading file #{@file_path}")
            end
        end
    end
    
  • edited February 2020

    Hi Thomas,

    To be frank, I don't fully unterstand the logic of your code. There seems to be a lot of experimental code and the lonely @target_view line at the end of evalSection does not make sense.

    Basically I like the idea of a progress, but I'd rather like to speed up the computation by using C++ functions where possible (right now, some limitations of the C++ features prevent this). Anyway, augmenting "output" probably is somewhat too weak as most scripts I've seen put the output at the end of the script and progress would not have an effect in the bulk of the script.

    For the ProgressReporter case here is some tested scenario for a progress reporter that should be similar to the XSection use case:

    class Runner
    
      def initialize
        # .. nothing yet ..
      end
    
      def run_for_some_time
        r = RBA::Region::new
        10000.times do |i|
          r = (r + RBA::Region::new(RBA::Box::new(i, 0, i + 1, 1))).merged
        end
        puts r.to_s
      end
    
      def one_step(num)
        run_for_some_time
        @p.value = num
      end
    
      def run(text)
        # NOTE: to count steps better use absolute progress
        # The second argument (incorrectly called "max_value" in the doc)
        # is the yield interval - if set to 1, the progress reporter
        # will enter the event loop on every inc() or value=()
        @p = RBA::AbsoluteProgress::new("Starting", 1)
        @p.format = "Step %.0f"
        @p.unit = 1
        @p.format_unit = 1
        begin 
          self.instance_eval(text)
        ensure
          @p._destroy
          @p = nil
        end
      end      
    
    end
    
    runner = Runner::new
    
    runner.run(<<"END")
    10000.times do |i|
      one_step i
    end
    END
    

    I'm using AbsoluteProgress as counting steps is IMHO better represented in absolute units. I don't know why ProgressReporters pile up in your case, but I think it's because the ProgressReport objects are not properly cleaned up by _destroy (destroy is the same, but _ marks it as a built-in method which is never overloaded). I make sure the object is destroyed in "ensure" which is always executed.

    A note of caution is required here: if you run the code in the debugger and stop execution, the object is not destroyed. A bad thing about embedding an interpreter is that the objects stay alive in the system. A nasty side effect of a pending ProgressReporter is to disable many UI features which might look like a stuck program in your case. The code above - not executed in the debugger - is a safe solution.

    Please also see the code comments about the "yield interval". This makes the progress reporter enter the UI event loop on every value change which is important to keep the "Cancel" button alive. KLayout basically is a single-threaded application (yet) which means the event loop needs to be triggered occasionally.

    Regards,

    Matthias

  • Hi Matthias,
    I inserted sleep commands everywhere to hunt down where the multiple bars are coming from. I think I found it !!!
    I noticed that the LayoutView methods affect it. Here is an example. This is executed in your xsection_generator::output method while a progressbar was on the screen.

        app = RBA::Application.instance
        mw = app.main_window
        lv = mw.current_view
        puts "about to zoom"
        sleep 1
        lv.zoom_fit
        puts "zoomed"
        sleep 0.5
    

    lv.zoom_fit made the relative progressbar double wide in this slow mode. If I comment lv.zoom_fit out (without sleep), then I get the normal behavior. Meaning always only 1 PB, and I can click on cancel and capture that in rescue. However, when lv.zoom_fit is still in there, the PB is multiple widths and sometimes double or triple. Clicking on 1 cancel button stops it, but I assume that more instances were created somehow during "lv.zoom_fit", and that's why it hangs.

    I tried this for other methods like update_content, save_image, and save_screenshot. They all result in a wonky progressbar that I cannot cancel right.

    Any idea what I can do ?

    Thanks.

    Thomas.

  • Hi Thomas,

    Thanks for tracking this down. But this is really weird ... I forgot to ask which system you're on. Is this Windows? I'm trying on Linux.

    Matthias

  • Hi Matthias,
    yes, windows 10.

    Thomas.

  • Sorry, I still don't get it ... the code parts you are pasting don't allow me to reproduce the problem.

    Please fork XSection at GitHub (https://github.com/klayoutmatthias/xsection), push your changes and let me see the whole code ... I don't want to waste my time guessing.

    Matthias

  • Hi Matthias,
    I can't put the whole code on there. You would need all the processing data and some layout. This may be a bit too much in terms of confidentiality. I will head over there though and finally get set up. I have another simpler problem I may need you wisdom for.

    However, I noticed that this also happens outside the xSection module. When I open a very big gds (>1GB), I do also see stacked progress bars, sometimes it is double or triple wide. I wonder if that's something in this version (0.26.3). I don't recall ever having seen this in my years with KLayout.

    Thanks.

    Thomas.

  • Hi Thomas,

    the stacking is due to a change which is supposed to make progress more transparent: a progress is wrapping a loop and the loop operations itself are having a progress. For example: iterating over cells and doing something elaborate on each cell. This is shown by the stacked progress bar - the upper one is the outermost and below are the inner progress bars. So stacking is an indication of nested progress objects.

    For GDS reading there is a final step of building the lookup tables and cache. This creates nested progress objects.

    In your case there should not be such a nesting because you call "_destroy" on a progress reporter before creating a new one. That's the weird thing I don't understand.

    Matthias

Sign In or Register to comment.