Python Scripting for reading an Excel file and creating a polar array on Klayout Macro

Hi,
I used the following python script to read an excel file (attached) and generate a text script as input for CNST (Java based nanofab toolbox developed by NIST) toolbox. This CNST toolbox is supposed to generate GDS2 file. However, for large size files, CNST toolbox stalls.
My script first create a line array of circles where position and radii for individual circles are given in the Excel file. Then this script creates a polar array of this line array of circles; the final layout is a metalens of circular symmetry.
Can someone either modify this python script (copied below)or guide me how to do this on Klayout Macro?
Thanks a lot.
KS

import csv
import math

circles = []

Open csv file

with open('circles.csv') as csvfile:
circleReader = csv.reader(csvfile)
for num, row in enumerate(circleReader):
if num != 0:
circles.append((float(row[1]), float(row[2]), float(row[3])))

Open file for output

with open('metalens.cnst','w') as output:

# Header
output.write('# This file is created using metalens.py\n\n')
output.write('0.001 gdsReso\n')
output.write('0.001 shapeReso\n')
output.write('1 layer\n')


# create all circle structures
for num, circle in enumerate(circles, start=1):
    output.write(f'<circle_{num} struct>\n')
    output.write(f'0 0 {circle[1]/2} {circle[1]/2} 0 ellipseVector\n\n')

# create metalens
output.write('# Metalens structure comment\n')
output.write('<metalens struct>\n')

#create each ring of the metalens from the inside out
for num,circle in enumerate(circles, start=1):
    theta = 180*circle[2]/math.pi
    if num == 1:
        output.write(f'<circle_{num} 0 0 N 0 0 instance>\n')
    else:
        output.write(f'<circle_{num} 0 359 {theta} {circle[0]} {circle[0]} 1 1 arrayPolar>\n')

Comments

  • Hi Matthias, Any input on the above? Is this doable on Klayout Macro Development or modified python script can directly create the gds2 file? In either case, can you direct me as to how to modify the above code to achieve this? Thanks.
    KS

  • Hi,
    I'm a complete outsider of CNST but challenged your problem since I have solved a similar problem very recently.
    Does this look like what you want finally?

    Regards,
    Kazzz-S

  • Hi Kazzz-S,
    Thanks for response. Yes, this is how it should look like, but the issue is when the number of initial set of circles (whose positions and radii given in excel file) exceeds, lets say 1000 (or approx 500 micron diameter metalens), CNST crashes.

  • Kazzz-S, If you can share how you solved your problem, it will be a great help and i would appreciate it.
    Regards,
    KS

  • Hi,

    Please refer to the files in the attached ZIP file, where
    1. I have remade your original Excel file based on my guess and created a CSV file to fit my solution.
    2. Even if you have 1000 rows in your real-life problem, I hope this solution can work gracefully. KLayout is powerful:-)
    3. In my solution, I used "Pandas" to handle a CSV file.
    4. Also, I used the "PCell" of KLayout in my approach. Read the KLayout's manual regarding PCell.
    5. There are a few global control parameters you can try. One of them is to convert "PCells" to "Static (ordinary)" cells.

    I hope the main PYA script KazzzS-KL1642.py is self-explanatory.

    Regards,
    Kazzz-S

  • Kazzz-S,
    Heartening to see someone taking time to help (unknown) another. Thanks so much. let me take a look and get back with you. By the way, i have over 13,300 points in the excel file with position, radii values, as well as angular incremental values for each circle having upto 7 decimal places, but i agree Klayout should be able to handle this.

    KS

  • edited September 2020

    Hi,

    I'm benefitting from KLaout a lot! My activity is a small token of thanks to it.

    I have found some typos and fixed them. Sorry about that. Please update with the second ZIP file.

    By the way, I got the reason why there are many digits below the decimal points in the sample data.
    Perhaps, you need to choose a more refined database unit other than 0.001 [um], which is the default.
    In my applications, I have never changed this value.

    Regards,
    Kazzz-S

  • Very nice comment about how you give back after benefiting a lot from Klayout.
    Yes, it makes sense to change the data values to make the smallest value correspond to Klayout resolution of 0.001 um. Will do.
    Also, i am new to Kalyout Macro, so it will take some time for me to report to you what happened. Will get back with you.
    Thanks again.

  • edited September 2020

    ..

  • Hi dsenuka,

    what's the platform you're on? If it's Windows 64bit, you can manually install pandas using this recipe: https://www.klayout.de/forum/discussion/1645/pandas-library-for-windows#latest

    Matthias

  • edited September 2020

    Hi dsenuka,

    1. Pandas issue: please follow Matthias' instruction. Thanks, Matthias :)
    2. LYP file: a layer property file is for customizing the layer color, stipple, etc. So, it does not work with an empty design.
      • In the ZIP file, you find "KL1642.gds" file for demonstration purposes. First, open it. The color and stipple settings are KLayout's default.
      • Next, File===>Load Layer Properties OR Drag&Drop "KL1642.lyp" then you can see the above image.
      • Once the PYA script run, you can do the same operation for changing the layer properties.
    3. File location: I've tested the script on Mac. So,
      • os.environ["HOME"] ==> "/Users/kazzzs"
      • then, the full path of MyRoot ==> "/Users/kazzzs/KLayout/Forun1642/", which is the directory where I stored the related files.
      • if you are using Windows and placed files under C:\KLayout\Forum1642\ you can change it to MyRoot="C:/KLayout/Forum1642/"
      • Similarly, you can specify any existing directory for GDS2 output. In the sample, I chose the same directory as the input CSV.

    Kazzz-S

  • THANK YOU, Matthias and Kazzz-S ! Let me try these.

  • edited September 2020

    .

  • Tried several variations of folder path, like double back slash etc. still without any luck.

  • edited September 2020

    It looks you are in a small pitfall.

    MyRoot = C:\\Users\\rabeysin\\KLayout\\Forum 1642     # Wrong! It's not a string object! Syntax error.
                                                          # See the color of "KL1642.gds" in the text editor.
    
    MyRoot = "C:\\Users\\rabeysin\\KLayout\\Forum 1642"   # Nearly but still wrong! It should end with "\\"
                                                          # for succeeding concatenation with the file names.
    
    MyRoot = "C:\\Users\\rabeysin\\KLayout\\Forum 1642\\" # OK
    
    MyRoot = "C:/Users/rabeysin/KLayout/Forum 1642/"      # OK (Unix style)
    
    

    Kazzz-S

  • edited September 2020

    Thanks

  • edited September 2020

    .

  • i wonder how Repository Browser on the left side of Macro IDE look like (in the image i sent out earlier), anything to do there?

  • edited September 2020

    Your left side pane of the IDE looks normal. It's more or less the same as mine. Notice that the path is in Unix style in the 2nd & 3rd lines.

    When I tested some "MyRoot" expressions mentioned above, I could see a space between Forum and 1642 in the image.
    That's why the above sample.
    However, the error message you got does not contain the space if you copied and pasted it.
    It seems, in your existing file system, there IS a space between Forum and 1642 but your script does not have it,
    which causes the mismatch. Am I correct?

    BTW, it would be better to choose a fixed-width font for the IDE text window. Then, the presence of a space character is easily visible.

    Kazzz-S

  • edited September 2020

    Kazzz-S, When you get a chance, pl. let me know if there's any other materials/tutorials other than those documentation in Klayout site if one wants to master Macro development in python.
    Thanks again.
    dsenuka

  • edited September 2020

    Hi dsenuka,

    Even with the given real-data Excel sheet, I do not fully understand your real-life problem.
    However, if my assumptions are correct, you are trying to layout about 1.4 million circles in the quadrant-I.
    Then, my main interest has moved to the performance aspect of KLayout when handling such a big number, which I never come across in my daily work.
    I have modified the previous PYA (no significant change was required) and created a new CSV file with 1335 rows.
    Below is the summary output of the modified PYA:

     :
    ### <KazzzS-RealKL1642.py> processed <RealKL1642.csv> ###
        KLayout version: KLayout 0.26.8
        Machine info: Darwin, MacBookPro2.local, x86_64
        Using PCell?: Yes
        Total number of instances inserted: 1,399,419
        Timing info: Sys=2.420[sec]  User=27.920[sec]  Wall=30.414[sec]
    

    The output GDS2 file size is about 43MB.

    As you can see, your real-life problem is an easy win for KLayout!
    KLayout won't crash when handling such a large number of instances.
    So I believe it's worth learning PYA for you. Try it by yourself!

    Some images follow.
    Near the origin:

    Near 12 O'clock:

    Near 2 O'clock:

    Near 3 O'clock:

    Regarding additional materials to study PYA, I think,
    1. KLayout document set (either on-line web or built-in assistant) comes first
    2. Then, this forum
    Recently I have commented to this topic, which is related to your problem.

    Kazzz-S

  • edited September 2020

    Great !

  • edited September 2020

    Hi dsenuka,

    I've attached the related files.
    You can make experiments changing different parameters.
    Good luck.

    Kazzz-S

  • Hi Kazzz-S, Great ! Thank you so much again !
    dsenuka

  • edited September 2020

    .

  • edited October 2020

    Hi dsenuka,

    It's weird!
    For a quick testing, on my Linux, I have created a new directory $HOME/KLayout/Forum1642x/, which holds only RealKL1642.csv to start with.
    After the new script KazzzS-RealKL1642.py ran, I got RealKL1642.gds in the same directory.

    Line-27 to -35 are as below in the testing.

    Debug          = True   # set False not to print debug messages
    NumVertices    = None   # set an integer like 32 or 16 to fix the number of vertices
                            # of a polygon that approximates a circle
    Convert2Static = False  # set True to convert PCells to static cells
    MyDBU          = 0.001  # set an appropriate database unit in [um]
    NumDigits      = 8      # set an appropriate number of digits below the decimal point
    MyRoot         = os.environ["HOME"] + "/KLayout/Forum1642x/"
    MyCSV          = MyRoot + "RealKL1642.csv"
    OutGDS2        = MyRoot + "RealKL1642.gds"
    

    Check your code carefully.
    Kazzz-S

  • Awesome, it worked ! Folder were all good, i had to hit "run from the current tab" button to get it to run and generate the gds2 file.
    Thank YOU Kazzz-S !!!

  • Cool application!

    And thanks to @kazzz also from my side!

    Matthias

  • Hi dsenuka,

    Thank you for your feedback. Hope the performance is enough for you.

    In my guess, you are going to finish up the layout by referring to the present quarter piece four times with 0-, 90-, 180-, and 270-degree rotation.
    You can also achieve that step by extending the existing PYA script. Try it if required.

    Kazzz-S

  • Hi Kazzz-S, Yes, i need to expand this to a full circular lens. Right now, i am checking if circle positions are correct and how you re-produced the columns in csv file. Thanks again.
    dsenuka

Sign In or Register to comment.