How to make Python script run as background server to receive request and display things iteratively

Hi, Guys,

I need to debug a program which is about 2d geometry processing, and i have to examine the coordinates all the time in gdb. While debugging, it is really difficult when there are >10 data records to look at.

I wrote a gdb custom command to convert the information from gdb memory to .gds and call Klayout to visualize my shapes. That usage is ok. Now i want to improve this flow by making klayout as a gui display server, which is alive all the time, and different gdb process can send requests to the server to display shapes. My klayout server can update the view every time it processed a request from client.

The problem I have now is, how to make my python script run in the background.
I tried to use python thread, but it does not work. When I run the script in Macro Development IDE, it would run without error, but klayout does not display any shapes that i added in my thread.run()

The other questions is about the "redraw()" function, once i updated the data model of a cellview (the Layout instance), i called "view.add_missing_layers()". Not sure if that's right function to call.

in "gdb_displayer.py"

[import pya
import sys
import threading
import time

class GdbDisplayServer(threading.Thread):
    def __init__(self):
        super(GdbDisplayServer, self).__init__()  
        #-- prepare first lyaout view.
        self.mw = pya.MainWindow.instance()
        self.lvIdx = self.mw.create_view()
        self.lview = self.mw.view( self.lvIdx )

        #-- prepare the layout db and load it to our layoutview.
        self.layout = pya.Layout()
        self.top = self.layout.create_cell('TOP')
        self.lview.show_layout(self.layout, True)

        #--
        self.layers = {}
        print('gdb display server initialized.')

    def run(self):
        print('GdsDisplayServer thread starts!')
        for i in range(10):
            for j in range(10):
                x1 = i * 1000
                y1 = j * 1000
                x2 = x1 + 1000
                y2 = y1 + 1000
                self.add_shape(1, 0, x1, y1, x2, y2)
                time.sleep(2)
                print('dump a box  {} {} {} {}'.format( x1, y1, x2, y2))

    def add_shape(self, layerid, dataid, *coords):
        if (layerid, dataid) in self.layers:
            layer = self.layers[(layerid, dataid)]
        else:
            layer = self.layout.layer(layerid, dataid)
            self.layers[(layerid, dataid)] = layer

        if len(coords) == 4:
            #-- rectangle
            box = pya.Box( *coords )
            self.top.shapes( layer ).insert( box )
        elif len(coords) == 8:
            box = pya.Box( coords[0], coords[1], coords[4], coords[5] )

    def redraw(self):
        self.lview.add_missing_layers()
        self.lview.zoom_fit()

in gdb_displayer.lym

import pya
from importlib import reload
import gdb_displayer
gdb_displayer = reload(gdb_displayer)

server = gdb_displayer.GdbDisplayServer()
server.start()  

Comments

  • Reply to myself.
    After reusing the klayout qt http server sample code, and use "klayout -rm server.py" to launch the klayout. I can successfully get my server run in background.

    Here's a working prototype:

    import pya
    import sys
    import threading
    import time
    
    class GdbDisplayServer(pya.QTcpServer):
        def __init__(self):
            super(GdbDisplayServer, self).__init__()  
            #-- prepare first lyaout view.
            self.mw = pya.MainWindow.instance()
            self.lvIdx = self.mw.create_view()
            self.lview = self.mw.view( self.lvIdx )
    
            #-- prepare the layout db and load it to our layoutview.
            self.layout = pya.Layout()
            self.top = self.layout.create_cell('TOP')
            self.lview.show_layout(self.layout, True)
    
            #--
            self.layers = {}
    
            #--
            ha = pya.QHostAddress("0.0.0.0")
            self.listen(ha, 8082)
            self.newConnection(self.new_connection)
            print('gdb display server initialized.')
    
    
        def new_connection(self):
            try:
                url = None
                # Get a new connection object
                connection = self.nextPendingConnection()
    
                while connection.isOpen() and connection.state() == pya.QTcpSocket.ConnectedState:
                    if connection.canReadLine():
                        line = connection.readLine().decode("utf-8")
                        if line.rstrip('\n').rstrip('\r') == "":
                            break
                        else: 
                            ts = line.split()
                            if len(ts) > 0:
                                print('Server gets', ts)
                                layerid = int(ts[0])
                                dataid = int(ts[1])
                                coords = [int(t) for t in ts[2:]]
                                self.add_shape(layerid, dataid, *coords)
                                self.redraw()
                            else:
                                print('Server received nothing')
                            break
                    else:
                        connection.waitForReadyRead(100)
    
                connection.disconnectFromHost()
                signal = pya.qt_signal("disconnected()")
                slot = pya.qt_slot("deleteLater()")
                pya.QObject.connect(connection, signal, connection, slot)
    
            except Exception as ex: 
                print("ERROR " + str(ex))
    
    
        def add_shape(self, layerid, dataid, *coords):
            if (layerid, dataid) in self.layers:
                layer = self.layers[(layerid, dataid)]
            else:
                layer = self.layout.layer(layerid, dataid)
                self.layers[(layerid, dataid)] = layer
    
            if len(coords) == 4:
                #-- rectangle
                box = pya.Box( *coords )
                self.top.shapes( layer ).insert( box )
            elif len(coords) == 8:
                box = pya.Box( coords[0], coords[1], coords[4], coords[5] )
    
        def redraw(self):
            self.lview.add_missing_layers()
            self.lview.zoom_fit()
    

    A simple client test driver:

    import socket    
    HOST = '0.0.0.0'
    PORT = 8082    
    while True:
        line = input('Input a box: :')
        ts = line.split()
        if len(ts) != 6:
            break
        s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        s.connect( (HOST, PORT) )
        line += '\n\r'
        s.send(line.encode())
        s.close()
    

    To run:
    /> klayout -rm server.py &
    /> client.py
    Input a box: 1 1 0 0 100 100
    input a box: 2 1 100 100 200 200

  • Very good. Thanks for sharing this code :)

    The key feature is Qt's event loop which polls event sources (including sockets). That is the way, background processing is done in classic UI applications. As KLayout is a Qt application you can hook into this event loop in many ways - one is the TCP connection object like you did it, but you can also user timers and poll some data source.

    Kind regards,

    Matthias

Sign In or Register to comment.