Autosave implantantion by python script?

Ernest

Hi everyone:
I'm working on a project that requires heavy loading on simulation. The Klayout is working on the same PC that runs the simulation tool.
Sadly, the PC sometimes froze due to the simulation program's instability, and I had to force restart the PC. If I didn't save the layout before the simulation, I would lose the layout. Sometimes I remember to save the layout, and sometimes I forget. So I want to implant the autosave ability with a python script.
Here is my code after surveying community :

import pya
import threading
import time

counter = 0


class Timer:
    def __init__(self):
        self.start = time.time()
        self.period = 0.1  # period of update in seconds
        self.last_update = self.start - self.period
        self.current = None

    def update(self):
        self.current = time.time()
        if self.current > self.last_update + self.period:
            pya.QCoreApplication.processEvents()
            self.last_update = self.current


class RepeatedTimer(object):
    def __init__(self, interval, function, *args, **kwargs):
        self._timer = None
        self.interval = interval
        self.function = function
        self.args = args
        self.kwargs = kwargs
        self.is_running = False
        self.next_call = time.time()
        self.start()

    def _run(self):
        self.is_running = False
        self.start()
        self.function(*self.args, **self.kwargs)

    def start(self):
        if not self.is_running:
            self.next_call += self.interval
            self._timer = threading.Timer(self.next_call - time.time(), self._run)
            self._timer.start()
            self.is_running = True

    def stop(self):
        self._timer.cancel()
        self.is_running = False


def auto_save():
    # Save the layout prior to exporting, if there are changes.
    mw = pya.Application.instance().main_window()
    global counter
    counter += 1
    print(f"{counter} autosave check.")
    if mw.manager().has_undo():
        mw.cm_save()
    layout_filename = mw.current_view().active_cellview().filename()

    print(f"Autosave to {layout_filename}.")
    if len(layout_filename) == 0:
        raise Exception("Please save your layout.")


def main():
    # autosave every 10 min
    rt = RepeatedTimer(600, auto_save)  # it auto-starts, no need of rt.start()
    timer = Timer()
    try:
        while True:
            timer.update()
    finally:
        rt.stop()


main()

The thing is if I run the script from the startup, it blocks every other script I made to run from the startup, and the macro development window is disabled. Besides, it is very CPU-consuming. Any suggestions?

BR
Ernest

Matthias

@Ernest you cannot let a script run infinitely in the background. The Qt UI main thread is blocked while you run scripts and that has severe consequences.

The way to do that is a QTimer object that delivers regular events out of the main thread's event loop.

The basic way to use a timer is this (taken from https://www.klayout.de/forum/discussion/1688):

class MyTimer < RBA::QObject

  def initialize
    @tick = 1
    @timer = RBA::QTimer::new(self)
    @timer.setInterval(500)
    @timer.start
    @timer.timeout { self.on_timeout }
  end

  def finish
    @timer._destroy
  end

  def on_timeout
     puts "Another tick: #" + @tick.to_s + " ..."
     @tick += 1
  end

end

my_timer = MyTimer::new

But honestly: there are other solutions for stopping another process from interfering with your work - such as going into a cloud with the simulation tool, using ulimits if you're on Linux etc. You cannot patch every working tool you're using to support autosave. Plus IMHO autosave is ugly. It overwrites your file and imagine if the freeze happens during autosave - in that case even your file is gone.

Matthias

Ernest

import pya
import os


class TimerAutoSave(pya.QObject):

    def __init__(self, parent):
        super(TimerAutoSave, self).__init__(parent)
        self._counter = 0
        self._timer = pya.QTimer(self)
        # autosave every 10 min
        self._timer.setInterval(10 * 60 * 1000)  # millisecond to minutes
        self._timer.timeout = self._timer_timeout
        self._timer.start()

    def _timer_timeout(self):
        self._counter += 1
        self._timer_auto_save()

    def _timer_auto_save(self):
        mw = pya.Application.instance().main_window()
        print(f"{self._counter} times autosave check.")
        ori_layout_filename = mw.current_view().active_cellview().filename()
        layout = mw.current_view().active_cellview().layout()
        if mw.manager().has_undo():
            if len(ori_layout_filename) == 0:
                layout.write("BackUp.gds")
                msgBox = pya.QMessageBox()
                msgBox.setText("Please save your layout.")
                msgBox.exec_()
            else:
                filename = change_filename(ori_layout_filename)
                layout.write(filename)
                print(f"Backup file autosave to {filename}.")

    def finish(self):
        self._timer.stop()


def change_filename(filename):
    file_path = os.path.splitext(filename)[0]
    file_ext = os.path.splitext(filename)[1]
    changed_filename = f"{file_path}#BackUp#{file_ext}"
    return changed_filename


def main():
    main_window = pya.Application.instance().main_window()
    autosave_timer = TimerAutoSave(main_window)


main()

Matthias

@Ernest Thanks for the code ... I took the liberty to format it properly.

Regarding the default location ("BackUp.gds") I'd propose to use a full path. Otherwise the file gets written to the current directory which is not well defined.

Matthias

crizos

Is there any other way to understand that changes exist other than manager().has_undo() after saving? cm_save does not change the undo state, and neither can turn off the action manually. Can we reset the undo state after saving, or do we have something else to look for?

Matthias

Hi @crizos,

LayoutView#clear_transactions should clear the undo buffer.

Matthias

crizos

Thank you Matthias, it worked.

For the first part, is there any other way to understand that changes exist other than manager().has_undo() after saving? Not only by code but also with File->Save or Ctrl+S. For example, I see a [+] in the tab title; is there any way to check for that? Or a diff way that what I am seeing is the original GDS file I opened?

Matthias

Well, yes. Individual cell views can be checked using CellView#is_dirty? (https://www.klayout.de/doc-qt5/code/class_CellView.html#method33). This flag falls back to false if the layout is saved from the user interface or programmatically via LayoutView#save_as or cm_save.

Matthias

crizos

Thank you Matthias, that is what I was looking for.

Chris

crizos

I have a follow-up question on this one:

Let's say we reach state A, we are saving in that state (by any means), and we do some actions B and C, which are first required to be done in a saved environment but not save the outcome of these actions.

We then revert back to state A by code (mainly adding and removing text objects are these actions, to be exact). There is no difference from the starting state A, but is_dirty() naturally has turned to true.

Is there a way to manipulate this flag by code; when we revert the actions, also to reset the state of that flag?

Chris

Matthias

Hi @crizos,

No, there is no way to directly manipulate "is_dirty".

If you have saved the layout in state A, you can basically revert back to state A by reloading the layout by using "LayoutView#reload", then the dirty flag should be cleared.

But I wonder what your application is. If you need to create temporary objects, e.g. for net naming, maybe it is easier to create a copy of the Layout object and then add the texts (provided your layout isn't too large). That does not spoil your loaded layout state.

Matthias

crizos

Hi Matthias,

I tried reloading, but I usually edit large files and I'm trying to avoid reopening/reloading which is the reason for reverting/deleting staff. Can you give an example of your suggestion for copying the layout? Would e.g. the text labels be visible to the viewer of the copied object? Can I keep the file in memory, and display only the copied version, and reset it when I am done?

Edit: I believe you suggesting keeping a backup of the layout in state A, doing everything that has to be done from now on and then doing something like this at the end, e.g.:

# reaching state A

layout_bck = pya.Cellview().active().layout().dup()

# irrelevant actions here

# getting back to state A
pya.Application.instance().main_window().current_view().show_layout(layout_bck, False)

Please let me know your thoughts,
Chris

Matthias

Hi Chris,

I meant I wonder why you need to manipulate a live layout. That is like if Word was changing your text to show you spelling errors (contrived example maybe). I though that in order to highlight some error regions for example, you could display a copy and then delete it once the user has reviewed the errors.

Can you maybe explain you use case? I think I am able to make some suggestions if I understand the actual problem you are trying to solve.

Regards,

Matthias

crizos

Hi Matthias,

Actually, you helped me with your last suggestion to copy the Layout object and reset that one when I have to revert to a checkpoint. I was misguided that I must use the CellView somehow since that has the is_dirty flag. Your directions have helped me a lot. The example I attached seems to fit in what I am trying to achieve.

Thank you again,
Chris