Setting the progress_check() virtual function from RBA

davei

During the read of large (many GB files) I would like my RBA-based
app to be able to update the screen. The ability exists, for instance, in the C++ to do this in the GDS2Reader class where the virtual function progress_check() is declared. I would like to do this for read and write of GDS2 and Oasis files.

Is there an RBA way to do this?

In lieu of that, could I do it without recompiling KLayout in
some C++ of my own?

Thanks,
Dave

davidnhutch

Are you looking for these?

http://klayout.de/doc/code/class_AbsoluteProgress.html

http://klayout.de/doc/code/class_Progress.html

http://klayout.de/doc/code/class_RelativeProgress.html

There are some examples on those pages

David

davei

These look related. The GDS2Reader class calls a function
progress_check() every time it reads a BGNSTR (begin structure)
in a GDS2 file. In C++ I would define this progress_check()
function to see if my application issued a cancel where I would
raise an exception, otherwise it would update a status output. In my case since I have no idea of how much (percentage-wise) of a GDS2 file has been read (just that it is still reading), I want the status output to be something that blinks/rotates or otherwise indicates the program is still running, but a bar graph is not quite right.

I don't know how to make these progress classes work such that an instance of them would be called at each BGNSTR.

It was kind of you to look into it for me.

Thank you,
Dave

Matthias

Hi all,

@David: thanks for taking care of this and pointing to these goodies :-)

But I wonder why there is no progress shown in your case. A progress reporter is already embedded inside the GDS reader.

This simple script:

lay = RBA::Layout::new()
lay.read("some_big_file.gds")

will show a progress when run from the command line ("klayout -r this_script.rb") or when run from the menu. This is because the main window features a "ProgressReporter" which will connect to the "Progress" object inside the GDS reader and show the current progress.

However:

• The progress is not shown if the macro IDE is open since it will block events to the main window. If you run your script outside the IDE, the progress should be shown. Maybe this is confusing.
• The progress is always shown in the main window, even if your script opens it's own dialog.

Regards,

Matthias

davei

Mattias,

My app does not show the main window. It did not occur to me to look there - silly of me. Though I encourage KLayout use (of course with the main window), this app does not show it.

As I alluded to in my prior remarks I need both an indication of the progress of the read and the ability to cancel an oasis or gds read mid-stream. These reads can be quit long. I also need the app to regularly update the dialog so that it does not appear dead as windows no longer cover it or the window stacking order changes.

Is there a way to do this with my own C++ or RBA Ruby code. In other words can I somehow devise and insert the code called by progress_check? The RBA Progress class looks like a base class, but it's not clear I could alter a Progress sub-class to meet the needs I mentioned above.

Thanks,
Dave

Matthias

Hi Dave,

unfortunately there is no way to have this without the main window currently.

The progress classes are just reporting progress. Display of the progress is the other end of the progress object and that end is not accessible through scripts currently. It's being displayed centrally in the main window right now. I can extend this feature so the progress will be shown in a separate window when the main window is not visible.

This is what you intend to have, right?

Matthias

davei

Matthias,

Your willingness to address this is MUCH appreciated.

If it is possible to register a Ruby method to be called by progress_check(), it would be ideal. You see, I need the ability to:

1. Update any dialogs (simply giving Qt a chance to update.)
2. Check to see if a user has requested the task be stopped/interrupted.
3. Produce graphical output of the progress made.

I can do all of this with by supplying a Ruby method for progress_check() to call.

The method for registering the method for progress_check() to call might have a bit-mask argument or a set of symbols to specify which progress checks my code cares about. You might have a better idea of other arguments it might need. I think this would be implemented as an 'observer' design pattern.

The method I would supply would have an argument indicating what progress check on the C++-side was hit. An additional argument might be a hash
to include other information. I haven't thought
this through enough to know what might be in the hash. If my method was understood to return a value indicating to interrupt, say, a GDS read, then it would make a clean way to stop reading.

I envision a progress_check() working through a collection of observers and calling each one. When no observers are present, it does what it does now. Observers could supply C++ or Ruby methods.

Of course it would be my responsibility to make sure my called methods ran very quickly so that
the flow of, say, a GDS read would appear normal.

If your code runs multiple threads I would have to know this so I could use mutexes as needed.

I would love to hear what you think about this.
Oh, if I can help I will certainly do so.

Thanks so much,
Dave

Matthias

Hi Dave,

I could basically open the internal API for receiving progress for Ruby too. The responsible C++ class is "tl::ProgressAdaptor". This is equivalent to what you call "progress_check". There is no elaborate receiver chain yet like the one you depicted, but this can be added.

But the ProgressAdaptor would be a major change and won't go into a minor release. There are a couple of problems to solve. For example, the progress events can basically arise from outside the main thread as you mentioned. This will cause problems in the Python or Ruby interpreter itself and cannot be mitigated by mutexes within the scripts. And performance wise, Ruby or Python will never come close to C++. So a noticeable performance penalty will be there.

The pop-up proposal is a low hanging fruit and will provide the three abilities you mentioned. It will deliver paint events - also to your dialog - presents a Cancel button for stopping the operation and provides a progress bar and text for graphical output.

Matthias

davei

Matthias,

I understand your points - especially the threading problem.
It would be very nice to have your pop-up proposal solution.
If you make this change I hope you can let me know and I'll
put it right to use.

Perhaps later on the general observer pattern implementation would work. Your worry about Ruby and Python performance is warranted and could be a problem. Perhaps a parameter during the registration of the method could specify a timeout value.
This way the progress check would call the registered method only when enough time had passed and mitigate the problem.
A timeout of 0 could turn the time check off and every progress check would run the registered method.

I still see it as my responsibility to make my registered method be quick.

You have been very helpful and if I can return the favor I would like to try.

Thanks,
Dave

Matthias

Hi Dave,

yes, I'll add the popup solution.

Just for clarification: you run your script with "-z", do you? If not, the progress is still shown in the main window.

Matthias

davei

Yes, I use the -z flag upon starting my script.
Thanks,
Dave

davei

Matthias,

I have not been watching the source code, but have you had a chance to implement the popup solution mentioned here? If not, do you have an idea when you might be able to do it? I know I'm doing some begging here, but it would help me plan just to have a rough idea of when it might be done.

Thanks,
Dave

Matthias

Hi Dave,

The solution should come with 0.24.7. This version shows a popup progress bar when you use a script with -z and perform some long running operation.

It should work, but to be frank, I am not happy with it. Now, progress bars pop up, even if the script does not do anything with the UI.

I plan to change is slightly this way: when there is a script that opens a UI (some top level window), the progress bar will pop up and be associated with that window. Otherwise, the progress is not shown.

Please tell me what you think.

Thanks,

Matthias

davei

Matthias,

I'm not sure I can safely know what is affected by the changes. Here's what I know.
My application is started this way:

klayout -z -rx -rm #{loader} -rd program=#{my_program}

The 'loader' above creates Qt interfaces from .ui files. My app looks like a QDialog
instance that incorporates all my Qt interfaces. It has a QTimer triggered event queue that
works but is not ideal. It has an instance of an RBA::Layout object that is used to read GDS/Oasis files. This is the slowest portion/event of my code. It requires some feedback to the user, telling them the job is still running and allowing them to interrupt the read as needed. While the GDS/Oasis read is the slowest it is probably not the only place that needs feedback/interrupt capabilities. The program reads Gigabytes of data, so it is large.

Where would the progress bar be located? Would I supply it a screen location or a UI object to parent it?

For me, the best solution is to have KLayout have a default behavior that makes the layout editing the best, BUT also has way for me to introduce my own progress_check method. This, I hope, makes any problems mine. I can work up a rough draft of a Ruby API a user might want if it helps, and we could see what happens from there. If you think this is helpful, tell me and I'll do it.

I suspect you are having to think through a lot that I am not aware of. Forgive me if my remarks aren't helping.

Thanks,
Dave

Matthias

Hi Dave,

as discussed before, opening the progress reporting API to Ruby code has severe implications. Which is why I am not considering this option right now.

But I think that the proposed solution is helpful in your case - if any user interface is present, a progress bar will be shown as a widget with a "Cancel" button. It will be shown centered to the active main window which is the main QDialog in your case.

I'll release some 0.24.8 version soon so you can take a look at this new proposal.

Matthias