Unit testing

davei

Does anyone have recommendations about how to do unit testing of my ruby classes interaction with Qt?

In general there seems to be a problem. To process an event generated in a test we need a QThread to start from the test, yet the Application is of another thread and Qt does not like doing many operations in different QThreads.
I don't see Ruby-based unit testing examples anywhere.

Can anyone direct me to information on the subject?

Thanks,
Dave

Matthias

Hi Dave,

I doubt that QThread and Ruby will work as expected, since Ruby by itself is single-threaded. Once you enter the Ruby interpreter, the global interpreter lock will ensure that only one thread is active. Plus, as you mentioned, Qt is not fully thread-safe.

You can use QApplication::processEvents to make events being processed. I tried to establish UI testing previously, but I found one issue exactly at this point: some Widgets don't immediately process events, but internally set up a timer doing that or process events later than expected. In general it's quite difficult to trigger operations through events.

In the end I found it's even more difficult to design user interface tests in a way that they don't provide a maintenance nightmare once you continue developing your user interface: whenever you change something substantially you'll have to update your tests. So finally I gave up trying to maintain automated user interface tests.

Matthias

davei

Thanks Matthias. Well, we do know that GUI unit testing is difficult. I may give up this approach and focus on tools that drive the GUI for testing e.g. Sikuli.

Matthias

Hi Dave,

let me know here if you can recommend this tool. I'm also interested in a GUI testing solution.

Thanks,

Matthias

davei

I would not recommend Sikuli yet. I work in Linux as a rule and my company is slow to adopt new versions of the OS, so I must support versions as old as RedHat 4. This means I generally have to compile, not just the app, but its supporting apps too. In the case of Sikuli this means compiling Tesseract, and other apps. This is important because Sikuli tries to match pixmaps specified in one's tests with those found on the screen. Tesseract is part of the system that identifies text characters in the pixmaps. As such it must be trained to identify each font one expects to use in testing. My training of Tesseract seems to be barely acceptable. The result is that my tests must specify a large margin of error when trying to find a matching pixmap. This means one must learn some tricks to improve tests. In my case, I frequently found my tests to fail to repeat themselves. I believe in my future tests I will restrict the tests to carefully crafted Gnome environments with the exact font and pixel dimensions of the test display controlled. Limiting these environments in this way may be unacceptable to others.

The author that helps me with Sikuli has been very nice (like you!). He admits that Linux is not his strong suit. I feel he helps as much as his time allows, but he seems spread a bit thin. The result is that installs are a bit difficult.

The API of Sikuli has been changing somewhat. I would like to be using the more recent versions where better tests can be crafted, but I cannot seem to get the more recent versions to build. The build is largely java and that's not my strong suit, so I may be the problem here.

I write in Ruby and would like to have Sikuli tests be written in Ruby. I have only been able to write them in Python - not really a terrible problem. My attempts to make Sikuli use a Ruby API failed.

With all that said, I have achieved acceptable tests on another application.

I'll try and remember to update you should my Sikuli environment change or should I get positive results using it with my current app.

Dave