On many occasions, Sikuli scripts won't behave as expected. Hence it is important to be able to debug the scripts step-by-step to understand what have caused the incorrect behavior.
I will discuss few techniques that can be used to debug Sikuli scripts.
NOTE: If you are having troubles with scripts after transferring them to another machine, refer to this troubleshootiong guide.
I will discuss few techniques that can be used to debug Sikuli scripts.
NOTE: If you are having troubles with scripts after transferring them to another machine, refer to this troubleshootiong guide.
Highlighting
When you search for a pattern in a region that resolves into a successful Match object, it can be highlighted. For example:
Java:
Java:
@Test
public void highlightTest() throws FindFailed {
ImagePath.setBundlePath("C:/temp/");
String imageName = "calcIcon.png";
Screen s = new Screen();
Match m = s.find(imageName);
Python:
As a result, you will see a red frame displayed around the matched area, like this:
calcIcon = "calcIconSmall.png";
calcIconMatch = find(calcIcon)
calcIconMatch.highlight(1)
As a result, you will see a red frame displayed around the matched area, like this:
 |
| Image1. Highlighted area |
Practical Example:
A good visualization of how highlighting a matched object can help us is a scenario where we have multiple almost similar objects. Consider the following image:
Java:
Match object is what you get as a result of successful match. Using highlight() method on the Match object will print its details on the screen. Let's examine the Match objects of the images in Image 3.
If you have a look at the S (similarity score) parameter, you can see that one ball was matched with a 1.00 score and the other with 0.53 score which stand for 100% match and only 53% match respectively.
Without question, 53% match is way too low and we cannot allow such deviation from the original pattern. To fix that, we will set the similarity score to a higher value (I recommend at least 95%). The information about how to configure the minimal similarity score can be found here.
 |
| Image2. Different images with some resemblance |
As you can see, these balls do look alike but still clearly different. Now say you want to find all the balls of a certain one type on the screen and count them, but you keep getting a wrong result two instead of one? Let's try to apply our highlight() method and try to understand what's going on.
Java:
String football = "footballImg.png";
for (Iterator<Match> list = s.findAll(football); list.hasNext();) {
Match m = list.next();
m.highlight(1);
}
Python:
football = "footballImg.png";
for x in findAll("football.png"):
x.highlight(1)
and we get the following result:
 |
| Image 3. Different images successfully matched as a result of using low similarity score |
as you can see, both images have been picked up! So now we have identified our problem and know what is causing it. All thanks just to highlighting.
Now, in next paragraph, let's try understand why two different images have been matched to a single pattern.
Match objects and the Similarity Score
Another way that can give you a good idea of what your script is doing is the Match object itself. You can find a more detailed description of Match objects in my other post here.
Match object is what you get as a result of successful match. Using highlight() method on the Match object will print its details on the screen. Let's examine the Match objects of the images in Image 3.
highlight M[-1036,224 184x180]@S(S(0)[0,0 1920x1080]) S:1.00 C:-944,314 [-1 msec] for 5.0 secs
highlight M[-1262,255 184x180]@S(S(0)[0,0 1920x1080]) S:0.53 C:-1170,345 [-1 msec] for 5.0 secs
Without question, 53% match is way too low and we cannot allow such deviation from the original pattern. To fix that, we will set the similarity score to a higher value (I recommend at least 95%). The information about how to configure the minimal similarity score can be found here.
Comments
Post a Comment