Wait for appearance Version 9 (Python)
Action group: Vision
Description
The action is intended to wait for an object matching the set pattern to appear and get its coordinates. If the transmitted image matches the image(s) on the screen, the coordinates of all found matches are returned
Action icon
Parameters
Input parameters
- Image - the path to the image for recognition. The most effective use of files with the extension «.png». Valid extensions: «.jpeg», «.jpg», «.bmp», «.png», «.tif», «.tiff». If the algorithm did not find the file extension or the file type is not applicable, a warning will be displayed.
- Similarity rate - required coefficient of similarity between the template image and the target area. The maximum value is «1».
- Timeout (ms) - the time during which the appearance of the specified object will be expected. Timeout is set in milliseconds.
- Give center - the action will return the coordinates of the centers of the images found on the screen. By default, the coordinates of the upper left corner of the image are returned.
- Interval of taking screenshots - the interval of taking a screenshot for comparison with the specified image. Measured in seconds. Acceptable values: integers from 1 and more.
- Comparison algorithm - an algorithm for comparing the specified image with the image on the screen. If one algorithm does not work correctly, you should try another one. Different image comparison algorithms will work more correctly in different cases. The following image comparison algorithms (TemplateMatchModes) are used:
- TM_SQDIFF - sum of squares of the color differences.
- TM_CCORR - sum of elemental productions.
- TM_CCOEFF (by default) - cross-correlation between images with no middle (Pearson's correlation).
- Direct Match - match pixels to the specified image. Make the image black and white. If true, the image color will be changed to grayscale.
- Search for image edges - if the value is «true», then the search for edges of objects in the image will be performed. Edges are the boundaries that outline the image object.
- Min edge search boundary - the minimum size of the edges of the image objects. Together with the value from the «Max. edge search boundary» field, it makes up the range of possible edge sizes. It is taken into account only if the «Search for image edges» parameter is true. Acceptable value: from 0 to 255. Default: 150.
- Max edge search boundary - the maximum size of the edges of the image objects.Together with the value from the «Min. edge search boundary» field, it makes up the range of possible edge sizes. It is taken into account only if the «Search for image edges» parameter is true. Acceptable value: from 0 to 255. Default: 200.
- Min image size change - the minimum zoom factor of the specified image. This parameter is used if the size of the image on the screen may differ from the size of the specified image. Recommended value: from 0.5. Default: 1.
- Max image size change - the maximum zoom factor of the specified image. It is recommended to use the parameter if the size of the image on the screen may differ from the size of the specified image. Recommended value: up to 5. By default: 1
- In cases where the algorithm has not found the file extension or the file type is not applicable, the robot will exit with an error.
The interval for taking a screenshot is adjusted depending on the waiting time for the element to appear. You can rely on the logic of the business process, how quickly objects can change on the screen. - To choose a suitable comparison algorithm, you need to run an action with different algorithms on your data several times and choose the algorithm that works best for this data. Due to the fact that images can be completely different (light, dark, geometry, abstract, text), the method may be suitable for images of a similar type.
Images are converted to grayscale to simplify calculations when comparing. We need to test it with our own examples. If it gives a good result on specific input data, then use. It is better to look for logos and application icons in color, and text in black.
- The "Search for image edges" algorithm works well when looking for rectangular objects.
- It is recommended to use "Minimum/maximum image resizing" if the size of the image on the screen may differ from the size of the desired image. The algorithm takes the interval from "Min. image size change" to "Max. image size change" and runs the image comparison algorithm, gradually enlarging the desired image. This increases the running time of the algorithm, but it can be useful if the user is not sure that the desired image will appear on the screen in the same size.
- Reasons why an action may not find the desired image:
- If the object in the template image is rotated compared to the target object in the large image (or vice versa), standard template matching methods will not be able to detect the object.
- Differences in lighting can affect pixel values, making it difficult to match patterns for reliable object detection.
- If the object is partially obscured or if there is a lot of interference in the large image (several objects, noise). The algorithm searches for an exact match and cannot process the hidden parts of the object.
- When comparing patterns, it is assumed that the object of interest always has a certain shape. This makes it difficult to work when objects are deformed, have different looks, or consist of different textures (for example, people's faces or bodies).
Output parameters
Coordinates - a variable that will contain the coordinates of all found objects. The order in which the result collection is populated: first comes the object found closest to the top-left corner of the search area. If the object did not appear within the specified period of time, the error «Image not found» will be displayed.
Settings
| Property | Description | Type | Filling example | Mandatory field |
|---|
| Parameters |
|---|
| Image | The path to the image for recognition. The most effective use of files with the extension «.png». Valid extensions: «.jpeg», «.jpg», «.bmp», «.png», «.tif», «.tiff». If the algorithm did not find the file extension or the file type is not applicable, a warning will be displayed. | Robin.Image | C:\doc\img.png | Yes |
| Similarity rate | Required coefficient of similarity between the template image and the target area. The maximum value is «1» | Robin.Numeric |
| Yes |
| Give center | If the parameter is «false», the action will return coordinates of the upper left corner of the image found on the screen, otherwise it returns coordinates of the center of the found image | Robin.Boolean | true | No |
| Timeout (ms) | The time during which the appearance of the specified object will be expected. Timeout is set in milliseconds | Robin.Numeric |
| Yes |
| Interval of taking screenshots | The interval of taking a screenshot for comparison with the specified image. Measured in seconds. Acceptable values: integers from 1 and more. | Robin.Numeric |
| No |
| Comparison algorithm | The algorithm for comparing the specified image. If one algorithm does not work correctly, you should try another one. The default value is TM_CCOEFF. | Robin.String |
| No |
| Make the image black and white | If the value is «true», then the image color will be changed to grayscale. | Robin.Boolean | true | No |
| Search for image edges | If the value is «true», then the search for edges of objects in the image will be performed. Edges are the boundaries that outline the image object. | Robin.Boolean | true | No |
| Min edge search boundary | The minimum size of the edges of the image objects. Together with the value from the «Max. edge search boundary» field, it makes up the range of possible edge sizes. It is taken into account only if the «Search for image edges» parameter is true. Acceptable value: from 0 to 255. Default: 150. | Robin.Numeric |
| No |
| Max edge search boundary | The maximum size of the edges of the image objects.Together with the value from the «Min. edge search boundary» field, it makes up the range of possible edge sizes. It is taken into account only if the «Search for image edges» parameter is true. Acceptable value: from 0 to 255. Default: 200. | Robin.Numeric |
| No |
| Min image size change | The minimum zoom factor of the specified image. This parameter is used if the size of the image on the screen may differ from the size of the specified image. Recommended value: from 0.5. Default: 1. | Robin.Numeric |
| No |
| Max image size change | The maximum zoom factor of the specified image. It is recommended to use the parameter if the size of the image on the screen may differ from the size of the specified image. Recommended value: up to 5. By default: 1. | Robin.Numeric |
| No |
| Results |
|---|
| Coordinates | A variable that will contain the coordinates of all found objects. The order in which the result collection is populated: first comes the object found closest to the top-left corner of the search area. If the object did not appear within the specified period of time, the error «Image not found» will be displayed. | Robin.Collection |
|
|
Special conditions of use
- The resolution of the search image and the screen resolution must match in order for the action to work correctly. For this purpose it is recommended to make a screenshot of the image. The most effective way to use files with ".png" extension.
- The screenshot interval does not take into account the image processing time. The action will take a screenshot at the specified interval. The interval is measured in seconds. The minimum is 1 second.
- The value of "Min image size change" must not exceed the value of "Max image size change". Otherwise, the action will generate an error.
- If the "Min edge search boundary" parameter is filled in and the "Max edge search boundary" parameter is not filled in (and vice versa), the action will take the values that these parameters have by default.
- If the "Min image size change" parameter is filled in and the "Max image size change" parameter is not filled in (and vice versa), the action will take the default values of these parameters.
Example of use
Task
Return the coordinates of the found images on the screen corresponding to the specified image when setting different parameter values according to the settings table.
Solution
Use the "Wait for appearance" action.
Implementation
- Add the "Wait for appearance" action to the test robot - 6 to the workspace we will be working with .
- Click on the icon on the right side of the screen and a sidebar with parameter settings will appear.
- Sequentially set the values of the parameters.
In the "image" field, set the path where the image to be recognized is located.
Consistently set the parameters that affect the similarity rate of the template of the image that appears.
Task 1
Return the coordinates of found images on the screen corresponding to the given image when setting the minimum settings of parameter values according to the setup table.
Run the robot.
The program robot completed successfully. The minimum set of configuration parameters is correct.
Task 2
Return the coordinates of the found images on the screen corresponding to the given image when you make advanced settings of parameter values according to the setting table.
Run the robot.
Task 3
Return the coordinates of found images on the screen corresponding to the specified image when setting individual incorrect advanced settings for parameter values other than the setting table.
When specifying an interval for taking a screenshot to compare with the specified image. Specified a null value. Valid values: integers from 1 and more.
The robot generated an error.
Task 4
Return the coordinates of found images on the screen corresponding to the specified image when setting individual incorrect advanced settings to parameter values other than the setting table.
When specifying the minimum coefficient of change of scale of the specified image, the value "-1" was set. Recommended value: from 0.5. Default: 1
The robot generated an error.
