Hello World

(For Sikuli Version 0.10)

Let's begin the tutorial with a customary Hello World example!

You will learn how to capture a screenshot of a GUI element and write a Sikuli Script to:

  1. Click on the element
  2. Type a string into the element

First we go through an example that can be used on a Mac. If you run Windows, just read on, to get a basic understanding. Further down we show you an equivalent example in a short form for Windows.

On a Mac

The goal of the Hello World script is to automatically type "Hello World" into the spotlight search box, like this:

http://script.sikuli.org/tutorial/spotlight.png

Now, open the Sikuli IDE. We begin by taking the screenshot of our target, the Spotlight symbol http://script.sikuli.org/tutorial/spotlight-symbol.png in the upper-right corner, so that we can tell Sikuli Script what to look for and click on.

We are going to use the  click function to simulate a mouse click on the given target. To tell Sikuli script how the target looks like, we have to capture its image from the screen.

Sikuli Script provides two methods to switch to the screen capture mode. The first method is to click on the camera button in the toolbar.

The second method is to press the hot-key (Command + Shift + 2 on Macs). This hot-key can be triggered even if the script editor is minimized.

In the screen capture mode, the screen will look darker and freeze momentarily. The entire desktop becomes like a canvas. We can draw a rectangle around the target, the spotlight symbol. The cross of red dotted lines shows the center of the selected rectangle, which by default becomes the click point, that will be used in the action click().

The image within the rectangle will be captured and inserted into the editor at the current cursor position.

As a convenience the IDE provides a toolbar called "Command List" on the left side, that contains the most often used functions, that have an image to search for as parameter.

Locate the click() function in the list and just click on it. If "Auto Capture" is turned on, which is the fact by default, you will directly be switched to the capture mode. Just select the spotlight symbol. You will find that the IDE as a result of your action will insert the complete function call including the image of the spotlight symbol.

http://script.sikuli.org/tutorial/ed2.png

The next action is to type "Hello World" into the search box. This is very straightforward.

http://script.sikuli.org/tutorial/ed3.png

The  type function will type the string given in the argument into whichever input control that has the focus. After clicking on the spotlight symbol, we can expect the spotlight search box will be the input that has the focus.

Congratulations! You have completed your first Sikuli Script. You can press the run button to see this script in action!

Make It Simpler

The API provided by Sikuli Script intends to simplify and minimize the code to perform GUI automation. For example, the  type function provides 2 types of interfaces: one accepts the string to type, and another one requires an additional image pattern for specifying the target to click on before typing in the string. (Actually, each type of interface can have an optional parameter "key modifiers." They are neglected here because we want to keep this tutorial simple.) In other words, the above 2-line hello world example can be reduced to the following 1 line, which has the same functions: 

type(, "hello world")

Make It More Robust

Some of you may find the hello world example does not work on your machine because of the lag of the spotlight search box. You may usually find that some programs respond very slowly. This delay may break Sikuli Script's automation, because Sikuli Script does not know how long it should wait between each action.

In the hello world example, the spotlight search box may delay to appear for several seconds. However, our script just types "hello world" anyway without waiting the search box appeared. This could cause unexpected results that another program with keyboard focus got the "hello world".

To resolve this issue, a more robust way is to make sure the appearance of the spotlight search box. Thus, we can write a  wait function to ask Sikuli Script for waiting the appearance of the search box. The wait() function can take an optional parameter timeout that specifies the maximum waiting time in milliseconds. If it is not specified, the default timeout is 3 seconds.

A robust version of hello world with wait() is illustrated as following: 

click()
wait()
type("hello world")

Consider a long script of GUI interaction, every action may need a wait() beforehand to make sure the GUI program has responded the previous action and entered the expected state. To simplify these redundant wait() functions, Sikuli Script provides a function  setAutoWaitTimeout function to let every action that needs an image target automatically wait the appearance of the target. If the auto waiting timeout is set to a non-zero value, Sikuli Script will keep looking for the image on the screen as many times as possible until the target is located or timeout. By default, Sikuli Script sets the AutoWaitTimeout to 3 seconds so that users do not need to call wait() before each action.

There is still the risk, that somehow even the default waiting time is not enough to wait for the spotlight input field to come up. In this case, the script will terminate with a FindFailed exception. You could use a special timeout value: 

...
wait(, 10) # would wait max. 10 seconds
...

Make it Faster

If you want your scripts to run faster, the best way is to whenever possible, tell Sikuli in what area of the screen it should look for an image. The smaller this region is, the faster your script will run.

With our example, we know that the spotlight symbol and its input field are by definition always located in the upper right corner of the screen. So we can tell Sikuli Script to look only in this area. To keep things simple for now, we just use the region button in the IDE to define our "Region of Interest" in the upper right corner of the screen and redirect our find operations to this far smaller area.

Normally we would have to qualify our actions with a region object, but to keep our script looking nice, we can use a new  feature "with" to form a block, where all operations are done within the given region.

The optimized script (runtime reduced to about 30% and less): 

with :
   click()
   wait()
type ("hello world")

On Windows

The goal is to type the text "hello world" into the search field, that is available after clicking on the start button (Windows Vista and Windows 7).

Basic version: 

click()
type("hello world")

Make it simpler: 

type(, "hello world")

Make it more robust: 

click()
wait()
type("hello world")

Make it Faster 

with :
   click()
   wait()
type ("hello world")

Sikuli script provides many more features to handle even complex situations in an elegant way. If you are interested, look at other Examples, Tutorials, HowTo's and especially at  The Complete Guide to Sikuli Script.

Attachments