Let a software control keyboard and mouse

Sikuli is a software controlling your keyboard/mouse devices in order to do some automatic tasks. It was initiated at MIT and is currently developed by Raimund Hocke. It works on all platforms and is available here.

In the sequel, I show an example to collect images from Google Maps and make a map from them. The aim is to draw a French island, l’île d’Oléron.

intro image

Such a software is useful when those tasks cannot be done through an API (e.g. for flash websites); or when you don’t want to learn an API (e.g.for a single work).

1 – Sikuli script. I launch Google Maps and set my position where I want to begin the map.

mapExample

Then, I use Sikuli to make print screens and to click on the arrows, using the following code (the written language is Python).

horizontal = 4
vertical = 5
for x in range(vertical):
    type(Key.PRINTSCREEN)
    for y in range(horizontal-1):     
        click(→); click(→)
        sleep(2)
        type(Key.PRINTSCREEN)
    for y in range(horizontal-1):
        click(←); click(←)        
    click(↓); click(↓); click(↓)
    sleep(2)
for x in range(vertical):
    click(↑); click(↑); click(↑)

In the code, each arrow corresponds to a ‘png’ file: → is; ← is ; ↓ is ; ↑ is .

2- Rename, crop and merge. I take all the screens and rename them with R:

my_list = list.files()
file.rename(my_list, paste0(1:length(my_list), ".png"))

Then I cut the borders with Image Magick:

mogrify -crop 768x867+75+65 *.png

Finally, I merge images with Image Magick:

montage -mode concatenate -tile 4x5 *.png output.png

3- Final output (click to enlarge, 3Mo).