Pyautogui: Common problems and fixes (Python RPA)
locateOnScreen is not working, or is returning None all the time (Mac OS/OS X)
Step 1. Verify that you're able to take a screenshot
To do this, run the code
Check the screenshot. Can you see all your windows, or just the background?
If all you can see is the background, then you need to update the "Security & Privacy" settings for the application that is running your code. This can be
Visual Studio Code, or
If you have already done this, close the application and reopen it. I had to completely logout, since my application was running as a deamon, that was the best way to make sure it was restarted.
After you've done this, test it again. It should work.
Step 2. Verify the images
This function needs the images to match perfectly, so make sure you have good screenshot of the images. Consider saving them in PNG format because of the way it compresses images without losing much information.
Step 3. Reduce the matching confidence
You can install
opencv to get access to the additional
If you haven't installed
opencv yet, you can do it by running
pip install opencv-python.
After that you can start using the confidence argument like so:
pos = pyautogui.locateOnScreen('btn.png',grayscale=True, confidence=0.9)
typewrite doesn't work for symbols or inputs wrong characters
PyAutoGUI describes itself as a "cross-platform GUI automation Python module for human beings.", Maybe instead the description should say "module for usonian human beings".
This bug with "PyAutoGui" is due to the fact that it does not know about keyboard maps, except the USA one.
The solution is simple, using the
keyboard library and the
keyboard.write function, instead of the
There are other workarounds like:
- changing the keyboard layout just for your automation (yuck 🤮);
- storing the string in the clipboard and copy-pasting (yuck 🤮);
- storing a character map and replacing the characters on each string you want to send (yuck 🤮).
pyautogui supports thirdparty libraries like
opencv for image matching, so it could do the same for keyboard input. Hopefuly this will happen soon.
typewrite doesn't work with unicode strings
This is a very similar problem as the one above with probably the same root cause. Simply use one of the workarounds described above, or just replace the
pyautogui.typewrite function with the
keyboard.write function, and that's it! Everything should work nicely.
ImportError: No module named pyautogui
Managing python environments can be a pain in the ass. Yes, you can use virtual environments and install all the libraries necessary every-single-damn-time, but that's boring AF and feels like too much when you just want to test something fast.
Typically, you would install
pyautogui with a simple
pip3 install pyautogui, however this doesn't always work when outside a virtual environment (looking at you, Anaconda.).
Instead, try the command
python3 -m pip install pyautogui, this gives you much better chances that the library is going to be installed in the same place that your
python3 alias is looking at.
Can't hold key down for X seconds
If you're automating something that requires you to hold a key down for sometime, like the movement of a game character, you'll realize that
pyautogui.keyDown doesn't cause key repeats.
The best way to do this would be by using the
hold() context manager, like so:
with pyautogui.hold(key): pyautogui.sleep(hold)
Mouse movements don't count as activity
Sometimes, you want to move the mouse using
pyautogui to simulate activity (preventing screenlocks, showing as active on a messenger,
Mouse movements aren't always counter for activity. Instead, consider using the keyboard events. Something like pressing the volume up and volume down keys can work wonders:
import pyautogui import time while True: pyautogui.press('volumedown') time.sleep(0.5) pyautogui.press('volumeup') time.sleep(60)
Pyautogui is not working in a particular application
Pyautogui can easily be identified by specific software, like games and anti-cheating software.
A lot of times, pyautogui isn't trying to hide. For example, the library uses Window's API and adds the
LLMHF_INJECTED flag to its events.
There are at least ways around this:
- Write your own driver (yuck 🤮);
- Use a library that simulates direct input (like PyDirectInput - for windows only)
- Run it in a VMWare machine;
I don't like any of these to be honest. It would be better if pyautogui had a flag that would allow you to either flag your input or not.
Note: If you need something really untraceable, you can always use an Arduino to simulate key strokes as an actualy keyboard and mouse.
The Pillow package is required to use this function.
The solution is simple, you install the
However, it should have been installed automatically, when you installed
pyautogui as it is a dependency of a dependency ;)
So I would recommend simply installing everything again with
pip install --upgrade --force-reinstall pyautogui
Work with me
Connect with me through LinkedIn, or through my company Enlightenment.ai.