[iOS] CrashMonkey - Automatic Testing with Random Operations

I found “CrashMonkey” to be very useful for iOS app automatic testing with random operations, so I’ll introduce it.

Installing CrashMonkey

Install the crash_monkey gem.

% gem install crash_monkey

How to Run CrashMonkey

Simply specify the app file name with the crash_monkey command to automatically run tests on the iOS Simulator.

% crash_monkey -a MyApp.app

For other options, please check github.

Solution When CrashMonkey Can't Run Due to Errors

Initially, CrashMonkey couldn’t run with the following error:

% crash_monkey -a MyApp.app
{:app_path=>"MyApp.app", :run_count=>2, :time_limit_sec=>100}
=================================== Start Test (1/2) =======================================
2 apps are found, USE NEWEST APP: /Users/your_username/Library/Developer/Xcode/DerivedData/MyApp-dmjxyavvpzcngmeyaoznanghixim/Build/Products/Debug-iphonesimulator/MyApp.app
ls: /Users/your_username/Library/Logs/DiagnosticReports/MyApp_*.crash: No such file or directory
Run: ["instruments", "-l", "100000", "-t", "/Applications/Xcode.app/Contents/Applications/Instruments.app/Contents/PlugIns/AutomationInstrument.bundle/Contents/Resources/Automation.tracetemplate", "/Users/your_username/Library/Developer/Xcode/DerivedData/MyApp-dmjxyavvpzcngmeyaoznanghixim/Build/Products/Debug-iphonesimulator/MyApp.app", "-e", "UIASCRIPT", "/Users/your_username/git/myapp-ios/crash_monkey_result/UIAutoMonkey.js", "-e", "UIARESULTSPATH", "/Users/your_username/git/myapp-ios/crash_monkey_result"]
xcode-select: Error: No Xcode folder is set. Run xcode-select -switch  to set the path to the Xcode folder.
Error: /usr/bin/xcode-select returned unexpected error.
tail finished: system.log
ls: /Users/your_username/Library/Logs/DiagnosticReports/MyApp_*.crash: No such file or directory
No matching processes belonging to you were found
=================================== Start Test (2/2) =======================================
ls: /Users/your_username/Library/Logs/DiagnosticReports/MyApp_*.crash: No such file or directory
Run: ["instruments", "-l", "100000", "-t", "/Applications/Xcode.app/Contents/Applications/Instruments.app/Contents/PlugIns/AutomationInstrument.bundle/Contents/Resources/Automation.tracetemplate", "/Users/your_username/Library/Developer/Xcode/DerivedData/MyApp-dmjxyavvpzcngmeyaoznanghixim/Build/Products/Debug-iphonesimulator/MyApp.app", "-e", "UIASCRIPT", "/Users/your_username/git/myapp-ios/crash_monkey_result/UIAutoMonkey.js", "-e", "UIARESULTSPATH", "/Users/your_username/git/myapp-ios/crash_monkey_result"]
xcode-select: Error: No Xcode folder is set. Run xcode-select -switch  to set the path to the Xcode folder.
Error: /usr/bin/xcode-select returned unexpected error.
tail finished: system.log
ls: /Users/your_username/Library/Logs/DiagnosticReports/MyApp_*.crash: No such file or directory
No matching processes belonging to you were found
EXIT 0

By specifying the correct Xcode installation path with the xcode-select command, it worked normally. If you get a similar error, try running the following command:

% sudo xcode-select -switch /Applications/Xcode.app/Contents/Developer

Summary

Since it can test automatically, I think it’s good to run CrashMonkey a few times before release for verification.

Reference Information

xcode-selectコマンドでXcodeのインストールディレクトリを変更する - Kazzzの日記

That’s all from the Gemba.