feat(ios): Migrate from tidevice to pymobiledevice3 for iOS 17+ support#717
Open
abhishekbedi1432 wants to merge 11 commits intomicrosoft:mainfrom
Conversation
## Summary Replace tidevice library with pymobiledevice3 to support iOS 17+ devices while maintaining backward compatibility with older iOS versions. ## Problem - tidevice is incompatible with iOS 17+ (uses deprecated DeveloperDiskImage) - Screenshots and device operations fail on modern iOS devices - tidevice development has stalled since 2021 ## Solution Migrate all iOS device management commands to pymobiledevice3: - Device discovery: usbmux list - Device info: lockdown info - App management: apps install/uninstall/list - Screenshots: developer dvt screenshot - Log collection: syslog live, crash pull - Port forwarding: usbmux forward ## Backward Compatibility - JSON parsing supports both tidevice and pymobiledevice3 field names - Device unlock failures are now non-fatal for XCTest execution - Device watcher uses polling mechanism (usbmux watch not available) ## Files Modified - IOSUtils.java: Updated all command invocations - IOSDeviceDriver.java: Updated parsing and capability requirements - EnvCapability.java: Added pymobiledevice3 capability keyword ## Testing - Verified on iPhone 11 Pro (iOS 26.2) - All core device operations functional Ref: TIDEVICE_TO_PYMOBILEDEVICE3_MIGRATION.md for detailed command mapping Co-Authored-By: Warp <agent@warp.dev>
## Problem iOS screen recording on Mac was failing with 0-byte video files. The root cause was a race condition between port forwarding setup and ffmpeg connection. ### Error Symptoms 1. ffmpeg reported 'Connection refused' when connecting to MJPEG port 2. Video files were created but remained 0 bytes 3. Appium's built-in recording failed with 'ffmpeg died unexpectedly' ### Root Cause Analysis 1. **Timing Issue**: getMjpegServerPortByUdid() started pymobiledevice3 port forwarding as a background process, but returned immediately without waiting for the port to become available. ffmpeg then tried to connect before the forwarding was ready, causing 'Connection refused' errors. 2. **Appium Conflict**: When mjpegServerPort capability was passed to Appium, it tried to set up its OWN port forwarding, conflicting with our pymobiledevice3 forwarding. Error: 'Cannot ensure MJPEG broadcast functionality by forwarding the local port XXXX' 3. **Mac Recording Approach**: The original IOSAppiumScreenRecorderForMac relied on Appium's iosDriver.startRecordingScreen() which internally uses ffmpeg + MJPEG. This had compatibility issues with pymobiledevice3. ## Solution ### 1. Added Port Readiness Wait (IOSUtils.java) - New method: waitForPortToBeListening(port, timeoutMs, logger) - Polls using lsof every 500ms until port is listening (max 10 seconds) - Called after starting pymobiledevice3 forward process - Ensures port is ready before returning to caller ### 2. Switched Mac Recording to ffmpeg-based (IOSAppiumScreenRecorderForMac.java) - Changed from Appium's built-in recording to direct ffmpeg approach - Uses pymobiledevice3 port forwarding (same as Windows implementation) - ffmpeg command: ffmpeg -f mjpeg -reconnect 1 -i http://127.0.0.1:PORT ... - Graceful shutdown with SIGINT for proper video finalization ### 3. Removed mjpegServerPort Capability (AppiumServerManager.java) - Removed mjpegServerPort from Appium driver capabilities - Screen recorder now handles its own port forwarding independently - Eliminates conflict between Appium and pymobiledevice3 forwarding ## Files Changed - common/.../util/IOSUtils.java - Added waitForPortToBeListening() method - Modified getMjpegServerPortByUdid() to wait for port readiness - common/.../screen/IOSAppiumScreenRecorderForMac.java - Rewrote to use ffmpeg-based recording with pymobiledevice3 - Added graceful ffmpeg shutdown with SIGINT - Added MJPEG port cleanup on recording stop - common/.../management/AppiumServerManager.java - Removed mjpegServerPort capability from iOS driver creation - Added comment explaining screen recorder handles forwarding - docs/iOS-Testing-Guide.md - Added 'Onboarding a New iOS Device' section - Added 'Screen Recording' documentation - Updated version history with v1.1.0 changes ## Testing - Verified video recording produces valid MP4 files (1.2MB+) - Confirmed MJPEG port becomes active within 500-600ms - No 'Connection refused' errors in agent logs - Video playback confirmed working Co-Authored-By: Warp <agent@warp.dev>
…ible video recording ## Problem iOS 17+ broke two key flows in the pymobiledevice3 migration: 1. **WDA Launch**: `pymobiledevice3 developer dvt launch` crashes WDA on iOS 17+ with SIGABRT in XCTRunnerDaemonSession because it doesn't create a proper XCUITest session. The original tidevice used `xctest` which communicates via the device's test manager daemon over usbmux. 2. **Video Recording**: ffmpeg output had non-standard sample_aspect_ratio (281:1218) and yuvj420p pixel format, causing QuickTime Player to reject the recorded MP4 files. ## Changes ### IOSUtils.java - Add `isIOS17OrAbove()` helper for version-based branching - Add `getWdaProjectPath()` to locate WebDriverAgent.xcodeproj from Appium - **proxyWDA()**: iOS < 17 keeps `dvt launch` (verified working); iOS 17+ uses `xcodebuild test-without-building` to start WDA with proper XCUITest session that keeps the HTTP server alive on port 8100 - **killProxyWDA()**: Updated to kill both `dvt launch` and `xcodebuild` WDA processes ### IOSAppiumScreenRecorderForMac.java - iOS < 17: keeps existing verified `scale=720:360 -vcodec h264` command - iOS 17+: uses `scale=720:-2,setsar=1 -pix_fmt yuv420p -vcodec libx264` which auto-calculates height preserving aspect ratio, sets square pixels, and uses standard yuv420p color space for QuickTime compatibility ## Known Issue On iOS 17+, proxyWDA() starts WDA via xcodebuild which conflicts with XCTestRunner's xcodebuild session (two test sessions on same device). Tests complete but xcodebuild hangs at cleanup. Next step: migrate WDA launch to `pymobiledevice3 developer dvt xcuitest` (the pymobiledevice3 equivalent of tidevice xctest) which uses the instrument protocol over usbmux and won't conflict with xcodebuild. Co-Authored-By: Warp <agent@warp.dev>
…mat, ensure finishTest always runs
On iOS 17+, xcodebuild hangs for ~600s at diagnostics collection after
tests complete. This caused the agent to wait with all processes (WDA,
ffmpeg, syslog, port forwards) still running, and video files to be
corrupted when the framework eventually force-killed ffmpeg.
XCTestCommandReceiver: detect test completion markers
- Added volatile boolean testComplete flag with isTestComplete() accessor
- Detects early completion: 'Test Suite All tests {passed|failed} at ...'
which appears immediately when tests finish, BEFORE the 600s diagnostics
- Also detects late markers: '** TEST {SUCCEEDED|FAILED|EXECUTE FAILED} **'
which appear only after diagnostics collection completes
XCTestRunner.runXctest(): poll for completion instead of blocking
- Replaced proc.waitFor() with a polling loop that checks both stdout and
stderr receivers for isTestComplete(), breaking immediately on detection
- After detection, gives 30s grace period then force-kills xcodebuild
- This reduces the post-test wait from ~600s to ~30s
XCTestRunner.run(): wrap test execution in try/finally
- finishTest() now runs in a finally block so it executes even if
analysisXctestResult() or other steps throw exceptions
- Previously, an ArrayIndexOutOfBoundsException in result parsing would
skip finishTest() entirely, leaving all processes orphaned
XCTestRunner.analysisXctestResult(): handle Objective-C test format
- Previously only handled Swift format: 'ClassName.testMethodName'
- Now also handles ObjC format: '-[ClassName testMethodName]' (no dot)
- Fixes ArrayIndexOutOfBoundsException when splitting on '.' for ObjC tests
IOSUtils.killProxyWDA(): scope kills to device UDID
- Changed 'xcodebuild test-without-building' (matches ALL devices) to
'xcodebuild.*WebDriverAgentRunner.*<UDID>' (matches only this device)
- Added UDID to dvt launch kill pattern for multi-device safety
Co-Authored-By: Warp <agent@warp.dev>
Implement iOS pullFileFromDevice using pymobiledevice3 apps pull to retrieve files from an app's sandboxed container after test execution. This is the iOS equivalent of Android's adb pull. Changes: - IOSDeviceDriver.pullFileFromDevice(): Full implementation replacing the no-op stub. Supports two path formats: - bundleId:/path (e.g. com.6alabat.cuisineApp:/Documents/) - /path (uses task's pkgName as bundle ID) - IOSUtils.pullFileFromApp(): New helper wrapping pymobiledevice3 apps pull with local directory creation and error handling. - Subfolder organization: Pulled files are placed in a named subfolder matching the remote path (e.g. /Documents/ -> Documents/) instead of being dumped flat into the result folder root. Falls back to pulled_files/ if the remote path is just /. Ensures parity with Android's adb pull behavior. - Updated CHANGELOG.md with Phase 7 details - Updated docs/API-Reference.md with subfolder organization notes - Updated docs/iOS-Testing-Guide.md with pullFileFromDevice section, result folder structure, and v1.2.0 version history Co-Authored-By: Warp <agent@warp.dev>
Add Android-Testing-Guide.md covering: - Prerequisites (Android SDK, ADB, ANDROID_HOME setup) - Device onboarding (developer options, USB debugging, verification) - Test execution for all supported types (Instrumentation/Espresso, Monkey, Appium, Maestro) with curl examples - Test scope options (TEST_APP, PACKAGE, CLASS) - Screen recording strategies (PhoneAppScreenRecorder vs ADBScreenRecorder) - pullFileFromDevice usage with common paths and retry behavior - Full device actions reference (14 methods with args and descriptions) - Automatic device setup/teardown (animations, screen timeout) - Test Orchestrator support for flaky test isolation - Troubleshooting guide (7 common issues with solutions) - CI/CD integration example (GitHub Actions) - File locations and project structure Co-Authored-By: Warp <agent@warp.dev>
13 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Complete migration of iOS device management from tidevice to pymobiledevice3 to restore iOS 17+ device support.
Why: tidevice is incompatible with iOS 17+ because Apple replaced the
lockdownd-based developer image mounting with CoreDevice/RemoteXPC. tidevice development has stalled since 2021. pymobiledevice3 is actively maintained and supports both legacy (iOS < 17) and modern (iOS 17+) protocols.What: This PR spans 7 phases of incremental changes covering CLI migration, video recording fixes, WDA launch for iOS 17+, XCTest runner improvements, auto device detection polling, iOS file pull implementation, and comprehensive documentation.
Linked GitHub issue ID: N/A (internal iOS 17+ support requirement)
Pull Request Checklist
Does this introduce a breaking change?
No breaking changes. All changes are backward compatible:
JSON parsing supports both tidevice and pymobiledevice3 field names
iOS < 17 devices continue to use
dvt launchfor WDAAndroid device management is unchanged
Yes
No
How you tested it
Tested on real hardware throughout all 7 phases:
c7ad90190806994c5c4d62117b4761adc37674c9), iOS 26.20R15205I23100583), Android 13com.6alabat.cuisineApp(iOS),com.talabat(Android)Please check the type of change your PR introduces:
Feature UI screenshots or Technical design diagrams
Architecture: Before (tidevice)
tidevice list --jsontidevice usbmux watch(continuous event stream -> IOSDeviceWatcher thread)tidevice xctest(instrument protocol over usbmux)startRecordingScreen()(internal ffmpeg + MJPEG)Architecture: After (pymobiledevice3)
Key Architectural Differences
tidevice list --jsonpymobiledevice3 usbmux listtidevice usbmux watch(event stream)@Scheduledpolling every 60s (no watch equivalent)tidevice xctest(instrument protocol)pymobiledevice3 developer dvt launchxcodebuild test-without-building(XCUITest session)startRecordingScreen()pymobiledevice3 apps pull(AFC protocol) with subfolder orgproc.waitFor()(hung ~600s on iOS 17+)Changes by Phase
Phase 1 - Core CLI Migration (
9cf52178): Replace alltideviceCLI invocations inIOSUtils.javawithpymobiledevice3equivalents (usbmux list, lockdown info, apps install/uninstall/list, developer dvt launch/screenshot, syslog live, crash pull, usbmux forward). UpdatedIOSDeviceDriver.parseJsonToDevice()for pymobiledevice3 JSON fields with fallback to tidevice field names. Addedpymobiledevice3toEnvCapability.Phase 2 - Video Recording Fix (
0e54ad41,0be2f9ce): Fixed 0-byte video files caused by race condition (pymobiledevice3 port forwarding not ready when ffmpeg connected). AddedwaitForPortToBeListening()polling. Switched Mac recording from Appium built-in to direct ffmpeg + pymobiledevice3 MJPEG forwarding. Removed conflictingmjpegServerPortAppium capability.Phase 3 - Zip Bomb Protection (
b0656788): Fixed zip bomb detection logic inZipBombChecker.java.Phase 4 - iOS 17 WDA Launch & Video (
b0b40b1a): iOS version-branched WDA launch (dvt launchfor iOS < 17,xcodebuild test-without-buildingfor iOS 17+). iOS 17+ ffmpeg usesscale=720:-2,setsar=1 -pix_fmt yuv420pfor QuickTime compatibility. AddedisIOS17OrAbove(),getWdaProjectPath()helpers. Added scripts:install_wda.sh,install_wda_below_ios_17.sh,cleanup_ios_ports.sh.Phase 5 - XCTest Runner Cleanup (
564b06d7): Fixed XCTest hanging ~600s after tests complete on iOS 17+. Added early completion detection inXCTestCommandReceiver. Replaced blockingwaitFor()with polling loop + 30s grace. WrappedfinishTest()in try/finally. Handle ObjC test format (-[ClassName testMethodName]). ScopedkillProxyWDA()to device UDID for multi-device safety.Phase 6 - Auto Device Detection (
11d2e5d3): Added@Scheduled(fixedDelay=60000, initialDelay=30000)polling inScheduledDeviceControlTasksto replace missingtidevice usbmux watchequivalent.Phase 7 - iOS pullFileFromDevice (
b074d368): ImplementedIOSDeviceDriver.pullFileFromDevice()(was no-op stub). ParsesbundleId:/pathformat. NewIOSUtils.pullFileFromApp()wrapspymobiledevice3 apps pull(AFC protocol). Subfolder organization: pulled files go into named subfolder matching remote path (e.g./Documents/->Documents/) for parity with Androidadb pull.Files Changed (22 files, +3492 -100)
Java Source
common/.../util/IOSUtils.javacommon/.../device/impl/IOSDeviceDriver.javacommon/.../entity/agent/EnvCapability.javacommon/.../management/AppiumServerManager.javacommon/.../screen/IOSAppiumScreenRecorderForMac.javacommon/.../util/ZipBombChecker.javaagent/.../runner/xctest/XCTestRunner.javaagent/.../runner/xctest/XCTestCommandReceiver.javaagent/.../scheduled/ScheduledDeviceControlTasks.javaagent/.../service/DeviceControlService.javaScripts
scripts/install_wda.shscripts/install_wda_below_ios_17.shscripts/cleanup_ios_ports.shDocumentation
CHANGELOG.mdTIDEVICE_TO_PYMOBILEDEVICE3_MIGRATION.mddocs/API-Reference.mddocs/Android-Testing-Guide.mddocs/iOS-Testing-Guide.mdTODO.mdDependencies
pip3 install pymobiledevice3)xcodebuild)Co-Authored-By: Warp agent@warp.dev