Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

All tests failing but Setup and Teardown #5

Open
aw-vm opened this issue Jul 30, 2018 · 5 comments
Open

All tests failing but Setup and Teardown #5

aw-vm opened this issue Jul 30, 2018 · 5 comments

Comments

@aw-vm
Copy link

aw-vm commented Jul 30, 2018

Hello,

I have tried running my tests on device farm but all the test are failing.
I have followed all the steps of the documentation and generated the files as the instructions said.
I have not encountered any issue uploading these files or creating a run with them but when the run is over I get these results:

screen shot 2018-07-30 at 5 30 12 pm

I'm getting this from the automation output of one test:

User defaults from command line:
    IDEDerivedDataPathOverride = /tmp/scratchnZxqrd.scratch/xcode_screenshots7MfdjK/UITests15TwilioE4UITests/testExample
    IDETestRunOnlyIdentifiers = (
    "TwilioE4UITests/UITests15TwilioE4UITests/testExample"
)
    IDETestRunSpecificationPath = /Users/rwx/Library/Developer/Xcode/DerivedData/Build/Products/application.xctestrun

2018-07-30 09:42:09.687 xcodebuild[535:7368]  IDETestOperationsObserverDebug: Writing diagnostic log for test session to:
/var/folders/tg/pk__v_v5785_6_qbhdg297gr0000gp/T/com.apple.dt.XCTest/IDETestRunSession-40A4F518-21AA-49FF-97D0-CDCD75DDF535/TwilioE4UITests-79D669D7-3634-4C57-8276-610333B02474/Session-TwilioE4UITests-2018-07-30_094209-hBcQkX.log
2018-07-30 09:42:09.689 xcodebuild[535:7354] [MT] IDETestOperationsObserverDebug: (032D8571-B44D-4931-A6DA-F7D1FA773B91) Beginning test session TwilioE4UITests-032D8571-B44D-4931-A6DA-F7D1FA773B91 at 2018-07-30 09:42:09.688 with Xcode 9F1027a on target <DVTiOSDevice: 0x7fc66fe14260> {
		deviceSerialNumber:         G6TVH3K9JCL8
		identifier:                 b2f96e8a3a53c1bd94d05fc1d0c06aa94954148a
		deviceClass:                iPhone
		deviceName:                 9902852853
		deviceIdentifier:           b2f96e8a3a53c1bd94d05fc1d0c06aa94954148a
		productVersion:             11.4
		buildVersion:               15F79
		deviceSoftwareVersion:      11.4 (15F79)
		deviceArchitecture:         arm64
		deviceTotalCapacity:        252371152896
		deviceAvailableCapacity:    249001328640
		deviceIsTransient:          NO
		ignored:                    NO
		deviceIsBusy:               NO
		deviceIsActivated:          YES
		deviceActivationState:      Activated
		isPasscodeLocked:           NO
		deviceType:                 <DVTDeviceType:0x7fc66fd80f20 Xcode.DeviceType.iPhone>
		supportedDeviceFamilies:    (
    1
)
		applications:              (null)
		provisioningProfiles:      (null)
		hasInternalSupport:        NO
		isSupportedOS:             YES
		developerDiskMountError:   (null)
(null)
	bootArgs:                  <unavailable>
		connected:                 yes
		isWirelessEnabled:         no
		connectionType:            direct
		} (11.4 (15F79))
MDMCreateDeltaDirectory:1920 calling MDMDirectoryDiff with:
state->old_bundle: /var/folders/tg/pk__v_v5785_6_qbhdg297gr0000gp/C/com.apple.DeveloperTools/All/Xcode/EmbeddedAppDeltas/977ae2eae09fab3f9a27a0f75731a39d/b2f96e8a3a53c1bd94d05fc1d0c06aa94954148a/TwilioE4UITests-Runner.app
state->new_bundle: /Users/rwx/Library/Developer/Xcode/DerivedData/Build/Products/Debug-iphoneos/TwilioE4UITests-Runner.app
state->dst_bundle: /var/folders/tg/pk__v_v5785_6_qbhdg297gr0000gp/C/com.apple.DeveloperTools/All/Xcode/EmbeddedAppDeltas/TwilioE4UITests-Runner.app.aYOj0V/TwilioE4UITests-Runner.app_sparse.ipa/Payload//TwilioE4UITests-Runner.app, binaryDiff flag: FALSE
	dst_ipa: /var/folders/tg/pk__v_v5785_6_qbhdg297gr0000gp/C/com.apple.DeveloperTools/All/Xcode/EmbeddedAppDeltas/TwilioE4UITests-Runner.app.aYOj0V/TwilioE4UITests-Runner.app_sparse.ipa
MDMDirectoryDiff_block_invoke:1473 calling writeDictToFile with: /var/folders/tg/pk__v_v5785_6_qbhdg297gr0000gp/C/com.apple.DeveloperTools/All/Xcode/EmbeddedAppDeltas/TwilioE4UITests-Runner.app.aYOj0V/TwilioE4UITests-Runner.app_sparse.ipa/ManifestCache.plist
writeDictToFile:1278 ==== Successfully wrote Manifest cache to /var/folders/tg/pk__v_v5785_6_qbhdg297gr0000gp/C/com.apple.DeveloperTools/All/Xcode/EmbeddedAppDeltas/TwilioE4UITests-Runner.app.aYOj0V/TwilioE4UITests-Runner.app_sparse.ipa/ManifestCache.plist
** TEST EXECUTE SUCCEEDED **

2018-07-30 11:42:16.421756-0500 TwilioE4UITests-Runner[588:26690] +[CATransaction synchronize] called within transaction
2018-07-30 11:42:16.442034-0500 TwilioE4UITests-Runner[588:26690] Running tests...
2018-07-30 11:42:17.072729-0500 TwilioE4UITests-Runner[588:26690] [AXValidations] UIAccessibility Error adding -[CAMLightingControl accessibilityLabel]
2018-07-30 11:42:17.072828-0500 TwilioE4UITests-Runner[588:26690] [AXValidations] UIAccessibility Error adding -[CAMLightingControl accessibilityValue]
2018-07-30 11:42:17.072880-0500 TwilioE4UITests-Runner[588:26690] [AXValidations] UIAccessibility Error adding -[CAMLightingControl accessibilityTraits]
2018-07-30 11:42:17.072926-0500 TwilioE4UITests-Runner[588:26690] [AXValidations] UIAccessibility Error adding -[CAMLightingControl isAccessibilityElement]
2018-07-30 11:42:17.072977-0500 TwilioE4UITests-Runner[588:26690] [AXValidations] UIAccessibility Error adding -[CAMLightingControl accessibilityIncrement]
2018-07-30 11:42:17.073017-0500 TwilioE4UITests-Runner[588:26690] [AXValidations] UIAccessibility Error adding -[CAMLightingControl accessibilityDecrement]
2018-07-30 11:42:17.073053-0500 TwilioE4UITests-Runner[588:26690] [AXValidations] UIAccessibility Error adding +[CAMLightingControl safeCategoryBaseClass]
2018-07-30 11:42:17.079126-0500 TwilioE4UITests-Runner[588:26690] [AXValidations] UIAccessibility Error adding -[PLManagedAsset accessibilityLabel]
2018-07-30 11:42:17.079200-0500 TwilioE4UITests-Runner[588:26690] [AXValidations] UIAccessibility Error adding -[PLManagedAsset accessibilityValue]
2018-07-30 11:42:17.079233-0500 TwilioE4UITests-Runner[588:26690] [AXValidations] UIAccessibility Error adding -[PLManagedAsset accessibilityTraits]
2018-07-30 11:42:17.079263-0500 TwilioE4UITests-Runner[588:26690] [AXValidations] UIAccessibility Error adding -[PLManagedAsset _accessibilityPhotoDescription]
2018-07-30 11:42:17.079292-0500 TwilioE4UITests-Runner[588:26690] [AXValidations] UIAccessibility Error adding +[PLManagedAsset safeCategoryBaseClass]
2018-07-30 11:42:17.096756-0500 TwilioE4UITests-Runner[588:26690] Continuing to run tests in the background with task ID 1
Test Suite 'Selected tests' started at 2018-07-30 11:42:17.717
Test Suite 'TwilioE4UITests.xctest' started at 2018-07-30 11:42:17.718
Test Suite 'TwilioE4UITests.xctest' passed at 2018-07-30 11:42:17.719.
	 Executed 0 tests, with 0 failures (0 unexpected) in 0.000 (0.001) seconds
Test Suite 'Selected tests' passed at 2018-07-30 11:42:17.720.
	 Executed 0 tests, with 0 failures (0 unexpected) in 0.000 (0.003) seconds

Also checking the System logs I could find this log many times:

Jul 30 11:41:56 9902015857 amfid(Security)[195] <Notice>: could not enable test hierarchy: ApplePinningAllowTestCertsiPhoneApplicationSigning not true

Watching the test videos looks like the app is not opening and nothing is happening there.
Reading the logs I found this one:
Unable to generate launch image for com.apple.test.TwilioE4UITests-Runner because the app has no valid launch interfaces.
screen shot 2018-07-30 at 5 35 01 pm
@nikhil-dabhade
Copy link

@valerymadiedo : Few questions:

  1. Have you ran the same tests of a real physical device locally? The reason I ask is at times the test may have been built and run for simulator rather than a real device and error out when running on a real device.
  2. Do you have entitlements in your iOS application? If yes, these will break on Device Farm as it re-signs the app. Using private devices gives you the ability to skip re-signing of your app.
  3. I also see MDM in the logs. Do you require MDM client to be present on the device while the tests are running?

@aw-vm
Copy link
Author

aw-vm commented Jul 31, 2018 •
edited
Loading

Hello @nikhil-dabhade Thanks a lot for your help.
Answers:

  1. Yes I have run these tests in a physical device (iphone 5s - ios 11.2.1).
  2. No. The ipa I uploaded to device farm was signed as developer, which is the right way to sign it?
  3. Yes. I save some data locally using sqlite.swift.

Edit: here says I must create ad hoc profile and add the devices and I can find the uuid of the device on the device instance and add these to my provisioning profiles but these devices must be private and I dont have access to these yet.

@aw-vm
Copy link
Author

aw-vm commented Aug 2, 2018

Hi,
I have resigned the ipa with an adhoc profile and I'm getting the same behavior. The test opens but is not able to open the app.

@af-mke
Copy link

af-mke commented Oct 22, 2018 •
edited
Loading

Hey @valerymadiedo , did you solve the problem? I also get these errors and can't figure out what the problem is

@croossin
Copy link

croossin commented Aug 9, 2019

Any updates? Facing same issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@croossin @nikhil-dabhade @aw-vm @af-mke