[iOS] Appium test automation slowed down by 4.5.0.725

[Veilenus]

Hello everyone,

I'm working on a cross-platform Xamarin.Forms app that uses Appium for its UI test automation. Recently we've had to update from Xamarin.Forms 4.5.0.617 to 4.7.0.968 for some map related features. Unfortunately, the update also introduced a significant slowdown of our automated UI tests on iOS. Now they take approximately 10 times longer than before. Tests on Android didn't seem to be affected.

After ruling out every other change to our app code, I've had a closer look at the Xamarin.Forms NuGet package and a fun time experimenting with different versions. What I've found is, that Xamarin.Forms 4.5.0.725 is the first version to show this undesirable slowdown and 4.5.0.657 is the last version that works as expected. Finally, and just to be comprehensive: I've also tested the latest release 4.7.0.1080 and it didn't help.
As a side note: We've had slow tests on iOS before. Back then the app was built using Xamarin.Forms 4.2. I was just getting started automating the project and kind of took the slowness for granted. The update to 4.5 really sped things up and it's painful to see it go back down again.

After reviewing the Xamarin.Forms version history, 725's in particular, I was not able find anything obvious to put my finger on. In fact, I'm not entirely sure it's only Xamarin at fault here. I guess there is always the off chance of some dependencies going haywire, despite the only thing being changed is Xamarin, I don't know. That's why didn't feel comfortable creating an issue and rather stuck with a discussion.

I would very much appreciate any help and advice you can offer. Happy coding!

Overview:

• Significant slowdown of UI test automation on iOS using Appium (1.17.1)
• Last working version: 4.5.0.657
• First "slow" version: 4.5.0.725
• Latest version tested (still slow): 4.7.0.1080
• Experienced this issue in the past with version: ~4.2
• Mobile device tested: iPhone 8 Plus, iOS 13.5.1

[hartez]

For reference, 657 would be 4.5 service release 5, and 725 would be service release 6.

What controls are in use in the app with the slow automation? What automation actions do you primarily use? Maybe that can give us a clue to what's going on.

[Veilenus]

Thank you very much for responding. There's nothing too fancy going on in the app. It consists of a couple of dozen pages, most of them contain little more than a button and some labels. There is one really crowded page with a lot of content that's selectively visible under certain conditions. The slowdown affects all pages across the whole app, there is no particular page or control associated with it.

However, considering that the automation is slowed down even on the barest of pages, I want to mention some peculiarities that are even true on those:

• The app uses a few custom controls. So, for example, instead of a label, it would use a custom label that inherits from Label and extends it.
• Styles, colors, etc. are read from a resource dictionary.
• The app is multi-lingual and utilizes .resx files for text resources.

The automation action I primarily use is finding controls with certain AutomationIds. Tapping them is probably the next most used action. From what I've seen the slowdown seems to stem from an overly long delay when retrieving the page source (or at least it clearly shows the effect the actual cause has). I've had a look at the Appium and XCUITest output and was able to compare a slow version of our app (XF 4.7.0.968) with a fast one (XF 4.5.0.617). The payloads of the messages show no distinguishable difference, but the time taken is as different as day and night: Whereas finding an element on the largest pages took like 500 ms before, it now takes over 20 seconds. That's kind of stating what was already known. Unfortunately, that is as much information as I can get without debugging the WebDriverAgent itself, which I don't feel competent enough to do.