I will be focusing on UI Automated Tests and Test Cloud. The purpose of this post is to do a basic overview of what it can do and a little kickstart into how to use Xamarin for Automated UI Testing.
UI Automated Tests are tests that perform the manual taps / scrolls / typing / other interactions a person would normally have to do (QA). These are usually the more tedious interactions and many places would save a lot of time and stress by automating these tests. It allows for QA to do their more difficult specific work and remove the manual time consuming stuff. It also sniffs out differences in different platforms and a large scope of devices. The scope of these tests are basically anything you can do with direct interaction on the phone manually, doesn't involve only the back-end (use unit tests here), and can be performed while the mobile application is being run. You have the choice of using Calabash or C# as a language for development -- at this point when I have spoken with the Xamarin team, the functionalities of both languages are now about equal and you can choose one or the other (in the past you had good coverage with C#, but full coverage using Calabash). Calabash is basically the framework with all the methods and whatnot defined for you to use to perform the testing. Now the C# Xamarin.UITest assembly has the same methods open for use too.
Let's start with how to set up a suite for testing. Purchase Xamarin licenses. Startup a new empty C# solution. Add NuGet packages for Xamarin.UITest and nunit.framework. Now you can separate some folder structure into Global, Pages, and Tests. In this main directory, you will also want to make a new class and call it AppInitializer or something like this. HERE is the Xamarin documentation for starting an app. You will get your api key from Xamarin, and the other constants are your local builds. You can choose to use an emulator or a physical device for testing.
public static class AppInitializer
{
const string apiKey = "YOUR_API_KEY";
const string apkPath = "YOUR LOCAL FILE";
const string appFile = "YOUR LOCAL FILE";
const string bundleId = "YOUR BUNDLE ID";
public static IApp StartApp(Platform platform)
{
IApp app;
if (platform == Platform.Android)
{
app = ConfigureApp
.Android
.ApiKey(apiKey)
.ApkFile(apkPath)
.StartApp();
}
// can add another if / else here for iOS
return app;
}
}
You will need a base abstract class, let's just call it Setup.cs for now..this is how you will cause the app to hit the AppInitalizer.StartApp() function every time before you run a test. You will inherit from this base class on each new Test class.
public abstract class Setup
{
protected IApp app;
protected Platform platform;
protected AbstractSetup(Platform platform)
{
this.platform = platform;
}
[SetUp]
public virtual void BeforeEachTest()
{
app = AppInitializer.StartApp(platform);
if (platform == Platform.iOS)
{
// implement here
}
if (platform == Platform.Android)
{
// implement here
}
ResetApp();
app.Screenshot("On Home Page");
}
public void ResetApp()
{
new GlobalPage(app, platform)
.YourMethodNameHere();
}
}
So, back to the 3 sections:
- Global: you can put methods you will use throughout the whole app, enums, a methods for swiping from left or right, things you will just need all the time no matter what page you are on and so you can avoid code duplication.
- Pages: for each page in your app you plan to test, you should have a page for it with relevant properties and methods it will need. Let's say the mobile developers named their Main Logo iOS-main-logo and Android-main-logo.. the page is a good place to map these to YOUR testing property name, such as simply mainLogo. Now this makes ease for you to deal with any tapping on this mainLogo in your testing app and it is already mapped to both the Android and iOS versions. You can write basic functions like, TapLogoGoHome() into this page so no matter what you can call this function in the tests without needed to rewrite it over and over again.
- Tests: Below is an example of a test. You have to have a [Test] attirbute. You can optionally have a "category" attribute which means that you can run a specific "playlist" of tests for example if you don't want to run them all every time. You will use this when sending it to Test Cloud and specify the cateogory of tests to run: in this example it can be smoke or secondCategory. You can add a test to as many lists as you like, just keeping adding them on a new line.
[Test]
[Category("smoke")]
[Category("secondCategory")]
public void SwipeOpenSideMenuAndGoHome()
{
string tapOrSwipe = "swipe";
new GlobalPage(app, platform)
.OpenSideMenu(tapOrSwipe);
new SideMenuPage(app, platform)
.NavigateToHome();
new HomePage(app, platform);
}
On GlobalPage, you will find the OpenSideMenu method, which does the actual .Tap() function.
public class GlobalPage : BasePage
{
protected string menuButton;
public GlobalPage(IApp app, Platform platform)
: base(app, platform)
{
if (OnAndroid)
{
menuButton = "MenuButton";
}
if (OniOS)
{
menuButton = "y";
}
}
public void OpenSideMenu(string TapOrSwipe = "tap")
{
app.Tap(menuButton);
app.ScrollUp();
app.WaitFor(isOpen, timeout: TimeSpan.FromSeconds(5));
}
}
Using the REPL is necessary to get the elements on the page needed to tap/scroll to get the test working. What you do is call app.Repl() inside of your code, you right click your test and put in Debug test and drop a breakpoint on your app.Repl line. After you hit it, step forward once.. and you will see a little pop up. Type "tree" with no quotation marks and this will give you a layout of everything on the page. SEE the Xamarin Repl tree explanation. Repl specifically spits out what is on the page you are on. You need the IDs (so ask your developers to ID everything in a logical fashion!) to find out what you need to tap and whatnot. You get it from the Repl, not from going through their codebase as this is much faster. If you find something you are not sure about, you can go into the Repl console box and type queries into like app.Flash(c=>c.Class("EditText")) and it will flash the item (blink so you can see it) and so you can tell if it is the right element or not. Sometimes there is no ID, now you have to grab the item some other way such as getting a Child(), Parent(), Descendant(), Sibling(), etc and the index of it (to see which number element it is). This can be good/bad.. it requires maintenance for if the index number or structure changes, but sometimes in a face paced environment without IDs there is no other choice. With app.Query(); you can get things like a count or the text from an element.
// for IDE Xamarin Studio or Visual Studio
stuffOnScreen = app.Query(x => x.Id(ListView).Child()).Count();
var currResult = app.Query(x => x.Id(ListView).Child(stuffOnScreen - 1).Descendant())[2].Text;
// for Repl() tree to find your element
app.Flash(x => x.Button())
// Everything on the page
app.Query()
// One element on the page by index
app.Query()[0]
app.Query()[36]
// total things on page
app.Query().Count()
// Certain matching element on page
app.query(x => x.Id("refresher")
// Flashes and finds something on the page.
app.Flash(x => x.Id("bill_iconImageButton"))
// Tap action on page
app.Tap("some_iconImageButton")
/// note that sometimes the Next button
/// is on the keyboard,
/// so this...
app.Flash(x => x.Id("Next"))
// brings back nothing
// but! doing this in the Repl..
app.PressEnter()
// moves the page for you out of the keyboard and onto the next page.
// Therefore.. add this to your test method
app.PressEnter();
Last minute tips.. use the Object Browser on the Xamarin.UITest assembly a lot with the Xamarin doucmentation, it makes a lot more sense that way. The Xamarin developers and support are great and there for you if you need it! You can use emulators or physical devices. If you normally work on OSX or don't mind it, it is easier because you can run everything (Android emulators/iOS emulators or physical devices for both) in one place and use Xamarin Studio which is fully integrated. If you are like me.. you can try doing Windows and Visual Studio to run Android and develop everything there (you CANNOT emulate iOS here unless you have a nearby Apple computer and can use it as a building environment by connecting to it.. or..), then move the files over to the OSX/Xamarin Studio environment to run the iOS tests. Likely you will have to tweak the IDs of everything in iOS and debug the tests all over again as iOS behaves differently and things are likely labeled differently, too. Good luck!