Every web developer knows how challenging it can be to get a web site working and looking great with every possible browser variation out there. This is one area where test automation has a significant value to offer. Even with responsive web sites that should scale all different resolutions, work across different types of devices (phones, tablets, wearables etc.) the test automation can harmonize the testing effort and bring you consistent foundation to build your web site tests.

Today, we’re presenting at Appium Meetup in San Francisco – and this topic will be addressed as a workshop, with the focus on best practices, how-to and bunch of tips and tricks on how to automate your mobile web testing, using real devices with Appium test automation framework.

As web services are increasingly accessed by mobile devices it is getting more and more important to test your web service using real browsers and real devices that your end users will use when are accessing your site. Furthermore, the emulators and simulators have severe limitations when it comes to browser testing. For instance Chrome browser, that according to W3Counter has the overall 42,5% market share and according to NetMarketshare 23.7% among Android users cannot be installed on an Android emulator.

Furthermore, Browser is also one of those areas where almost all devices manufacturers want to add their ‘innovation’ leading to happy mix of different webkit versions with different functionality. Another area where any mobile web developer needs to pay attention is performance: Web page rendering time vary significantly from one device to another and to ensure acceptable end user experience you need to test how your CSS & Javascript renders across devices.

Selenium has already cemented its place as a de-facto web test automation standard and Appium is providing very clean high level API that is familiar to all Selenium developers of the world but it nicely abstracts lot of the messy stuff related to running tests on real devices. We at Testdroid were the first device cloud service to offer Appium on real devices and naturally mobile web testing using Appium is part of the deal.

On January’s San Francisco Appium meetup Testdroid’s Appium expert Saad Chaudhry will be presenting how to automate testing of your mobile web service using Appium and real devices, what you need to take into account when using real devices for web testing and naturally will run some nice demos on the topic.

Please sign up at http://www.meetup.com/Appium-SF/

Dear Testdroiders,

Many of today’s apps utilize Internet browsers and web technologies, even those native apps on Android and iOS. The effectiveness of testing these apps varies widely among users, which frameworks and tools are used. Majority of teams that are building either solely native, hybrid or web apps use Agile methodology wherein test automation has a significant value to offer.

We’ve introduced support for variety of different frameworks since Testdroid was launched – and one of the great use case (and benefit) for this approach is the flexible use of real mobile devices, real mobile browsers and different types of apps – with the diverse selection of different test automation frameworks.

We discuss about different test automation frameworks here at Testdroid blog quite often, and now we’ll take a look at how Appium/Selenium can be used to test your mobile web stuff running on REAL environments (real devices and browsers, and not emulators).

First Thing First: Appium equals Selenium

“Appium is like Selenium – but for mobile apps and games“. You’ve probably read/heard this several times before, but in fact, Appium is much more than that. Appium is also suitable for mobile web testing, when real devices and real browsers are used in testing. Emulators won’t do the trick as e.g. Chrome cannot be even installed on standard Android QEMU-based emulator. Some emulators may work (with Chrome), but why bother – those are still emulators and no single end-user uses emulators to run your app, game or web products. With the same effort (and cost) you can build your test automation with Appium using real stuff.

In a nutshell, Appium is a mobile test automation framework (with a tool) that works for all: native, hybrid and mobile-web apps for iOS and Android. Appium is a great choice for test automation framework as it can be used for all these different app/web types. Basically, Appium derives its roots from Selenium and it uses JSONWireProtocol internally to interact with iOS and Android apps using Selenium’s WebDriver.

In its architecture, Appium is an HTTP server written in Node.js that creates and handles multiple WebDriver sessions. Appium starts tests on the device and listens for commands from the main Appium server. It is basically the same as the Selenium server that gets HTTP requests from Selenium client libraries.

Appium has proven its flexibility and scalability in native mobile app and game testing, as those apps and games built by app and game developers tend to be pretty much identical on both platforms, Android and iOS. The benefit of having these sort of cross-platform framework helps using the identical script either app/game is running on either platform. The very same applies for mobile web. Appium scripts run nicely whether you are testing website on Chrome, Firefox, Safari, WebKit-based browser on Android and iOS (where applicable).

Another significant benefit of Appium is that users can write tests using their favorite development tools, environment and programming language, such as Java, Objective-C, JavaScript, PHP, Ruby, Python or C#, among many others.

How to Get Started and What Things to Consider?

Don’t worry about the learning curve. If you are familiar with Selenium, then you’ve got Appium covered: They use the same WebDriver, and DesiredCapabilities is used the same way. There is a great chance your existing Selenium test scripts work as they are with Appium setup. Nevertheless, configuring an application to run on Appium has a lot of similarities to Selenium — for example, those DesiredCapabilities. If you are looking an easy way to get started, check this out. These instructions provide you everything to ensure a smooth start with Appium, real devices and real web browsers.

Bear in mind that the mobile ecosystem is very different compared to the desktop world. As mobile OEMs are building their own stuff, they also tend to ‘differentiate’ things on various standard things – such as browsers. In fact, mobile browsers – regardless how standard they are – is also one of those areas where almost all devices manufacturers want to add their ‘innovation’ and this is quickly leading to a happy mix of different webkit versions with different functionalities. That combined with the OS version, hardware configuration, different sizes of displays, memory and so on can significantly ‘differentiate’ what user gets as form of these browsers. Building your test scripts with Appium/Selenium and automating testing can quickly get you understanding how well does your web stuff work on those devices and browsers.

Another area that any mobile web developer needs to pay attention at is performance: Web page rendering time vary significantly from one device to another and to ensure acceptable end user experience you need to test how your CSS & Javascript renders across devices. There are already great benchmarks – such as BrowserMark by Rightware – that can be used to measure the performance of hardware together with certain version of a browser. This gives understanding how well certain work load (unfortunately not configurable nor your own web stuff is usable on these browser benchmarks) does on certain device with certain web browser. But to understand how your own web stuff behaves under different conditions, test automation with Appium/Selenium has a lot to offer.

Selenium has already cemented its place as a de-facto web test automation standard and Appium is providing very clean high level API that can be quickly adopted by Selenium developers, as it nicely abstracts lot of the messy stuff related to running tests. Our Testdroid Cloud provides this sort of environment where everything is packaged and pre-build for your Selenium / Appium tests, using only real mobile devices (Android and iOS) and real browsers that are usable with those devices (Safari for iOS, Chrome, WebKit, Firefox and others for Android).

Example: Appium for Mobile Web Testing

Here is one example how we’ve enabled Appium with real devices and real browsers, using the Testdroid infrastructure to get Selenium-based test scripts up and running on real devices.

Happy Web Testing!

Dear Testdroiders,

The Basics of Mobile Web Testing blog sparked some serious interest and many of you have been curious how to build a very basic script for the mobile web test using Selenium/Appium. I’ll walk you through here with the brief example how to build a simple test for your website, using real devices and real browsers. The script below follows the video example so let’s start with that one.

I’m using Python here so if you don’t have Python installed and configured, check this out first.

The first thing to do is to do the imports and write a basic function for log outputting.

import os
import sys
import time
from time import sleep
from selenium import webdriver

##
## More help on writing tests:
## => http://selenium-python.readthedocs.org/en/latest/navigating.html
##

def log(msg):
print (time.strftime("%H:%M:%S") + ": " + msg)

As a first real step, define the URL for device access. To access Testdroid Cloud’s devices (both Android and iOS) use the following URL:

appium_URL = 'http://appium.testdroid.com/wd/hub';

Then, define your local screenshot path where all taken screenshots will be stored:

screenshot_path = '/Users/vhelppi/Documents/Appium-Samples/Screenshots/Chrome';

Let’s take a look at Desired Capabilities. These describe your login credentials, used target (e.g. which web browser do you want to use for the session), project name and test run (by definition what you see under your Testdroid Cloud projects), what device(s) will be used for the session, platform and browser. Make sure you follow the instructions with setting proper desired capabilities – things can quickly go wrong here. Also, please make sure you use correct device names. These can be fetched from Testdroid Cloud device listing and name defined here must map identically to the one in cloud:

desired_caps = {}
desired_caps['testdroid_username'] = ‘ville-veikko.helppi@bitbar.com’
desired_caps['testdroid_password'] = 'xxxxxxxx'
desired_caps['testdroid_target'] = 'chrome'
desired_caps['testdroid_project'] = 'Appium Chrome'
desired_caps['testdroid_testrun'] = 'TestRun 1'
desired_caps['testdroid_device'] = 'Asus Google Nexus 7 (2013) ME571KL'
desired_caps['platformName'] = 'android'
desired_caps['deviceName'] = 'AndroidDevice'
desired_caps['browserName'] = 'chrome'

Now, let’s initiate the Selenium driver by calling Remote() from Selenium library. As a parameter the Appium URL (pointing to Testdroid Cloud) and Desired Capabilities are given. And naturally, it’s always a good idea to output some status information to logs/screen.

log ("WebDriver request initiated. Waiting for response, this may take a while.")
driver = webdriver.Remote(appium_URL, desired_caps)

It is also a good idea to take screenshots when ever appropriate. More is surely better, but try to keep number of those manageable. Before we start the session with any browser or URL we’ll take a screenshot of web browser (plain view, nothing shown yet in the browser):

log ("Taking screenshot of home page: '0_chromeLaunched.png'")
driver.save_screenshot(screenshot_path + "/0_chromeLaunched.png")

Now, specify your website here. I use our Testdroid Community page for this test and take a screenshot of it:

log ("Loading page")
driver.get("http://testdroid.com")
log ("Taking screenshot of home page: '1_home.png'")
driver.save_screenshot(screenshot_path + "/1_home.png")

The next thing for the script is to find an element – “Products” button on Testdroid Community page. It’s defined as “Menu” and button number “1” (“0″ is the “Why Testdroid?”). The syntax given to find_element_by_xpath function call looks like this:

log ("Finding 'Products'")
elem = driver.find_element_by_xpath('//*[@id="menu"]/ul/li[1]/a')
log ("Clicking 'Products'")
elem.click()

Remember that there are a great set of different find_element function calls available in Selenium. For example, you can search elements by id, name, xpath, and many others.

After clicking the “Products” menu item on top of the screen the script takes a screenshot and locates “Learn More” button from the page. As you see, there are several “Learn More” buttons on this page, so searching by name wouldn’t help here. For sake of example, I’ll use xpath again. If you look at the page source code (www.testdroid.com/products -> inspect element) you’ll find this:

Now, based on the implementation, the script locates the right button with “Learn more” text and clicks it:

log ("Taking screenshot of 'Products' page: '2_products.png'")
driver.save_screenshot(screenshot_path + "/2_products.png")
log ("Finding 'Learn More'")
elem = driver.find_element_by_xpath('//*[@id="products"]/div[1]/div/div[1]/div[3]/a')
log ("Clicking 'Learn More'")
elem.click()

Allright. Now we’ve inspected some HTML elements, clicked those and some screenshots are taken. In the next few lines, we’ll find “topMenu” and identify “Supported Frameworks” button in that. The element inspection of the page looks like this:

And again, for sake of using xpath, the code for locating the right link looks like this:

log ("Taking screenshot of 'Learn More' page: '3_learnmore.png'")
driver.save_screenshot(screenshot_path + "/3_learnmore.png")
log ("Finding 'Supported Frameworks'")
elem = driver.find_element_by_xpath('//*[@id="topMenu"]/div[1]/div/a[2]')
log ("Clicking 'Supported Frameworks'")
elem.click()

Now we’ve landed to the page where supported test automation frameworks by different Testdroid products are shown.

log ("Taking screenshot of 'Supported Framworks' page: '4_supportedframeworks.png'")
driver.save_screenshot(screenshot_path + "/4_supportedframeworks.png")
log ("quitting")
driver.quit()
sys.exit()

If you want to try out this, you can download this Python script here.

Happy Web Testing!

Google’s Android ecosystem continues to expand rapidly. It is evident that new mobile OEMs are emerging in every corner of the world, bringing in different screen sizes, ROM/firmwares, chipsets, and etc. For Android developers, it becomes rather difficult to cope with the fragmentation.

Luckily, Android (including iOS) developers have had unlimited access to some advanced cloud-based solution, like Testdroid Cloud, to run automated tests on a large scale of real devices for quality assurance. Also the emergence of different Android testing frameworks has substantially eased Android developers’ lives.

Today we are going to behold 5 most used Android testing frameworks and break down the basics and code examples of each.

Robotium

Undoubted, Robotium was once the most widely used Android testing framework in the early days of Android world. With a similarity with Selenium in Android, it makes testing API simpler.
Robotium is an open source library extending JUnit with plenty of useful methods for Android UI testing. It provides powerful and robust automatic black-box test cases for Android apps (native and hybrid) and web testing. With Robotium you can write function, system and acceptance test scenarios, and test applications where the source code is available.

Robotium code example:

// Public void for the operation public void testRecorded() throws Exception { // Wait for the text 'Hello!' to be shown for newbie if (solo.waitForText("Hello!")) { // R class ID identifier for 'Sign in' - and click it solo.clickOnView(solo.findViewById("com.twitter.android.R.id.sign_in")); // R class ID identifier for entering username solo.enterText((EditText) solo.findViewById("com.twitter.android.R.id.login_username"),"username"); // R class ID identifier for entering password solo.enterText((EditText) solo.findViewById("com.twitter.android.R.id.login_password"),"password"); // R class ID identifier for clicking log in solo.clickOnView(solo.findViewById("com.twitter.android.R.id.login_login")); // Wait until log in is done solo.waitForActivity("HomeTabActivity"); } // Activate the text field to compose a tweet solo.clickOnView(solo.findViewById("com.twitter.android.R.id.menu_compose_tweet")); // Type the tweet solo.enterText((EditText) solo.findViewById("com.twitter.android.R.id.edit"), "Testdroid"); // Tweeting! solo.clickOnView(solo.findViewById("com.twitter.android.R.id.composer_post")); }

For your convenience, Testdroid Recorder is an awesome recording tool built with Robotium for test script creation. By performing actual actions on your real device, it records every step or action you take and converts to Javascript for your further modification.

In addition, you are also entitled to fully download and utilize our Extension Library – ExtSolo. It includes useful methods that have not been merged into Robotium, for instance:

Automatic scaling of x,y clicks for any resolution
Multi-path drags
Automatic screenshots on test failure
Mock locations
Change device language
Control WiFi connection

uiautomator

While Robotium is a good yet basic framework, uiautomator allows you to do more in testing Android apps and games. Google’s test framework allows you to test user interface (UI) of your native Android apps on one or more devices. Another advantage of uiautomator is that it runs JUnit test cases with special privileges, which means test cases can span across different processes. It also provides five different classes for developers to use, including
com.android.uiautomator.core.UiCollection; com.android.uiautomator.core.UiDevice; com.android.uiautomator.core.UiObject; com.android.uiautomator.core.UiScrollable; com.android.uiautomator.core.UiSelector

Similar to its time of birth, it only works on Android devices with API level 16 or higher. Another downside of uiautomator is that it doesn’t support webview, with no way to directly access Android objects.

uiautomator’s code example:

// Public void for the operation public void testSignInAndTweet() throws Exception { // Starting application: getUiDevice().wakeUp(); // Press Home button to ensure we're on homescreen getUiDevice().pressHome(); // Select 'Apps' and click button new UiObject(new UiSelector().description("Apps")).click(); // Select 'Twitter' and click new UiObject(new UiSelector().text("Twitter")).click(); // Locate and select 'Sign in' UiSelector signIn = new UiSelector().text("Sign In"); // If button is available, click UiObject signInButton = new UiObject(signIn); if (signInButton.exists()) { signInButton.click(); // Set the username new UiObject(new UiSelector().className("android.widget.EditText").instance(0)).setText("username"); new UiObject(new UiSelector().className("android.widget.EditText").instance(1)).setText("password"); new UiObject(new UiSelector().className("android.widget.Button"). text("Sign In").instance(0)).click(); // Wait Sign in progress window getUiDevice().waitForWindowUpdate(null, 2000); // Wait for main window getUiDevice().waitForWindowUpdate(null, 30000); } new UiObject(new UiSelector().description("New tweet")).click(); // Typing text for a tweet new UiObject(new UiSelector().className("android.widget.LinearLayout").instance(8)). setText("Awesome #Testdroid!"); // Tweeting! new UiObject(new UiSelector().text("Tweet")).click();

Espresso

Espresso is the latest Android test automation framework that got open-sourced by Google, making it available for developers and testers to hammer out their UIs. Espresso has an API that is small, predictable, easy to learn and built on top of the Android instrumentation framework. You can quickly write concise and reliable Android UI tests with it. It is supported on API level 8 (Froyo), 10 (Gingerbread), and 15 (Ice Cream Sandwich) and afterwards.
It’s quite reliable, synchronizing with the UI thread and fast because there is no need for any sleeps (tests run on same millisecond when an app becomes idle). But it does not have support for webviews as well.

Espresso code example:

public void testEspresso() { // Check if view with the text 'Hello.' is shown onView(withText("Hello.")).check(matches(isDisplayed())); // R class ID identifier for 'Sign in' - and click it onView(withId(getInstrumentation().getTargetContext().getResources() .getIdentifier("com.twitter.android:id/sign_in", null, null))).perform(click()); // R class ID identifier for entering username onView(withId(getInstrumentation().getTargetContext().getResources() .getIdentifier("com.twitter.android:id/login_username", null, null))).perform((typeText("username"))); // R class ID identifier for entering password onView(withId(getInstrumentation().getTargetContext().getResources() .getIdentifier("com.twitter.android:id/login_password", null, null))).perform((typeText("password"))); // R class ID identifier for clicking log in onView(withId(getInstrumentation().getTargetContext().getResources() .getIdentifier("com.twitter.android:id/login_login", null, null))).perform(click()); // Activate the text field to compose a tweet onView(withId(getInstrumentation().getTargetContext().getResources() .getIdentifier("com.twitter.android:id/menu_compose_tweet", null, null))).perform(click()); // Type the tweet onView(withId(getInstrumentation().getTargetContext().getResources() .getIdentifier("com.twitter.android:id/edit", null, null))).perform((typeText(”#Testdroid"))); // Tweeting! onView(withId(getInstrumentation().getTargetContext().getResources() .getIdentifier("com.twitter.android:id/composer_post", null, null))).perform(click()); }

Calabash

Calabash is a cross-platform test automation framework for Android and iOS native and hybrid applications. Calabash’s easy-to-understand syntax enables even non-technical people to create and execute automated acceptance tests for apps on both of these mobile platforms. Calabash’s tests are described in Cucumber and then converted to Robotium or Frank in run time. It supports about 80 different natural language commands (controllers), and new controllers can be implemented in Ruby or Java.

Calabash code example:

Feature: Login feature Scenario: As a valid user I can log into my app I wait for text "Hello" Then I press view with id "Sign in" Then I enter text "username" into "login_username" Then I enter text "password" into "login_password" Then I wait for activity "HomeTabActivity" Then I press view with id "menu_compose_tweet" Then I enter text "Testdroid" into field with id "edit" Then I press view with id "composer_post"

Appium

Appium is a mobile test automation framework (and tool) for native, hybrid and mobile-web apps for iOS and Android. It uses JSONWireProtocol internally to interact with iOS and Android apps using Selenium’s WebDriver. It supports Android via uiautomator (API level 16 or higher) and Seledroid (API level lower than 16), iOS via UI Automation, and mobile web as Selenium driver for Android and iOS.

One of the biggest advantages of Appium is that you can write your Appium scripts on almost any programming language (e.g. Java, Objective-C, JavaScript, PHP, Ruby, Python or C#, etc), freedom from having to select tools, compatibility across the most important platforms (Android and iOS), freedom from having to install and configure devices to test and more. Also if you are familiar with Selenium, then it’s easy for you to use Appium in mobile app testing. They use the same WebDriver and DesiredCapabilities is used in the same way. Configuring an application to run on Appium has a lot of similarities to Selenium.

Appium code example:

# wait for hello sleep(3) textFields = driver.find_elements_by_tag_name('textField') assertEqual(textFields[0].get_attribute("value"), "Hello") # click sign-in button driver.find_elements_by_name('Sign in')[0].click() # find the text fields again, and enter username and password textFields = driver.find_elements_by_tag_name('textField') textFields[0].send_keys("twitter_username") textFields[1].send_keys("passw0rd") # click the Login button (the first button in the view) driver.find_elements_by_tag_name('button')[0].click() # sleep sleep(3) # click the first button with name "Compose" driver.find_elements_by_name('Compose')[0].click() # type in the tweet message driver.find_elements_by_tag_name('textField')[0].send_keys(”#Testdroid is awesome!") # press the Send button driver.find_elements_by_name('Send')[0].click() # exit driver.quit()

Wrapping up

Comparing Android Testing Frameworks

Here we have listed top 5 testing frameworks for your daily Android builds, creation, and correction. Certainly, each of them has its pros and cons. Appium is good in testing both your Android and iOS versions at the same time. But if you are a loyal Android developer with only Android-version app, for instance, then using Robotium is not a bad idea too. Relying on Testdroid Recorder will definitely save your lots of time and money (It’s free!) in generating test scripts. Therefore, think about your testing need – functional testing, compatibility testing, UI testing, etc. – and pick up the right and best Android testing framework(s).

Many of you must know what London Selenium Meetup is. The good thing of LDNSE is that it provides an open platform and channel for Selenium users of all skill levels where they can network with others and learn plenty of technical topics like Agile Testing, Continuous Integration, etc.

The monthly meetup does not only include topics about Selenium, but also Appium, Mobile and Test Automation. And we are glad to give a talk on April’s meetup together with Dan Cuellar, creator of Appium, tonight at 7pm GMT+1.

Following Dan’s Introduction to Appium, Ville-Veikko Helppi, our Technical Product Manager will have a thorough presentation covering how web and mobile apps can be tested using Testdroid’s automated testing solution.

Though RSVPs have been closed, you can make full use of good resources on our website.

We will share the presentation slide deck soon. Stay tuned.

Dear Testdroiders,

Appium has been the number one smoking hot framework during the past quarters and there are no signs of this trend cooling down anytime soon. We’ve been providing Appium/Selenium support for Testdroid Cloud now for more than a year and significant portion of Testdroid Cloud test runs are Appium/Selenium runs nowadays. To make things even more convenient for our users, we’re about to introduce an awesome new feature that will make executing those test runs much easier and less error-prone – as you don’t have to play those desired caps all the time.

We recently released a new version of Testdroid Cloud and with this version, we’ve brought out a new feature that enables completely new way to execute Appium/Selenium tests on our devices. Basically, with this implementation users with Appium/Selenium scripts are not required to configure desired capabilities as they would do with current client-side execution. We call this implementation “Appium Server Side” execution that basically means that test scripts can run locally on our environment – you just upload your .APK or .IPA and test package to Testdroid Cloud and our our system takes care of proper configuration and manages all details for desired capabilities.

How it Works?

The application is uploaded to Testdroid Cloud the same way in both approaches. You can do it manually or use the Testdroid API to get your application file (APK or IPA) included in the project where test execution will take its place. Also, the test package (as a zip file including all test assets, script etc.) can be uploaded the same way as you would be doing with other test automation frameworks. Like the application file, you can push the test zip package through Testdroid API (as a test file) or use the manual web front-end.

Great news is that all the other Testdroid Cloud features work seamlessly with the server-side Appium and you can – for example – configure and create new device groups and run those tests within these device groups without any modifications to desired capabilities. This itself can save you hours from configuration time.

With client-side Appium you will need to configure those Testdroid desired capabilities and make sure everything is ready for execution on Testdroid Cloud. The test script will then go through and communicates with Appium Broker that passes all configuration details to Appium Server and then to the device (one device at time). The server-side Appium does this parallel with all selected devices, making the whole process much faster.

Desired Capabilities – And How to Get Rid of Those

Let’s look at things from both, localhost and Testdroid Cloud point of view. When you run any Appium tests, you configure desired capabilities accordingly and point Webdriver either to localhost (http://localhost:4723/wd/hub) or Testdroid Cloud (http://appium.testdroid.com/wd/hub). For example, if you use Testdroid Cloud you also add Testdroid Desired Capabilities for your test script – for example like this:

{ “testdroid_username”: “user@domain.com”, “testdroid_password”: “p4s$w0rd”, “testdroid_target”: “ios”, “testdroid_project”: “My First Project”, “testdroid_testrun”: “Test 1”, “testdroid_device”: “iPad Mini 7.0.4 A1432”, “testdroid_app”: “application.ipa” . . “app”: “com.bitbar.testdroid.BitbarIOSSample” }

There are few more options for configuring desired capabilities, so please check this if you want to know all details of it.

Behind the Scenes

Despite the usability is much more smooth with server-side Appium, from the infrastructural point of view both are pretty similar. The picture below illustrates what happens behind the scenes regardless of which way user is using it.

The things behind the scene are pretty much the same regardless of which approach you take. If you use client-side execution, you must configure those desired caps and our Appium Broker together with Appium Server takes care of running tests one by one on those devices. With server-side Appium execution, our system takes care of simultaneous test runs on those devices you have selected. There aren’t any limitation of how many devices you can use simultaneously so even hundreds of different variants if you want.

The Benefits of Server-Side Appium

First of all, one the most significant improvement provided with this server-side Appium implementation is the ability to execute Appium tests parallel on real devices. As Appium was originally designed to provide tight relation to emulator or one device at time, we’ve now extended the possibility to use it across even hundreds of real devices simultaneously.

Secondly, users do not need to configure any of those devices on desired capabilities, but all can be done with Device Group creation and configuration by dragging and dropping desired devices into your device groups. This eliminates the variety of errors that typically came with configuring devices in script.

Also, all test runs are now easy to monitor through real-time views at Testdroid Cloud. At a glance, you will see how test runs are doing, get instant feedback on failures, and possibility to maintain all test results (screenshots, logs, performance stats etc.) and file assets under your projects.

Want an access to Server-Side Appium? Please let us know (sales at bitbar dot com) and let us know in comment section what you think about this feature! We’re looking forward for your comments and questions.

Appium是目前最受欢迎的同时支持Android和iOS测试的开源测试框架之一。从支持Appium开始到现在的一年多内，我们已经看到有很多开发人员与测试人员在Testdroid Cloud上执行了Appium测试。今天，我们将为你介绍一个Appium测试执行的新功能，使测试人员更简单地使用Appium执行应用程序的测试，并且保证了更高的测试结果的正确率，也简化了你总需要配置Desired Capabilities的繁琐事项。

在上周发布的最新版本的Testdroid Cloud中，就包含了一种全新的Appium测试方式，即在Testdroid Cloud的所有设备上同时执行Appium测试。在前几个版本中，客户需要在 “客户端执行测试（Client-Side Appium Execution）” 前配置Desired Capabilities，而此次的更新将简化客户配置的工作流程。我们将这个新的功能命名为 “服务器端测试执行 (Server-Side Appium Execution)”。顾名思义，用户可以非常简单地在我们的云端执行测试脚本 – 你只需将.apk或是.ipa文件以及你的Appium测试脚本分别上传到Testdroid Cloud的云端，我们的系统将会进行正确合适的配置并且管理所有Desired Capabilities的细节。

这是如何运作的？

首先，你可以选择通过Testdroid Cloud的网页端或是用Testdroid API将你的应用程序(.apk或是.ipa)以及含有测试脚本的.zip文件（包括所有测试用例，脚本等）上传至Testdroid Cloud。

好消息是，Testdroid Cloud其他所有功能都能和服务器端（Server-Side）Appium无缝链接。比如说，在不修改Desired Capabilities的前提下，你可以创建新的设备组并对该设备组中的设备执行测试。通过这种方法，你就避免了原本需要对Desired Capabilities配置的过程，并省下了大量的时间。

相反，在客户端（Client-Side）执行Appium测试时，整个流程就没有那么简单。首先，你需要配置那些与Testdroid相关的Desired Capabilities，并需要确保所有先决条件都已满足于在Testdroid Cloud上执行测试。接着，你的测试脚本需要通过Appium Broker并与之交互，然后Appium Broker会将所有的配置细节传递给Appium服务器，并以每次一台设备的方式，传递给测试设备。而在服务器端执行Appium测试时，所有设备将以并行的方式在同一时间配置完成，大大减少了时间，加快了整个流程。

当然，无论是用客户端（Client-Side）执行还是服务器端（Server-Side）执行Appium，你都可以在Testdroid Cloud项目中找到所有的测试资源（测试脚本，测试截图，测试日志等），但如果你在服务器端（Server-Side）执行Appium测试，你还可以使用Testdroid Cloud的其他所有功能给你的测试带来更多的利益。

如何避免配置Desired Capabilities？

首先，我们先来了解一下本地主机端和Testdroid Cloud端。在你执行Appium测试时，你需要相应地对Desired Capabilities进行配置。另外，你需要定义WebDriver的目的地URL – 本地主机端（http://localhost:4723/wd/hub）或者是Testdroid Cloud端（http://appium.testdroid.com/wd/hub)。举例来说，如果你使用Testdroid Cloud，你需要在你的测试脚本中添加Testdroid Desired Capabilities，具体如下：

除此以外，你还可以在这里找到更多配置Desired Capabilities的方法。

具体的流程

虽然在服务器端（Server-Side）执行Appium测试更方便，但客户端（Client-Side）执行和服务器端（Server-Side）执行两种方式在基础设备需求上是比较相似的。而且，无论你使用哪种方式，Appium测试都会如下图所示的流程执行。

如果你在客户端（Client-Side）（Client-side）执行测试，你必须配置Desired Capabilities。然后，Appium服务器和我们的Appium Broker会以每次一台设备的方式依次进行测试。而在服务器端（Server-Side）执行Appium测试时，我们的系统会在所有的设备上同时进行测试。更值得一提的是，同时进行测试的设备数量是没有限制的，也就是说你可以在数百台设备上同时进行Appium测试。

服务器端执行Appium测试带来的优势

服务器端（Server-Side）测试方法带来的第一个重要的提升是用户可以在真机上执行并行的Appium测试。由于最初的Appium使用对象是模拟器上或只是以每次一台设备的测试方法执行测试，而我们Testdroid在原有的基础上扩展了Appium在数百台真机上同时执行测试的能力。

第二，用户不需要配置任何设备的Desired Capabilities。用户只需在Testdroid Cloud上创建设备组，然后将需要测试的设备陆续添加到新创建的设备组。这种方式既简便，且减少了因为编辑脚本而产生的不同类型的错误。

第三，在Testdroid Cloud上执行测试可以让用户即时浏览测试状况。也就是说，你可以查看到测试执行的进度，即时的错误反馈，以及保留和查阅所有测试项目，测试脚本和测试结果（测试截图，测试日志，性能数据等）。

如果你对服务器端执行Appium测试感兴趣的话，可以联系我们 sales (at) bitbar (dot) com。

Dear Testdroiders,

We hosted a webinar about Appium and our new feature – server-side execution – this Tuesday. The feedback I’ve got has been truly awesome and we’d like to share those materials with you.

We recently released a new version of Testdroid Cloud and in this version we’ve introduced a feature that enables running Appium tests on our side, server-side. Basically, with this Appium/Selenium users are not required to configure desired capabilities as they would do with current client-side execution. We call this implementation Appium Server-Side execution that basically means that test scripts can run locally on our environment – you just upload your .APK or .IPA and test package to Testdroid Cloud and our our system takes care of proper configuration and manages all details for desired capabilities.

If you are interested to know more about Appium Server-side execution, you might also want to check out this blog.

You can go through the slides on Slideshare:

And on-demand video recording can be found here:

In case you want to try it out with the example project that was demonstrated in this webinar, please let me know and I’ll be happy to share the project with you on Testdroid Cloud.

Enjoy the on-demand webinar!

Dear Testdroiders,

Many of you are using image recognition or other types of visual/element/object character recognition implementations for testing. Especially for mobile games this is very handy as all of the graphical content is based on OpenGL ES or directly coming from UI engines that are difficult to recognize by test automation scripts.

In this example, I’ll be walking you through of basic example how to use image recognition for mobile game testing and what sort of assets and test script you need for that.

Introduction

In this example, we’ll be using Hill Climb Racing (directly downloaded from Google Play) for Android. For testing, we use a set of real Android devices (from different OEMs, different OS versions, different form factors etc.) and compare those results. Also, we’ll be using Appium test automation framework with our own image recognition/object character recognition feature implemented for Testdroid. And to make things easy and straightforward, we’ll be using server-side Appium execution so that minimalistic configurations are required from test automation perspective.

File Structure

As it comes to putting the zip file together with all required test assets, the basic file structure is as illustrated in the picture above. The three core files for test package – pom.xml (for maven including the build specific stuff), testdroid.properties (with Testdroid and Appium specific URL information) and run-tests.sh (shell script for execution) – are the actual configuration for the test infrastructure. Image files (under queryimages/hc) are the png files that certain visual elements from Hill Climb Racing and will be used to define certain area and action for test script.

The Test Script and Images as Visual Identifiers

With the help of Akaze and OpenCV, we can quickly build functions that compare the screen and graphics content in .png files. The idea is to give visual assets (.png files) as they are and test script will compare and perform action whenever those visual assets are shown on screen. For example, with some of the test frameworks the timing and delays have been problematic. With this sort of implementation you don’t need to implement delays or any timing logic for scripts but instead script can wait until certain visual assets are shown on screen.

    public Point[] findImageOnScreenAndSetRotation(String image) throws Exception {
        int retries = 5;
        Point[] imgRect = null;
        
        while ((retries > 0) && (imgRect == null)) {
            if (retries < 5) {
                log("Find image failed, retries left: " + retries);
            }
            takeScreenshot(image + "_screenshot");

            //this will identify the rotation initially
            imgRect = findImage(image, image + "_screenshot", "notSet"); 
            retries = retries - 1;
        }
        assertNotNull("Image " + image + " not found on screen.", imgRect);
        return imgRect;
    }

The above function is used to determine if the screenshots need to be rotated and by what degree. This will also set the rotation for screen and images will be recognized with the proper orientation. Next we will perform a simple click for any identified visual asset described in .png file:

For example, the image with “more button” content is compared here with the screen content and if suitable element is spotted, the click is performed. The function is called with the right coordinates and time how long button/visual element should be pressed is given as a parameter.

    public void tapImageOnScreen(String image) throws Exception {
        Point[] imgRect = findImageOnScreen(image);
        //imgRect[4] will have the center of the rectangle containing the image

        if (automationName.equals("selendroid")) {
            selendroidTapAtCoordinate((int) imgRect[4].x, (int) imgRect[4].y, 1);
        } else {
            driver.tap(1, (int) imgRect[4].x, (int) imgRect[4].y, 1);
        }
        sleep(SHORT_SLEEP);
    }

    public void selendroidTapAtCoordinate(int x, int y, int secs) throws Exception {
        TouchActions actions = new TouchActions(driver);
        actions.down(x, y).perform();
        sleep(secs);
        actions.up(x, y).perform();
    }

In the game-play, we will use selendroidTapAtCoordinate directly. As a parameter, the png name is given and the time how long the key will be pressed. The test script has advanced to the actual game-play stage and we really only have three possible (and clickable) items on screen. With this configuration, the ‘gas’ pedal is pressed down for 15 seconds and then released:

As you might now what happens in this game after car runs out of fuel, the game-play ends and social media sharing view with scoring etc. will be shown. Before that, the test script checks if test passed or failed – and this is also done based on images shown on the screen.


    public Point[] findImageOnScreenNoAssert(String image, int retries) throws Exception {
        Point[] imgRect = null;
        
        while ((retries > 0) && (imgRect == null)) {
            if (retries < 5) {
                log("Find image failed, retries left: " + retries);
            }
            takeScreenshot(image + "_screenshot");
            imgRect = findImage(image, image + "_screenshot");
            retries = retries - 1;
        }
        return imgRect;
    }

Sudden Notification Messages/Distractions for Test Scripts

The typical “issue” with visual image recognition on Android is those notification “pop-ups” that may disturb the execution of the script. However, those notifications can be also visually identified as a png and included in the script. For example, if the following notification message comes up your script can simply do the click on OK and everything keeps going on:

Reviewing the Test Results

As this sort of test can be executed simultaneously on yet hundreds of devices, you will get screenshots, logs and Appium logs, test data etc. as a result at Testdroid Cloud. Let’s take the screenshots and comparison of those first. Screenshots are taken along the way when test script advances, those are used to compare the test script progress, perform actions and to define how quickly test script advances.

Another very important part of the testing is the performance. This is especially the case with mobile games and we’ve integrated Gamebench as part of the test execution at Testdroid Cloud to provide comprehensive and accurate performance results.

Then, after the execution on variety of different devices, the logs can be very instrumental for understanding if app/game has any problems or potential issues and how those can be fixed. We gather all important metrics (from logs) under Testdroid Cloud projects and all of those can be inspected after the test run with correct stamped time/date:

Finally, with the test automation and possibility to do simultaneous test runs on variety of different devices, users get PASS/FAIL status from each device. This is great overview of which devices might require special attention as you fine-iron your app/game for market/users:

Want Access to Full Source Code?

If want to try out how the image recognition with this described setup works on variety of real devices on cloud, let me know and I’ll be happy to share this project and source code with you. You can reach me via ville-veikko dot helppi at bitbar dot com.

Happy testing!

Droidcon is the annual global conference series that Android enthusiasts can’t miss out. Following Google I/O this year, Droidcon Berlin surely is the place for European Android developers to learn and share every bit of new technology in Android development. And the great news is that Testdroid will be present throughout the whole event, on June 3-6.

droidcon_picture

Different than last year, this time we will raise an interesting topic on June 3rd during the barcamp session. And if possible, we will hold a full session to go through the capabilities and latest technologies of Testdroid.

If you are interested in getting an early contact and meeting us at the event, please do get in touch with Robert Seege (robert.seege@bitbar.com) or Meirav Minitzer (meirav.minitzer@bitbar.com) to make an appointment. In case you are not able attend the event, nevertheless, we have the following treats for you.

Parallel Appium Test Runs on Real Devices

Since the support of Appium 1.0 to all of our products, thankfully, the huge need of Appium has pushed us to advance further – with achievement on running parallel tests with Appium on real devices.

Image Recognition for Mobile App/Game Testing

The technology of image recognition is nothing new in computing world, but something emerging in the landscape of mobile app/game testing. Along with the advantages of Appium, image recognition makes mobile app/game testing more efficient and effective than ever.

Dear Testdroiders,

We’ve been discussing about different angles and flavors of Selenium / Appium in mobile app testing, and I wanted to write down a brief blog about how your can use your existing Selenium tests also in mobile app/game context. Many of you – or some web developers out there – have used Selenium for desktop, PC-powered web testing 7-8 years and basically today the very same logic and functionality is provided for creating those tests for mobile apps and games today.

Mobile Automation: Android Automation & iOS Automation Testing Tool

Some people have been confused when we talk about Appium and Selenium in the same sentence. In a nutshell: Appium is a mobile test automation framework (plus inspecting tool) for native, hybrid and mobile-web apps for iOS and Android. It uses JSONWireProtocol internally to interact with iOS and Android apps using Selenium’s WebDriver. In few words, what works with Selenium, likely works with Appium as well. This is because the framework is basically a wrapper that translates Selenium Webdriver commands into UI Automation (iOS) or uiautomator (Android, API level 17 or newer) or Selendroid (Android, API level 16 or lower) commands commands depending on the device type.

Appium has basically two different ways to use it for getting your app, game or some web piece tested. For example, many of popular Android mobile apps/games get tested with Appium Client-Side tests. This is the original idea of how test automation should work – and probably slightly even primitive, as you can’t easily handle parallel test runs or not at least start those. This also requires some configurations every now and then as you need to set up Selenium desired capabilities. Better option – and one to help you to get rid of those desired capabilities – is to use Appium Server-Side tests. In this case, you don’t run test from your localhost, but you just upload your .APK or .IPA and test package to Testdroid Cloud and the system takes care of a proper configuration and manages all details for desired capabilities.

How Are These Approaches Different – How those Work?

Working with Selenium – and Selenium testing – is very easy and you will learn it quickly. First, something else: take a look at Selenium IDE, which is a Firefox plugin and records and plays back user interactions with the browser. Use this to either create simple scripts or assist in exploratory testing. It exports those scripts for WebDriver. Then, as said, the app is uploaded to Testdroid Cloud the same way in both approaches. You can do it manually by logging in, select project, and upload the file – or you can use the Testdroid API to get your application file (APK or IPA) included in the project where test execution will take its place. Also, the test package (as a zip file including all test assets, script etc.) can be uploaded the same way as you would be doing with other test automation frameworks. Like the application file, you can push the test zip package through Testdroid API (as a test file) or use the manual web front-end.

Also, worth of mentioning, but all the other Testdroid Cloud features work seamlessly with the server-side Appium and you can – for example – configure and create new Android or iOS device groups and run those tests within these device groups without any modifications to desired capabilities. This itself can save you hours from configuration time – and get your tests running earlier as you don’t need to play with those desired caps. To get your tests running, you can select native Android project or more specifically Selenium Android and/or Selenium iOS for project types you should be seeing under project configuration.

Both approaches will allow you to reserve all test assets (scripts, screenshots, logs etc.) under your Testdroid Cloud projects, but with server-side Appium all other features are available and easy to use. It’s worth of mentioning that with the release of Selenium 2.0, the Selenium Server also included built-in Selenium Grid functionality which can help you to scale / distribute tests on several machines ( helps in local parallel execution).

Desired Capabilities – And How to Get Rid of Those

Let’s look at things from both, localhost and Testdroid Cloud point of view. When you run any Appium / Selenium tests, you configure desired capabilities accordingly and point Webdriver either to localhost (http://localhost:4723/wd/hub) or Testdroid Cloud (http://appium.testdroid.com/wd/hub). For example, if you use Testdroid Cloud you also add Testdroid Desired Capabilities for your test script – for example like this:

There are few more options for configuring desired capabilities, so please check this if you want to know all details of it.

What Happens to Your App on Our Server

In addition, the usability being light-year ahead on server-side Appium, those two pretty similar from the infrastructural point of view. This picture below explains what happens behind the scenes regardless of which way you are using it.

Why Appium Server-Side – and Why Should You Take A Look At It

Clearly, one the most significant improvement provided with this server-side Appium implementation is the ability to execute Appium tests parallel on real devices. As Appium was originally designed to provide tight relation to emulator or one device at time, we’ve now extended the possibility to use it across even hundreds of real devices simultaneously. Furthermore, as Appium provides cross-platform approach for Android automation testing and iOS automation testing you can also run tests on both platforms, at the same time (different device groups though).

Want an access to Server-Side Appium? Please let us know (sales at bitbar dot com) and let us know in comment section what you think about this feature! So weigh in with comments – we’re looking for those!

Dear Testdroiders,

Appium has quickly become one of the most prominent test automation framework for mobile app, game and web testing. To celebrate this great framework going forward, people developing it further (yes, we highly appreciate the great work this ecosystem of talented and dedicated engineers are doing!) and people adopting it with a growing pace, we’d like to share our knowledge, some best practices as well as bunch of tips and tricks how you can really take all out of it. So…

We’re glad to announce a blog series that will run the next 8 months including 37 unique blogs, sharing a lots of great and relevant information for all expertise levels of Appium users – from newbies/beginners to tech-savvy test automation folks that already use it. Stick with us – it’ll be surely an awesome ride!

What is Appium – and Why Should You Care?

As defined at Appium.io, Appium is an open-source tool/framework for automating native, mobile web, and hybrid applications on Android and iOS platforms. It works well on native apps – the ones that are written using the iOS or Android SDKs, mobile web apps that are accessed using a mobile browser as well as hybrid apps that are utilizing webview and are wrapped inside your app. For example, Appium is also very popular among mobile game developers who typically use some advanced techniques for testing input-driven mobile games even running on two different platforms at the same time and by the same script. Sounds good? More to come…

What Will Be Included?

We’ll try to offer the fresh look at this framework. There are many different blogs out there that focus on Appium and provide tips and tricks for variety of topics. In this blog series, we’ve divided topics – and tips and tricks for those – based on some clear and related topics, such as how to setup and configure Appium, what things you should consider for different platforms, how to use real devices and browsers efficiently in the process, what complementary tools to consider, and many other topics. We also define what different concepts are, what those include and what will (most likely) work your app the best.

Meanwhile, we’ve provided some insight to Appium, different flavour of it and how to efficiently run tests on real devices and browsers – take a look! Also, here is one great webinar to get yourself tuned for Appium goodness:

The Schedule and Content

Ok, here is the preliminary schedule and list of content for each blog:

30 September – How to Get Started, Setup and Run Your First Tests
2 October – Using Python to Configure and Upload Your Apps and Tests in Cloud
5 October – Java Integration with Real Devices on Cloud Service
7 October – Mastering C# for Your Tests, Setup and Some Basics
12 October – Ready to Rumble with Ruby
19 October – Tips and Tricks Using JavaScript (and some other languages)
26 October – How to Master Appium from Command Line
2 November – How to Take Screenshot on Failure
9 November – How to Automatically Dismiss Dialogs and Autoaccept Alerts
16 November – Catch Errors with Try-Catch Block to Avoid
23 November – BONUS!
30 November – AppiumBroker: New Capability for Specifying Test Timeouts
7 December – Appium Inspector and Additional Tools
14 December – Tips and Tricks to Generate Appium Scripts
21 December – The Best Kept Secrets – Commands/Calls
28 December – Appium Calls – findelement #1
4 January – Appium Calls – findelement #2
11 January – xpath for Elements
18 January – Element Commands #1
25 January – Element Commands #2
1 February – Desired Capabilities
8 February – BONUS!
15 February – Get Rid of Desired Capabilities
22 February – Client vs. Server-Side Appium
29 February – How to Automatically Retry Failed Tests
7 March – How to Test Websites on Safari
14 March – Appium Makes These Components Obsolete
21 March – Appium and API to Real Devices
28 March – Web: Real iOS Devices and Safari
4 April – Web: Chrome and Android Devices
11 April – Web: Other Browsers
18 April – Best Practices for Non-Native Apps
25 April – Image Recognition with Appium – How To Setup
2 May – Image Recognition with Appium – Assets, Tests, Scripts
9 May – Image Recognition and Best Practices for Appium
16 May – Network Infrastructure and How to Improve It
23 May – BONUS!

Interested in Specific Topic or Want Answers to Some Appium Problem?

We’re happy to hear your thoughts on our blogs, how they could serve you even better and what sort of information you’re specifically looking for. Naturally the above highlighted topics are preliminary and if you are interested in some other – but important – topic, please let us know and weigh in a comment below.

Let’s get this blog series started!

This is the first blog in our 37 Things You Should Know About Appium blog series, and we’ll naturally start with the basic things, getting environment set up, Appium pieces installed and running on your machine. Depending on which programming language you are using, we’ve tried to provide here everything, step-by-step, for those 4 main languages.

Getting Started

To get started, first download our Appium example. The example is available in C#, Python, Java and Ruby, and we’ll be using these as a basis for next examples. Okay, then depending on which programming language you will be using, select the appropriate client library as follows.

Most of the Testdroid examples are located in our Github repositories. To make most out of the existing samples you should have Git installed. If you are new to Git, there is a very good guide on how to install Git on popular operating system here. Below is the abbreviated version of the installation procedures.

Mac OS X

Download the latest git command line tool from http://git-scm.com/download/mac install it using normal Mac installation procedure.

Linux

Use the following command to get Git installed on your Linux machine:

$ sudo apt-get install git

Windows

The easiest and most straight forward way is to install the Github Windows application.

Python

Mac OS X and Linux

Our Python samples have been created with 2.7.x version in use, so best compatibility can be expected with the same. Check that newest Python 2.7.x version is installed by using the following command:

$ python --version

If not installed, you can install Python by using the following commands:

Linux: $ sudo apt-get install python2.7
OSX: $ brew install python

Brew is a handy package manager tool, similar to apt-get. If you don’t have it, check the brew website for its one-liner installation.
Then, check if ‘pip’ module is installed. Use the following command:

$ pip --version

If ‘pip’ is appropriately installed, you’ll see something like this:

pip 1.5.6 from /Library/Python/2.7/site-packages (Python 2.7)

If not, use the following command to install it:

Linux: $ sudo apt-get install python-pip
OSX: $ sudo easy_install pip

Next, install the Selenium module for Python:

$ pip install selenium

And finally, verify that Selenium got installed:

$ pip list | grep selenium

Windows

Ensure that newest Python 2.7.x version is installed. Go to command line and use the following:

> python --version

If Python 2.7 or newer is not installed, download and run the setup from Python’s download center. To add Python into environment variables, go to Windows “System properties” → “Advanced System Settings” → “Environment Variables” → “System Variables” → choose “Path” and press “Edit…” and then insert (assuming you have installed Python in the default location) ;C:\Python27;C:\Python27\Scripts at the end separating each path with semicolon ; . Make sure to re-launch the command prompt to bring new environment variables into effect.

Then, check whether Python’s pip module is already installed:

> pip --version

Install pip if it’s not already (we assume here that you have cURL installed. If not, check this out):

> curl https://raw.github.com/pypa/pip/master/contrib/get-pip.py > get-pip.py
> python get-pip.py
> del get-pip.py

Install Python’s Selenium module:

$ pip install selenium

Java

Appium testing supports writing tests in multiple programming languages, including Java. Testing can be done against native or hybrid apps or responsive web pages on both iOS and Android devices. Only the test setups differ.

With Java, things are pretty simple and you only need to configure your test file accordingly. You can use SampleAppium(iOS)Test.java as an example/template.

In case you don’t have an IDE with Maven included and would like to launch the example from command line, you will need to make sure that Maven is properly installed. Here’s a link to the installation instructions.

C#

Windows

Launch the AppiumTest.sln on Visual Studio and make sure that NUnit Test Adapter is installed through the Extension Manager. Use Test Explorer to run your tests.

Linux/OSX

First, install Monodevelop for C# support. Then, download dependencies using Nuget:

$ nuget install Test123/packages.config -OutputDirectory packages

To build the package, simply use the following command on correct path:

$ xbuild

Ruby

First, install the latest stable release of Ruby:

$ curl -sSL https://get.rvm.io | bash -s stable
$ rvm install ruby

Then, make sure RVM is using the correct Ruby by default:

$ rvm list
$ rvm --default use 2.1.1

In case you have an old Ruby/RVM, you can upgrade those with the following commands:

$ rvm get head
$ rvm autolibs homebrew
$ rvm install ruby

Check that it’s installed properly by printing out the Ruby version:

$ ruby --version

Update RubyGems and Bundler:

$ gem update --system
$ gem install --no-rdoc --no-ri bundler
$ gem update
$ gem cleanup

Check that RubyGems is >= 2.1.5

$ gem --version

Run bundler at the Ruby example directory to install dependencies:

$ bundle install

All Programming Languages: How to Configure Testdroid Specific Settings

If you used any of those example files as a template, add your Testdroid user credentials in this script. You can also use environmental variables TESTDROID_USERNAME and TESTDROID_PASSWORD to get your credentials used:

String testdroid_username = env.get("TESTDROID_USERNAME");
String testdroid_password = env.get("TESTDROID_PASSWORD");

Or alternatively, you can edit testdroid_username and testdroid_password in your source file:

capabilities.setCapability("testdroid_username", 'john.doe@bitbar.com');
capabilities.setCapability("testdroid_password", 'secretPassword123');

If you are new with desired capabilities or if you are looking for more information on how to use those efficiently, take a look at this article.

Also, if you want to run tests against your application, make sure to change a file path to your application binary (whether you are running against APK or IPA):

private static final String TARGET_APP_PATH = "../../../apps/builds/BitbarSampleApp.apk";

In addition, there are lots of possible ways to configure your test run for our devices. In order to do this, you need to configure those desired capabilities. We’ll get the bottom of this later in the blog series, but the current examples are as follows:

DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability("platformName", "Android");
capabilities.setCapability("testdroid_target", "Android");
capabilities.setCapability("deviceName", "Android Device");
        
capabilities.setCapability("testdroid_project", "LocalAppium");
capabilities.setCapability("testdroid_testrun", "Android Run 1");
        
// See available devices at: https://cloud.testdroid.com/#public/devices
capabilities.setCapability("testdroid_device", "Samsung Galaxy Nexus GT-I9250 4.2.2"); 
capabilities.setCapability("testdroid_app", fileUUID);

In this Java example, you only really need to configure testdroid_username and testdroid_password, since sample application upload is included.

Running Your First Test

Before the test can start you need to upload you application to Testdroid Cloud. This can be done either via API or you can do it manually as well.

Python

To upload your app file (either APK or IPA) to Testdroid Cloud, open and configure the upload.py script. As we walked it through, you only need to configure your username (email) and password that you registered with to Testdroid Cloud. Also, you need to set the full path to your mobile app. This can be an Android or iOS application. Then execute this:

$ python upload.py

To run a test:

$ python testdroid_android.py

Java

You can run test from your IDE or directly from command line using Maven:

> mvn clean test -Dtest=SampleAppiumTest

or to be more precise:

> mvn clean test -Dtest=com.testdroid.appium.android.sample.SampleAppiumTest

or run all the tests:

> mvn clean test

C#

To run tests, either launch them in Visual Studio via the Text Explorer or use the nunit console command:

$ nunit-console Test123/bin/Debug/TestdroidAndroidSample.dll

Ruby

Run the tests with rspec:

$ rspec testdroid_android.rb

Conclusion

There’s a myriad of different ways you can setup, build and run your Appium tests. The ones provided here were just one example of how things can be done. Stay tuned – more good stuff about Appium is coming out in few days!

Happy Appium Testing!

This is the second blog of our 37 Things You Should Know About Appium blog series, and this time we’ll continue with some basic things around using Python with Appium. For starters, we’ll take a look at basic examples of how to use Python and write few lines of code to get your applications automatically uploaded for Testdroid Cloud to be executed on real Android and/or iOS devices.

If you followed through the first blog where we walked through the basic setup, installation of Python, and some other things, you can continue with the basic code example. If you don’t have Python properly installed or you need to check whether everything is ready go to forward, read this blog.

Python is a worthwhile script language choice for those who want to get down to business quick and simple. The syntax is really easy to learn and implement and there isn’t typically a huge cluster of different files required. Usually when I want to test some Appium feature, which I haven’t touched before, I turn to python to check it out.

Creating and Running an Upload Script

First of all, you should have an account in Testdroid Cloud to get your tests running. You can create account here for free and use it as much as you want, regardless of what service level (FREE or PAID) you’ll end up using.

Before the actual testing can happen, you need to upload you mobile application to the cloud. The app can be in form of APK or IPA – and to get you an effective start, we’ll create a brief upload script here. The upload script example is also available at Github. For IPA files, please also make sure they’re properly built for use in Testdroid Cloud.

import requests
import base64

username = 'USERNAME'
password = 'PASSWORD'
upload_url = 'http://appium.testdroid.com/upload'
myfile = '../../../apps/builds/Testdroid.apk'

def build_headers():
  return { 'Authorization' : 'Basic %s' % base64.b64encode(username+":"+password) }

files = {'file': ('Testdroid.apk', open(myfile, 'rb'), 'application/octet-stream')}
r = requests.post(upload_url, files=files, headers=build_headers())
print r.text

Configure your username, password and myfile attributes with your Testdroid Cloud login credentials, and file path for your application (this can be APK or IPA).

Now, you can run the upload script using the following command:

$ python update.py

As an output you’ll get something like this:

{"status":0,"sessionId":"99becd25-3183-4c7f-998e-84ef307de7a5","value":{"message":"uploads successful","uploadCount":1,"rejectCount":0,"expiresIn":1800,"uploads":{"file":"99becd25-3183-4c7f-998e-84ef307de7a5/test.apk"},"rejects":{}}}

From this response message you need to store the application’s ID and file name in Testdroid Cloud. The above example is '99becd25-3183-4c7f-998e-84ef307de7a5/test.apk'.

Creating and Running a Test Script

Generic Settings

There are some common settings that you need to set in all scripts regardless of the app type that you are testing. Each testdroid_*.py file needs to be updated with the appropriate values. For this example, here are all the values that you need to edit:

screenshotDir – where should screenshots be stored on your local drive
testdroid_username – your email that you registered with to Testdroid Cloud
testdroid_password – your Testdroid Cloud password
testdroid_project – the name of the project you want to use in Testdroid Cloud. Each project must have a unique name, which can also be modified later. The project will be automatically created, if it doesn’t already exist in cloud.
testdroid_testrun – name of this test run. Test run names can also be modified afterwards and they don’t have to be unique.
testdroid_app – the location of the app you uploaded to cloud. Eg. if you uploaded your app using the upload.py script this would look like ’99becd25-3183-4c7f-998e-84ef307de7a5/test.apk’. Alternative way to this is uploading the app through the cloud portal and then use the value ‘latest’ instead of an id path.

Native Android Specific Settings

You can use our template file as an example: testdroid_android.py from Github.

In order to configure this script for your testing needs and for your own app, you can edit two additional capabilities as follows:

appPackage – Java package of the Android app you want to run
appActivity – Activity name for the Android activity you want to launch from your package. Typically this is the main activity.

Native iOS Specific Settings

You can use our template file as an example: testdroid_ios.py from Github.

NOTE! In addition to other configurations and desired capabilities, Appium requires a bundle id to be included in the test script:

bundleId – this is your application’s unique name. For example, it can be used in your Python test script as follows:

'bundleId': 'com.bitbar.testdroid.BitbarIOSSample'

An Example Template for Desired Capabilities and Setup

You can find all the template scripts from our Github repository. There are examples for Android, iOS, Safari, Chrome, and additionally Android hybrid for hybrid apps.

Okay, let’s look at a concrete Python test script example then and what goes in the code:

Imported libraries, bundles and other libraries/dependencies:

import os
import sys
import time
import unittest
from time import sleep
from appium import webdriver
from device_finder import DeviceFinder
from selenium.common.exceptions import WebDriverException

Next, let’s create a class with the device, login and configuration specific desired capabilities. For example:

class AndroidTest(unittest.TestCase):
    def setUp(self):
    self.screenshotDir = os.environ.get('TESTDROID_SCREENSHOTS') or "/absolute/path/directory"
    testdroid_url = os.environ.get('TESTDROID_URL') or "https://cloud.testdroid.com"
    testdroid_username = os.environ.get('TESTDROID_USERNAME') or "user@email.com"
    testdroid_password = os.environ.get('TESTDROID_PASSWORD') or "password"
    appium_url = os.environ.get('TESTDROID_APPIUM_URL') or 'http://appium.testdroid.com/wd/hub'

To choose a device from Testdroid Cloud, you can search for the desired model from our Device Groups page. Once you found the model you want to use, simply copy the full name of the device to testdroid_device variable.

You can also use the DeviceFinder class to automatically find an available freemium device for your test run at Testdroid Cloud:

deviceFinder = None
testdroid_device = os.environ.get('TESTDROID_DEVICE') 

deviceFinder = DeviceFinder(username=testdroid_username, password=testdroid_password, url=testdroid_url)
if testdroid_device == "":
  # Loop will not exit until free device is found
  while testdroid_device == "":
        testdroid_device = deviceFinder.available_free_android_device()

Then, to choose correct automation backend for the server, you need to fetch the API level of the device which will be used for a test run:

apiLevel = deviceFinder.device_API_level(testdroid_device)

Finally, let’s take a look at desired capabilities configuration with your Python script.

desired_capabilities_cloud = {}
desired_capabilities_cloud['testdroid_username'] = testdroid_username
desired_capabilities_cloud['testdroid_password'] = testdroid_password
if apiLevel > 17:
  desired_capabilities_cloud['testdroid_target'] = 'Android'
else:
  desired_capabilities_cloud['testdroid_target'] = 'Selendroid'

desired_capabilities_cloud['testdroid_project'] = os.environ.get('TESTDROID_PROJECT') # eg.'Demo'
desired_capabilities_cloud['testdroid_testrun'] = os.environ.get('TESTDROID_TESTRUN') # eg.'MyTest'
desired_capabilities_cloud['testdroid_device'] = testdroid_device
desired_capabilities_cloud['testdroid_app'] = 'sample/BitbarSampleApp.apk'
desired_capabilities_cloud['platformName'] = 'Android'
desired_capabilities_cloud['deviceName'] = 'Android Phone'
desired_capabilities_cloud['appPackage'] = 'com.bitbar.testdroid'
desired_capabilities_cloud['appActivity'] = '.BitbarSampleApplicationActivity'

One more thing! Let’s initiate and set up WebDriver:

desired_caps = desired_capabilities_cloud;
self.driver = webdriver.Remote(appium_url, desired_caps)

Tips!

You can set Environment Variables to store all the desired capabilities that won’t be changing often. This logic is already included in the sample scripts so all you need to do is set the variables! For example, to set your Testdroid Cloud username, simply apply TESTDROID_USERNAME=user@testdroidcloud.com to your environment and the script will then find it with the os.environ.get() function. If the variable doesn’t exist, the script will move on to the next declaration after the ‘or’ key word.

Playing around with the Android appPackage and appActivity values may be useful when creating tests to automate specific activity screens of your app. There is also a command to launch another app while one is already being tested! This functionality is available for Android only though.

Happy Appium Testing (and weekend)! We’ll be continuing with a new topic on Monday!

We discussed about the Python setup for Appium on Friday and it’s time to move forward and see how things are getting done on Java. Furthermore, we’ve been covering the basic setup, installing all required bits and pieces, and preparing the environment for the first test runs in our Appium blog series.

The beauty of Java is in its extensibility and long existing history. Most developers these days have at least some experience in Java, which makes it easy to approach. You should check the Appium Java-client’s README, if you haven’t already!

Before you attempt to launch the Testdroid Java samples, remember to check that all the desired capabilities are correct. It may be a good idea to also set as many capabilities in Environment Variables as possible to reduce the amount of editing when creating additional test scripts. Frankly, this will save a lot of time and also makes your test run launching a way more robust (no typos or flaky issues with the script config).

TIP! Put as many desired capabilities in Environment Variables as possible. It saves time and reduces errors due wrong configuration or typos in desired caps.

Similar to the Python sample, our Java implementation uses the Environment Variables for capabilities such as testdroid_username, testdroid_password, app file location and Appium server:

private static final String TARGET_APP_PATH = "../../../apps/builds/BitbarSampleApp.apk";
private static final String TESTDROID_SERVER = "http://appium.testdroid.com";

String testdroid_username = env.get("TESTDROID_USERNAME");
String testdroid_password = env.get("TESTDROID_PASSWORD");

Notice how we create the test class by extending to another class called BaseTest.java:

public class SampleAppiumTest  extends com.testdroid.appium.BaseTest.

Thanks to this extend, we get to use some of our self created code behind the scenes of the test class.

Let’s take a closer look at the BaseTest.java

Since we have both Android and iOS samples combined in the same Java project, we decided to create the convenience class called BaseTest.java. The class includes functions for both application upload to Testdroid Cloud and taking screenshots. Appium java-client’s screenshot implementation is based on Selenium Webdriver’s TakesScreenshot interface, which is basically too cumbersome to write multiple times.

While writing your own tests, extending to use such a convenience class can be a real time saver. This is what we do a lot also in our Image Recognition solution, which we will be covering later in this blog series.

Typically, our samples are built for Maven build automation tool. This tool is really handy as it automates downloading all required dependencies before building and executing the project. All the required dependencies and other project settings are described in the pom.xml file.

Running tests in Testdroid Cloud

You can run test from your IDE or directly from command line using maven:

>mvn clean test -Dtest=SampleAppiumTest

or to be more precise:

>mvn clean test -Dtest=com.testdroid.appium.android.sample.SampleAppiumTest

or run all the tests:

>mvn clean test

More Tips!

When you have more than one test case to run, you should be aware of the fact that every test session to Testdroid Cloud can use only one instance of the Appium Webdriver. If you use driver.quit(), the current session towards the cloud will also end.

In other words, bundling multiple test cases to one test run session requires you to keep using the same webdriver instance in each of your test cases. To make sure each of your test case starts off from a clean slate, take a look at the driver command resetApp.

If you’re unsure of your possibilities with the java-client, you may find some ideas by checking the Appium Java-client API documentation.

See you again on Wednesday! Happy Testing!

We’ll continue our 37 Things You Should Know About Appium blog series with a focus on C# (or csharp). In short, C# is a multi-paradigm programming language and was created to be as simple, modern and general-purpose as possible, not forgetting the object-oriented approach as a programming language. For Appium, C# is really a valid option and we’ll walk you through here with some basic things.

Before jumping into our today’s topic: we’ve received tons of great feedback and ideas for blogs to be part of this series, so please keep them coming, you definitely have been heard, and we’ll keep producing useful and step by step material for your needs. Now, let’s go!

Similar to the syntax of Java, C# is a widely used language for all kinds of programming and scripting needs. The primary reasons to choose C# as THE scripting language for Appium tests are those benefits and advantages also close to what Java programmers have. The recommended full IDE for C# scripting is Visual Studio for Windows and or if you use MAC/UNIX there are other options available as well. Anyway, I do recommend using an IDE, as having any type of smart auto-completion mechanism with proper listings of available methods is always a win and makes your day easier.

Without further ado, let’s get started with the C# sample! Take a look at this first and prepare your environment with the same way as we’ve done with the prior examples on Java and Python.

First, start with those desired capabilities and make sure those link properly to your Testdroid Cloud account. In case you don’t have an account yet, you can create one here or there.

For Visual Studio you should make sure that the NUnit Test Adapter is installed through the Extension Manager. In order to install the NUnit Test Adapter in the Extension Manager, select Tools -> Online Extensions (panel on left) and then locate NUnit Test Adapter and highlight it.

For running those tests, you can use Test Explorer. Using UNIX, first download the dependencies of the solution by following these steps:

1. Download all dependencies – with one command

We use NuGet to install a package using specific resources. Using the Testdroid example, you have a folder “Test123″ that includes packages.config. The following components will get fetched:

<packages>
<package id="Appium.WebDriver" version="1.3.0.1" targetFramework="net45" />
<package id="Newtonsoft.Json" version="7.0.1" targetFramework="net45" />
<package id="NUnit" version="2.6.4" targetFramework="net45" />
<package id="Selenium.Support" version="2.46.0" targetFramework="net45" />
<package id="Selenium.WebDriver" version="2.46.0" targetFramework="net45" />
</packages>

To get these components installed, use the following command line:

> nuget install Test123/packages.config -OutputDirectory packages

Now, all required bits and pieces are downloaded, installed and you are ready to build the package.

2. Build Test Package

Use the following command to build the test package for your local setup:

> xbuild

3. Run your tests

Then, use the following command to run recently built test package:

> nunit-console Test123/bin/Debug/TestdroidAndroidSample.dll

What about those test results?

When you begin your Appium testing with Testdroid, you may notice that by default the portal doesn’t show detailed information regarding your test run successes and failures. The reason for this is, of course, that you need to upload the results first!

Why is this so? During a test run, Appium server doesn’t keep count of succeeding and failing test cases. To put it simply, the server’s job is to take in the commands from the client, execute those commands on the connected mobile device and finally report back to client the results of those commands. Any test suite formatting, reporting and all that hassle is all handled by the client. So any test results will only exist on the client side.

Let’s take a look of how you can upload those test details to cloud to ensure everything is shown properly in the right context.

Uploading your test results XML to Testdroid Cloud

From our help pages, the test results upload goes like this:

Uploading your Test Suite output to Cloud
If your test suite generates a JUnit XML results file, you can upload the XML to Testdroid Cloud. When you do this, all your test cases and their statuses will be shown on the Testdroid Cloud testrun view and after this, the complete test report is downloadable. Couple of good tips to ensure everything goes fine:

> Add the testdroid_junitWaitTime Desired Capability in your TestScript.
> Get Appium sessionId from your script using driver.session_id. Note that this only can be done after the WebDriver connection has been properly established.
> After your test run has finished, and JUnit XML has been generated, use cURL to upload the XML to Testdroid Cloud:

curl -s -F result=@"/absolute/file/path/TestOutput.xml" "http://appium.testdroid.com/upload/result?sessionId=<sessionId>"

But wait, Webdriver in C# doesn’t seem to have driver.session_id!

Yikes! That’s what I was wondering too while I was preparing this post. After some digging though, I found a way to print out the session id.

Fortunately this Stack Overflow post has a well working solution to make the session_id value available. Here what you should do – first add the function for session_id:

public class CustomRemoteDriver : RemoteWebDriver  
{  
   public CustomRemoteDriver(Uri remoteAddress, ICapabilities desiredCapabilities)  
          : base(remoteAddress, desiredCapabilities, TimeSpan.FromSeconds(400))  
          {  
          }  

   public String getSessionId()  
          {  
            return this.SessionId.ToString();  
          }  
   }

Then, you should add testdroid_junitWaitTime in desired capabilities:

capabilities.SetCapability("testdroid_junitWaitTime", 120);
...
driver = new CustomRemoteDriver(new Uri("http://appium.testdroid.com/wd/hub"), capabilities);

Finally, use the following line to print out the session id:

Console.WriteLine (driver.getSessionId());

These edits that I did to print out the session id can be also found here.

So far so good.. and TIPS!

One last thing to consider is that Testdroid Cloud expects the test results xml file to be in JUnit format, as the desired capability leads to believe. NUnit and JUnit result files are very close to each other, so using some converter is more than possible. For example, here’s an open sourced Jenkins plugin for NUnit reporting, which includes JUnit conversion as one of its functionalities.

Great! I hope you learnt something about how to get in started or if you use it already, make things working smoother with your Appium and Testdroid Cloud combination on C#.

We’ll continue on Friday about the wonderful world of Appium and Ruby! Stay tuned – and Happy Testing!

Dear Testdroiders,

Awesome news: We’ve pushed out a new release today, and with this new version things are about to get a whole lot better and easier for you guys! We’ve done lots of work on back-end to improve the efficiency, configurations and improvements in scalability. Some of these things aren’t really visible to users but you’ll surely notice the difference as more devices are added every week, the UI works faster and accessing devices is yet easier for everyone.

The new version of Testdroid Cloud – 2.9 – has been thoroughly tested during the whole week and our QA just gave us green light to push it out. This version will surely pave the way to your success by improved product features, better reporting and visualized results, more efficient handling of your plans and a faster access to any device supported in Testdroid Cloud.

Let’s take a deeper look!

Calabash Upgrade, Enhancements – and More!

You can now get visualized test cases and test steps with your Calabash test runs. This makes a quick or deep analysis of test results easy and spots out those important steps in your runs. Also, we’ve added Cucumber report view for all Calabash test runs. You can also quickly and easily navigate through your Calabash test steps – and here is the example:

Furthermore, Calabash stack trace is now readable and you’ll have access to all fine-grained details under your device runs (device run view) with more log and data files available:

Efficient Handling of Your Plans

We’ve rebuilt our buying widget and renewed the entire buying process with new, beautiful single screen widget. You can manage your subscriptions, adjust plans and so all in one view:

Appium Improvements and Instant Access to Devices

We constantly upgrade and improve things around Appium framework, and one of the highlights in this release is the API key implementation for better access to devices. Basically, you don’t need user credentials anymore but access can be also gained by using API key in your desired capabilities to push your test from your local machine to our devices. And the same thing applies for uploading your local test results to Testdroid Cloud. Very nice, clean and easy way to run your client-side Appium tests!

The API key can be fetched from your profile (top right-corner) -> My account -> API Key. Then just use it in your desired capabilities as follows:

capabilities.setCapability("testdroid_apiKey", "bd9db158f508e852df05b331950e4740");

Remember! If you use testdroid_apiKey, you don’t need to configure user credential (testdroid_username and testdroid_password in your desired capabilities).

Have fun with the new 2.9 and keep testing!

We’re back with a new blog in our 37 Things You Should Know About Appium and this time we will focus on Ruby. Ruby has become popular in past years as a similar scripting language to Python and Ruby is a dynamic, reflective, object-oriented general-purpose programming language using in variety of different purposes.

In a nutshell, Ruby is a language comparable to Python in its simplicity and clearness, although it does require a bit more investment of time to learn. Once you get used to it though, Ruby can become a very powerful tool due to its well defined and robust design.

You’ll find here all steps you need to start your tests – using Ruby – against real mobile devices in Testdroid Cloud. Before continuing with running with these scripts you should register with Testdroid service.

Getting Started

First and the foremost, you should install Bundler to ensure all necessary gems and dependencies gets installed properly. Then, update your test script (testdroid_*.rb) with necessary information. Always do remember to modify your desired capabilities before you start a test. This will always save you time and effort as you don’t have to run a test to realize there was something wrong with desired capabilities.

Ok, here is how to install:

$ gem install bundler
$ bundle install

Launch a test:

$ rspec testdroid_android.rb

NOTE! There are some common settings that you need to set in all scripts regardless of the app type that you are testing. Each testdroid_*.rb file needs to be updated with the following values.

Here are all the values that you need to configure:

screen_shot_dir – where should screenshots be stored on your local drive
testdroid_username – your email that you registered in Testdroid Cloud
testdroid_password – your Testdroid Cloud password
testdroid_project – has a default value, but you might want to add your own name to this. Project name is visible in your project view in Testdroid Cloud. Each project must have a unique name
testdroid_testrun – name or number of this test run. Test run names can be modified even at every test run
testdroid_app – should be the name of the app you uploaded to cloud. Eg. if you uploaded your app using a script this would look something like this: ‘f4660af0-10f3-46e9-932b-0622f497b0d2/Testdroid.apk’ If you uploaded your app through the Testdroid Cloud web UI, you can use here the value ‘latest’ that refers to the latest uploaded app file.

Native iOS Specific Settings

Example script: testdroid_appiumdriver_ios.rb

To run your Appium tests against your native iOS application with real devices you need to edit the testdroid_appiumdriver_ios.rb script.

In addition to the above mentioned Appium capabilities for iOS testing you need set:

bundleId – this is your application’s unique name

Two Tips for Ruby on Appium

(1) You can launch your Appium test to use the latest app file you uploaded through Testdroid Cloud portal by setting the testdroid_app desired capability value to ‘latest’.

(2) To automate device finding from Testdroid cloud, you could check this Ruby tool contributed by bootstraponline. The tool is actually a port from our Python device_finder.py script based on the Testdroid Cloud API. The API can be utilized to do basically all the things our Portal allows you to do, and then a bit more! Documentation for the API can be found here.

See you again next week!

This is the sixth blog in our 37 Things You Should Know About Appium and this time our focus will be on JavaScript, with some additional flavors on Node.js stuff. Furthermore, I have great news regarding getting all these tips and tricks delivered to you: everyone who has signed up to our Appium Weekly will have these blog posts automatically delivered and also get some additional info within those emails (some extra tips and tricks).

JavaScript has become a de-facto programming language in web-based applications and it is widely and commonly used in other software domains, such as in game development and naturally server-side network programming, as well. Appium can be run with JavaScript through Node.js – thanks to different testing frameworks such as Mocha and Nightwatch. We truly encourage you guys to take a look at these – IF you consider using JavaScript and/or Node.js for your mobile app testing.

Getting Started

There is one awesome example (with some source code) by Aaron Colby using Nightwatch framework and this example provides an insight on how to launch Safari browser and use your tests on navigating to a specific website. Nightwatch is the framework for browsers and web apps specifically. Take a look at the example first – it’s available at https://github.com/acolby/Testdroid_Example.

To recap some of the essential things from Aaron’s example and some further things to consider:

Add Dependencies

1.) npm dependencies

$ npm install

2.) install mocha globally

$ npm install -g mocha

Add Credentials to ./.creds.json

You will need to add some an object that looks this to the directory ./.creds.json, as it is .gitignored

{
    "testdroid": {
        "username": "TESTDROID_USERNAME",
        "password": "TESTDROID_PASSWORD"
    }
}

to Run the test

$ mocha ios_safari.js

All dependencies MUST be installed as instructed in the readme file. Then, create a new file called .creds.json and add your Testdroid credentials as illustrated and exampled above.

You also have to create a directory named as "temp" for all screenshots that will be taken during the test session. Explore the ios_safari.js file to see how the test itself is built. In order to make the device finder look for freemium devices, add the creditsPrice value as 0 (or 1 for premium devices) to the criteria, as follows:

if(devices.data[i].locked === false && devices.data[i].creditsPrice === 0) 
{
  device = devices.data[i];
  i = devices.data.length;
}

Example of Appium for Mocha based Node.js tests

Let’s use the earlier sample as a base to create a native iOS app test next! The difference here is that we won’t be needing the Nightwatch framework at all.

Desired capabilities

Edit the desired capabilities, namely the testdroid_target, testdroid_app and bundleId. Also, make sure to remove the ‘browser’ capability.

desired.testdroid_username = creds.testdroid.username;
desired.testdroid_password = creds.testdroid.password;
desired.testdroid_target = 'ios';
desired.testdroid_project = 'Test iOS Mocha';
desired.testdroid_testrun = 'TestRun A';
desired.testdroid_device = device.displayName;
desired.platformName = 'iOS';
desired.deviceName = 'iOS Device';
desired.testdroid_app = 'sample/BitbarIOSSample.ipa';
desired.bundleId = 'com.bitbar.testdroid.BitbarIOSSample';

Using testdroid_app = 'sample/BitbarIOSSample.ipa' you can default to the pre-existing sample app which is always available in cloud! For android the same would be 'sample/BitbarSampleApp.apk'.

Test case

Let’s edit the test case to do something with the native sample app, instead of the safari targeted commands:

it("should set text to 'Tester'", function() {
return driver
.sleep(4000)
.saveScreenshot(SCREEN_SHOT_PATH + '/test.png')
.sleep(4000)
.elementByClassName('UIATextField')
.click()
.keys('Tester')
.saveScreenshot(SCREEN_SHOT_PATH + '/test2.png')
.hideKeyboard('Return')
.sleep(4000)
.saveScreenshot(SCREEN_SHOT_PATH + '/test3.png')
.sleep(4000)
});

You may of course change the name of the ios_safari.js file to something that makes more sense, like ios_native.js or so. The test case function describe("ios safari", function() { should probably be re-named at this point too.

Documentation

For the Selenium based driver commands, best place to refer would be the Selenium Webdriver’s Node.js documentation’s full jsonwire mapping.

Running Tests in Cloud – with REAL Devices

Missing some hot Android or iOS device in your testing roster? Or maybe all of them? That’s why Testdroid Cloud was originally created: to serve all Android and iOS developers and testers to access any device, any time, and from anywhere.

First, get your environment set up properly.

Secondly, select suitable programming language that you use regular basis and adopt/build your tests around it.

Finally, follow up with our best practices – and you’ll be good.

See you again next week and keep up the great work on your testing!

Today, we’ll be focusing on using command line for your Appium tests and testing. This blog is part of our 37 Things You Should Know About Appium blog series that will provide you the best practices, variety of tips and tricks, plus more insights on how Appium can be used efficiently for your mobile app, game, or web testing.

Appium is a fast growing platform with an active open source community. This means that its code gets updated and further developed on a regular basis, especially because of the fast moving mobile OS world that we live in. It is not unusual to see developers sweat over the news that a new mobile OS version is coming up soon. The amount of time to make sure that your apps work on the upcoming version tends to be short, which also puts a lot of stress for the people handling Quality Assurance. If developers have a short time to prepare, the QA probably has even less time to do so.

To get the latest version of Appium, you should pull the source code from Github and then compile it on your designated host machine. Appium is provided with a script to handle the build process, although it does require some prepping when doing so for the first time. Once all pre-requisites are found on your system, building shouldn’t be a problem from there on.

If you need any help getting the environment, any Appium stuff properly setup, please take a look at these instructions.

Prerequisites for Appium

First, depending on what you intend to automate it’s probably easiest to see if your machine already has all the requirements installed. You can try this by navigating to Appium directory and running the appium-doctor.js script found at the bin directory. If you don’t intend to automate both iOS and Android devices, you can safely ignore any warnings of the platform you don’t need. Using JavaScript? Take a look at these guidelines for getting everything running smoothly.

Here in Testdroid when we want to build Appium on a Mac device to support all the usual automation needs for both iOS and Android devices, we use this command:

$ ./reset.sh --android --selendroid --ios --real-safari --verbose

The --verbose flag is important, if you suspect that you might be missing any dependencies as it shows detailed information on the build process.

Furthermore, do check that everything is properly configured, whether you are using Java, Ruby, Python or C#.

Prepping the Test Devices

Once you have Appium server built up and ready to go, the next step is to connect your test device(s) to the machine and accept any trust dialogs the devices may show. On Android devices you need to enable USB debugging in the Developer options and also allow the device to install APK files from “unknown sources”.

iOS devices require UI Automation to be enabled in the Developer settings, which should be visible as long as you can see the device in Xcode Devices window (cmd+shift+2 on Mac). To make sure your device is activated for test automation, you should also open up Instruments from the Xcode and select the devices you have as active devices.

Using Appium from Command Line

To launch Appium, use the appium.js script found in the bin directory to fire up the server. For iOS you will need to add the -U, or --udid, flag with the udid of your target device.

If you want to automate multiple devices at the same time, simply fire up multiple instances of the Appium server, all with different ports in use. With multiple devices, you need to set --port flag for the actual Appium port (default 4723) and a --bootstrap-port for communication with the different Android devices. Selendroid and ChromeDriver also need their dedicated ports when automating multiple devices so don’t forget to set --selendroid-port and --chromedriver-port as well, if you need Selendroid or Chrome.

You should note that no more than one iOS device can be automated in a single host machine due to restrictions in Xcode.

Launching Tests Toward Your Server

Okay, so hopefully you have now an Appium server happily running. You’re ready to launch your tests towards it then! All you need to do is let your test scripts know which address to connect to.

If the server runs in your localhost, and in default port, you can set the webdriver to initialize towards http://localhost:4723/wd/hub. If you have multiple servers running, simply make sure the IP address and port number are matching to the server you want to use. That’s it!

In Testdroid Cloud we have developed our very own Appium Broker, which handles the communications of all on-going sessions between clients and Appium servers. In other words, we handle the creation of all the Appium server instances with their dedicated ports completely on the cloud side so that all you need to do is launch your tests towards http://appium.testdroid.com/wd/hub.

Happy Appium Testing through Command Line!