Dear Testdroiders,

Appium is getting more popular and widely used day by day and we have been trying our best to bring you the greatest experience of Appium test execution at Testdroid Cloud. Today we have great news to share that Appium Server-Side Execution is now available for all users with Testdroid Cloud SOLO plan or above.

Appium Server-Side Execution

To get access to Appium Server-Side Execution, subscribing to Testdroid Cloud SOLO plan as low as $99 per month will enable you to automate Appium tests at scale on multiple real devices. It’s very easy to conduct your first Server-Side Execution:

1. Create a Project named with Appium Android Server Side or Appium iOS Server Side depending on your need.

2. Upload your .apk and test file. Make sure that you compress all needed test files into a zip file. If you need more details on what files are needed, check it from here – Java Server Side Example and Python Server Side Example.

3. Select devices, do necessary advanced configurations and start the test.

Appium Client-Side Execution

In addition to the greater availability of Appium Server-Side Execution, we also updated the way of how Appium Client-Side Execution works that there is no need to provide an exact device name to start your Appium test run.

Appium Broker now searches all the devices at Testdroid Cloud and picks one close available device that matches to the requested device. For example, by giving iPhone as testdroid_device capability name, Appium client will pick any of the available iPhone devices, regardless of iOS versions or iPhone models. That means your test could be executed against an iPhone 5S 8.2 or an iPhone 6S 9.1.

If you just want to make sure your test can be started on any device in the Android group, you can simply define give testdroid_device capability an android.

If you want to turn off this feature and run your test on the specific device, then giving FALSE value to testdroid_findDevice desired capability will do the trick. By default the value is TRUE.

desired_capabilities_cloud['testdroid_findDevice'] = FALSE

Happy testing with Appium in Testdroid Cloud

Dear Testdroiders,

As you may know, Appium 1.5.x is a complete rewrite of Appium from the scratch. It is a technical re-architecture that keeps the code organized at best and makes it easier for new contributors to improve the framework. To all Appium lovers, we have made it executable on all of Testdroid solutions and you can now select either Appium version 1.4 or 1.5 to drive the Server-Side Execution for your mobile app testing.

Since the introduction of Appium on Testdroid solutions, we have been hard at work to sharpen the Appium support and bring the best experience of Appium test execution to all of you. The Server-Side Appium Execution at Testdroid has enabled every Appium user to run mobile app tests at scale and speed up the QA cycle tremendously. Now coupled with Appium 1.5, it will give you the access to run parallel Appium tests on all of the devices as usual as well as iOS 9.3 devices and hopefully soon on iOS 10 devices.

How to Use Appium 1.5 at Testdroid Cloud?

By default, you will be using Appium 1.4.16 when running server-side tests on Testdroid Cloud. The approach looks like below:

# on iOS
/opt/appium/bin/appium.js -U ${UDID} --command-timeout 120 >appium.log 2>&1 &

# on Android
/opt/appium/appium/bin/appium.js >appium.log 2>&1 &

To get started with Appium 1.5, the above lines need to be updated to as below. It is suggested to update the script even if older Appium version is used. The launch procedure has been improved with this new approach.

# on iOS
#appium-1.4 -U ${UDID} --command-timeout 120 >appium.log 2>&1 &
appium-1.5 -U ${UDID} --command-timeout 120 >appium.log 2>&1 &

# on Android
#appium-1.4 -U ${UDID} >appium.log 2>&1 &
appium-1.5 -U ${UDID} >appium.log 2>&1 &

Due to the way that Appium is started, there is no more a need for a constant sleep after the launch. The following line should be commented out in the run-tests.sh as unnecessary.

sleep 20 # Sleep for appium to launch properly This is not needed anymore

To all of users who are mainly using Client-Side Execution at Testdroid Cloud, we will upgrade it to version 1.5 in the next few weeks. That being said, we would strongly recommend you checking out Server-Side method to test your mobile apps or games. It’ll surely save your time to configure desired capabilities all the time.

Happy Testing with Server-Side Appium Execution at Testdroid Cloud!

Over the past years, more and more organizations have been striving for the goal of a >99.99% availability to ensure that their users or customers can get access to their offerings at most of the time. Within the period of an unplanned downtime, the usage of the service is expected to be trivial in an attempt to minimize the loss. However, with 2.6 billion users globally that can access to internet via smartphone, the loss of a service downtime would be tremendous for any type of organizations. For SMEs a service outage may sabotage the growth or even lead to a business failure at worst, and it could also wipe whopping millions of revenue on large Enterprises’ annual report and negatively affect the brands.

For many of the organizations, the definition of downtime means more than ‘the system is not accessible’. It also includes the following two scenarios:

• The system is accessible but some functions are not operational
• The system is accessible but performance is highly degraded

With those in mind, do you know how much a service unavailability would cost and how much your organization risks to lose during downtime?

Revenue Loss And Hit on Cash Flow

First things first, revenue loss is inevitable due to downtime. According to Gartner’s research in 2014, the monetary loss is close to $5,600 per minute, equaling over $300,000 per hour. This is just an average number and there’s a large degree of variance. But let’s take some examples.

In case your mobile ecommerce website generates $50 million of revenue in a typical year, it looks like this.

For a 99.5% uptime, your website can be down over 43.8 hours. The total cost would be $250K.
For a 99.7% uptime, the downtime is over 26.28 hours, extrapolating a total loss of $150K.
For a 99.9% uptime, the downtime is about 8.76 hours. The total revenue loss would still be $50K.

Now imagine the downtime happens on Black Friday or Cyber Monday, the loss is much higher. Your cash flow would take a big hit. This applies to mobile applications or mobile games as well. As more people nowadays order products or consume your services with mobile phones while they are the go, that monetary cost of downtime is surely rising.

Productivity loss

When service outage happens, your internal employees’ workflow will get affected as well. The affected departments are not able to complete tasks, which produces unplanned delays to deliver the accomplishments. This can bring down overall productivity.

Your marketing and business team might have troubles to carry on the ongoing sales campaign.
Your financing team might not be able to finalize the budgeting and calculations due to no access to data from your database.
What could be even worse is that you or your boss might be presenting the current progress of the company to the shareholders.

All of associated parties have to slow down their pace due to the service outage. That leads to a low responsiveness. And at the end of the day, time to market is one key success factor in today’s competitive industries.

Customer Churn and Strategic Partners

One of today’s mobile characteristics is that there are so many selections out there and consumers have much more alternatives to meet their needs. When you undergo a period of downtime, your customers can simply go somewhere else to get what they need. And your loyal customers might also complain the service unavailability.

Now if your mobile websites or applications suffer a downtime in some locations, your strategic partners that are planning or executing campaigns will experience the delay or inaccessibility as well.

Loss of Usage/Visits

If your mobile website is not accessible or very slow to visit, your website usage will simply drop and that would definitely have a bad impact on your sales. The longer the downtime lasts, the less visitors you will get. In the long run, your website SEO will be punished.

The same rule goes to mobile apps or games. Your mobile apps or games might be perfect in terms of quality, with no flaws, no bugs, great content, etc. But when your app doesn’t launch in an acceptable time, users will just quit, never try the second time, and uninstall it from their mobile gadgets. What’s worse is that they go to app store and leave negative reviews.

Damage to reputation

Like mentioned above, app users will spread the frustrations with your services on app store and social media. The news of downtime in your services reach across the globe very fast. Your mobile apps or websites can be labeled as ‘unreliable’ or ‘slow’ based on a shared experience that touches public’s consciousness.

At the same time, your direct and indirect competitors would be quick to avail your service downtime and win over the market and consumers to stay with them. And you are ended up with a less market share and a bad brand image.

How to prevent downtime?

How can hidden issues be visible upfront to prevent the performance pitfalls? Many organizations have implemented synthetic monitoring strategy to observe their mobile applications and websites. Synthetic monitoring helps IT professionals and developers to proactively check the availability of their service in every predefined interval, especially making sure that mobile apps and websites are up and running smoothly at peak hours. By incorporating synthetic monitoring into your mobile devops process, your devops guys are able to get insights on different types of metrics – accessibility, mobile performance, competition, etc. and take responsive actions quickly before your products or new versions get published.

Earlier last week, Facebook unveiled its mobile device testing lab equipped with physical Android and iOS devices for testing Facebook app, Messenger and Instagram. While it shouldn’t be surprising that Facebook chooses an in-house testing solution to test their mobile apps, the amazing thing here is the determination of bringing higher quality and top-notch performance to its apps by building a large-scale testing environment with almost 2000 devices using 60 racks to test against.

To ensure the scalability, stability and concurrency of a device lab with such a large amount of devices, Facebook obviously has done a great job there for mobile app testing. Let’s first review some of the details about what they’ve done for the setup.

• All real, physical devices with no emulators
• Not only mainstream device models, but once-red-hot ones are also included
• All the latest versions of Android (KitKat and above) and iOS (7 to 10)
• Phones are rebooted after every test
• The new version of their app will be automatically tested on in-house devices
• Using PCs to connect Android devices and Mac Mini’s for iOS devices

Why In-House?

Like many of the other companies out there, Facebook could have relied on a cloud-based public testing solution like Testdroid Cloud to guarantee their apps stay on top. However, the decision behind using a dedicated mobile device lab is made normally for couple of reasons. And we believe Facebook wants to

• Fix bugs with as little turnaround time as possible
• Shorten the dev-and-test cycle to a large degree
• Accelerate the app release by delivering more features to end users
• Cover the widest range of mobile device users
• Get team the agile philosophy and mobile devops methodology

Does Facebook’s Story Inspire You?

Now, does their decision of the setup inspire you? Or have you been considering an in-house solutions for testing your applications or games of Android or iOS or both versions? From the perspective of HW and SW, it shouldn’t be very difficult for you to get all of needed devices and machine servers. And all you need is a robust and reliable solution to get your automated tests working on all of your device variants.

Testdroid Enterprise is the go-to solution that can fulfill your in-house testing needs immediately. From the beginning of Testdroid, we have been providing Testdroid Enterprise solution to organizations including Government, Banking system, Mobile OEMs, Telecommunications, Gaming companies, etc. The size of setup varies largely depending on the needs, from tens to hundreds of devices regardless of Android or iOS phones, bringing the high scalability and concurrency to our customers by running automated tests on all of the devices in parallel mode.

Though Testdroid Cloud is a public cloud where all devices are shared with every free user and paying customer, the environment itself is deemed as an enterprise-grade setup and configuration hosted by Bitbar. Over 1000 devices in total (430+ unique devices) are hosted and connected to the server 24/7 to ensure our users have access to these devices at any time from anywhere. Testdroid Cloud on our premise itself is a great example to prove you the reliability and scalability of a Testdroid Enterprise solution. With Testdroid Enterprise in your team, your distributed teams can run mobile automated tests on real devices no matter what timezone they are located around the world.

To get a closer look at how to build, operate and maintain a large scale mobile device test lab, don’t miss this great ebook that illustrates all the info from basics to specific HW we use at Testdroid Cloud to daily maintenance tips. The ebook will surely help you better understand the process of setting up one and do better mobile app testing.

‘But Investing in a Spacious Data Center Is Not My Top Choice?’

It is very normal that due to multiple reasons, building and maintaining a data center with physical devices is not the ideal option for your company and the nature of the application or project that you’re working on determines that a private environment is needed to meet the requirements.

In that case, Testdroid PrivateCloud is the de facto solution that you should think about. Like Testdroid Enterprise, we have been hosting a private cloud environment for customers from various industries, with the setup size ranging from dozens to hundreds of devices. After all, Testdroid PrivateCloud has the same technology and capabilities to do custom configurations as in Testdroid Enterprise. It’s solely the matter of who hosts and manages the environment.

If you are in the middle of evaluating an on-premise or privately built solution for your dev and test team, welcome to send your inquiries to sales (at) bitbar (dot) com.

Happy Testing!

As you may know, we have been working on Bitbar Monitoring for couple of months. The beta version is available for application and will be soon accessible for all of you. At the same time, we are running a survey regarding our new product and how application performance monitoring is handled in your company.

In the past month, 91 participants have contributed to the development of Bitbar Monitoring by filling out the poll and giving their comments. We have taken some great tips and ideas into consideration and now are making these as features in the new product. If you’d like to help us build a better monitoring product that can actually benefit you in the future, fill out the survey here and leave your comments. We will share the results of the survey when the beta version of the product is available.

In the survey, there is a question – ‘Do you currently use Real User Monitoring to trace customer issues?’ Up to now, approximately 80% of the responders don’t have RUM implemented for monitoring the performance of their app or website. Though we didn’t catch the reason of that, let’s take a look at the differences between synthetic monitoring and real-user monitoring.

Synthetic Monitoring or Real-User Monitoring

Before we look into the differences between two types of monitoring, let’s review the definition of both.

Synthetic Monitoring, a.k.a proactive monitoring, simulates the real user interactions with your mobile apps and real user behavior on your websites to periodically check the performance of your mobile apps and websites and to catch the real issues before your users do.

Real-User Monitoring, a.k.a passive monitoring, helps you constantly track the actual customer experience they are having with your mobile apps or websites in real-time and determine if your users/customers are served in actual service-level quality.

Be the first to know if your mobile app or website is working

• If your mobile applications fails to go through the transaction in the midnight, how will you know?
• If your application is working well in your home country or home city but not in other geolocations, how will you know?
• If you have a new mobile application/website or an upgrade, how will you know if it would be available in different countries before it goes live?

Well, synthetic monitoring is the answer to these. Synthetic monitoring helps you select the preferred locations to test the availability and performance of your mobile application or websites. It ensures that the transactions perform properly at any time when tested at a regular interval. With synthetic monitoring, you will be able to release any new products or releases with full confidence.

Conversely, real-time monitoring can’t address these issues and deliver the same value as synthetic monitoring in this scenario. At 3am of the day, no one is on your website and no real-time data can be captured to determine if your website or service is accessible. And it’s mostly like a shot in the dark without testing the performance of your new products before they hit the market.

Data Reliability

Synthetic monitoring and real-user monitoring have both pros and cons on the reliability of the data captured.

As it says, synthetic monitoring stimulates the user behavior with the help of test automation and test scripts. It assures if the mobile app or website is available at the moment, but the captured data cannot reflect real end-user experience in real world. On the contrary, the real-user monitoring does provide a piece of actual data generated by a real visitor or user to your mobile app or website. Overall synthetic monitoring lacks the diversity of the data collected in the real world: geolocations, user network conditions, hardware, etc.

Data Usability

On the other hand, synthetic monitoring provides more useful insights than RUM, just because synthetics don’t track real user sessions. The script used for synthetic monitoring is executing a predefined set of steps at a regular interval from a known location/city. The performance of your mobile apps or websites is predictable.

Though RUM can provide your enormous actual data from your end-users, lots of noises will be generated as well that it makes hard to analyze the data and pinpoint the real issues. In that regard, data provided by synthetic monitoring is more useful for altering that very noisy data collected by RUM.

A better approach

Looking at synthetic monitoring and real-user monitoring, either of them has its advantages. Each seems to be superior in certain aspects of performance monitoring. A traditional way is to have them both applied in your mobile devops strategy.

In short,

• Synthetic monitoring is best to be used to check if your service is available at a regular interval to prevent a potential and risky service downtime.
• Real-user monitoring is best to check the real end-user experience and user behavior when they are engaging with your mobile app or surfing your mobile website.

That being said, synthetic monitoring should become a must in your daily mobile devops strategy, if you had to or could only incorporate one of them. Due to its nature, you can prevent a huge loss of annual revenue for your business if your synthetic monitoring system is properly setup and configured to get alerted of a potential service downtime.

Want to get alerted with the progress of Bitbar Monitoring? Sign up for beta version and stay tuned for the next generation synthetic monitoring for mobile app and website.

Dear Testdroiders,

To reach the optimal condition of revenue generation through your mobile projects, the first thing you need to do is to ensure that your mobile apps or games always work perfectly on the most popular devices or the latest devices. At Testdroid Cloud, you have the chance to access all of these devices with a very low threshold.

Since the past weeks, we have acquired and added several cool devices that either already have a high market sshare or are trending. With your current Testdroid Cloud plan, you can expect to avoid an over $2200 (equivalent of €1840) upfront payment to buy all of these devices. The instant accessibility of these new devices shall bring more value to your app testing within your current Testdroid Cloud plan.

What devices have we recently added?

The most notable devices we have added lately are as below with specs info.

Talking about OnePlus, the whole family can be found on Testdroid Cloud. In addition to the newly added OnePlus X, you can also test your mobile apps/games instantly on OnePlus One and OnePlus Two. In case you want to cover both OS versions of OnePlus One, the OnePlus One CyanogenOS and OxygenOS are both available on Testdroid Cloud.

I need to test on (some of) these devices. How to get access?

As mentioned above, existing Testdroid Cloud customers have the instant access to these devices with no extra cost. If you don’t have a valid plan at the moment, you can spend less than $100 per month to use these devices for 10 hours. You can either do automated testing or manually control these devices as if they were in your hand. In fact, all of the devices will be at your disposal for testing. Check out our pricing

On top of the access to all devices, you will receive our authentic and cool Testdroid t-shirt for free in this summer. The campaign is still ongoing and you will become a true Testdroider when you become a Testdroid customer and put on your Testdroid t-shirt.

What’s Next?

As usual, Fall is always the critical season for the smartphone giants – Apple and Samsung , announcing and releasing their new flagships. Surely this year we’ll be expecting the next generation iPhone – ‘iPhone 7’ and the new Samsung handsets – ‘Samsung Galaxy S8’ and ‘Samsung Galaxy S7 Note’. Again, you will be able to save thousands of USD or EURO for acquiring them. Instead with $99, you will get the full access to these devices on Testdroid Cloud.

Stay tuned for new devices and happy testing with Testdroid.

The launch of Pokemon Go has proved itself a success. In only one and half month the game has broken Apple download records and it is rumored to help Apple get a $3 billion boost this year. Once it becomes available in more countries, the game certainly can reach a higher peak.

Given the success story of Pokemon Go, many companies are believed to rush into the world of augmented reality (AR) thanks to its great potential. It’s true that the technology of AR is one of the factors that have contributed to the success of this game. But like any other great apps and games, the continuation of its success relies on not only the technology or game concept, but also other traits and app quality strategy from the team.

In this blog we’ll look into couple of things that the most successful companies or the teams behind mobile apps and games would do.

Do faster releases with new features

The bar of users’ expectations has been set much higher than before. When your app or game just gets to end users’ smartphones, they are already expecting new things in the coming releases. And the timely releases of new features certainly help you hook your end users to stick around with your apps or games.

In today’s competitive mobile market, organizations have realized that they need to accelerate the pace of mobile app and game releases to meet customer expectations. A slow-to-market may cost a bank or a gaming company thousands of hundreds of annual revenue. But many companies have been struggling with the challenges of achieving the faster releases.

Strategy to be considered:

Accelerating the pace of go-to-market requires app developers and game studios to adopt new technologies and methodologies. Higher frequency of releases means more testing tasks to avoid poor quality of apps or games. In that regard, test automation plays a significant role to your mobile strategy.

Along with test automation, you also need to make sure your tests are executed on multiple devices in parallel. With one test script, you can test your project on hundreds of real Android and iOS devices easily and repeat it at no cost.

To start you need to figure out how and what tests to automate. A common scenario is that you can start with a small number of test cases and automate them to verify if they can be executed in parallel.

Always cover new devices and OS versions

The mobile market may change as quickly as the rank of App Store or Google Play. You may find it hard to keep up with the releases of new devices from every major smartphone maker or the updates of the OS versions from Android and iOS. However, your users never care about the hardness of getting apps compatible with the latest HW and SW.

Many companies find it challenging to keep up the pace of new devices and OS versions, as acquiring a single new device and adding copies of different variants quickly burns the budget and health of the company, regarding of Enterprises or SMEs.

Strategies to be considered:

Instead of investing in every single device and variant, organizations can choose to subscribe to a on-demand cloud-based device farm. The benefits of this strategy is that with a monthly fixed cost you not only have the first access to the latest devices and operating systems, but also any other old popular devices on which your end users have your apps installed on.

At the same time of revenue generation, your business can be running healthily to pay a small amount of fixed fee to a third party vendor without the risk of going bankrupt at the early stage.

Take into account real user environment

End users get access to apps or games in any circumstances – 3G/4G, GPS location, strength of signal, low memory, interruptions during the app interaction, etc. Many organizations are figuring out a way to replicate the same use case as their users to improve the user experience at best. But it’s hard to achieve this in a public environment with shared devices, as cloud-based platform is usually predefined in a neutral way so that every type of apps can be tested without meeting any limitations.

Strategies to be considered:

Companies can build an in-house solution to mimic the real user environment and achieve the best user experience. When it comes to in-house solution, there are two options – homegrown mobile test labs or the mobile test labs powered by more robust technology.

Investing time and efforts in a homegrown test lab may only give you a very low ROI – restrictions on the test parallelism, the stability of the system, the device concurrency, the support of different open source frameworks, etc. All of these will extend the time-to-market.

Conversely, it’s much easier to adopt a commercial in-house solution. All you need to do is to get essential hardware, set them up and get the solution delivered to your environment. A robust and reliable commercial in-house solution should give you the plug-n-play magic and enable you to start mimicking the real user environment shortly.

Dear Testdroiders,

Bitbar will exhibit at IBC 2016 in RAI Amsterdam during Sep. 9th to 13th with an array of our latest mobile app testing solutions and our thoughts on how to consolidate mobile strategy for business. IBC is the premier annual trade show held in September for professionals engaged in media and entertainment, content creation and technical associations around the world. As the first time exhibitor, we will be presenting us at IBC Launch Pad at Hall 9. Come to visit our booth at 9.LP28 if you are also attending the conference and we would look forward to having a face-to-face talk with you about your current and future mobile plans.

Speaking of mobile strategy, most organizations that have a strong customer-driven belief have been utilizing mobile applications to respond the market more rapidly and even to drive the direction of what end users should expect. Most of the executives have the sense that today’s end users are demanding, provided that multiple digital choices are available for accomplishing end users’ tasks. They are always longing for a new experience or a simpler method to streamline daily jobs with mobile applications over smartphone.

To keep users delighted, a faster go-to-market strategy needs to be created. At the end of the day, no one would doubt that the faster one can ship mobile applications to Android and iOS app markets, the higher chance one will have to win the market. Yes, the sheer speed matters here to make your mobile applications continuously successful. But what should be done to speed up time-to-market for your mobile apps? In a nutshell, accelerating app release cycles with faster and more iterations. Let’s break it down.

Adopting DevOps for the Whole Mobile Development Lifecycle

Many companies have adopted agile practices within development teams, but that’s not enough or not agile enough with regards to the whole mobile development lifecycle. The feedback from QA to developers is usually delayed, for example, if QA teams don’t have an equal implementation of agile methodology.

To overcome this, mobile DevOps is one of the hottest approaches nowadays that have been adopted to achieve the agility. DevOps is a cultural shift and it streamlines the collaboration between development, operations and testing teams. It enables app releases to happen rapidly and frequently with more confidence. When a company speeds up to a monthly app release cycle, QA can already start testing their apps while developers are working on the new features. This helps QA find and report bugs earlier and improve the visibility of the issues found in apps to developers at an earlier stage. In the end, the number of defects can be split and reduced in different phases and it prevents developers to fire-fight with bugs right before the production.

Implementing Effective Mobile Test Automation

Test automation has gained popularity and more companies have realized what an important role it plays in getting apps tested and verified in a fast manner. While test automation per se does help to speed up testing and app releasing, the way of how it’s implemented makes a significant difference to the outcomes.

The first thing you need to think about is the stability of the setup or environment over which you are running your test scripts. Without a stable and reliable deployment, the test results will by no means be promising and testing jobs can hardly be deemed as effective. In turn, your environment will only do harm to your mobile app testing and hinder your app releasing. The best way to avoid it is to start with small test scripts and small setup. It’s suggested to run small scripts on small setup to verify the stability and scalability of the environment.

Once you’ve passed the above stage, you need to think about expanding the size of the environment to speed up the app releasing cycle further. Testing on 5 devices at a time vs. testing on 100 devices at a time – which one is more effective and efficient? From manual testing to test automation, you cut the testing cycle from weeks to days. And from 5 concurrent devices to 100 concurrent devices, the testing cycle is shortened to hours from days. This changes a lot on the speed and frequency of app releasing.

Take Dedicated Devices and Infrastructure in Place

Shared public cloud with thousands of real Android and iOS devices is beneficial to small companies/teams who have limited budget, yet are determined to bring the high quality to end users faster. For SMEs and Enterprises, setting up a dedicated devices and maintaining own infrastructure will take the app release speed to the next level.

Besides the customization features, the readiness and vacancy of devices in a dedicated environment means more in terms of expediting app releasing cycle. With reserved and dedicated environment, every team around the globe can get easy access to any of the devices at any time, literally 24/5.

• Developers can get daily and nightly builds tested at an early phase and fix the bugs where it can and minimize the efforts that are needed for the next.
• Unlike shared mobile device farm, QA teams with access to dedicated infrastructure and devices don’t need to compromise to the waiting queue along with many other small market players from different industries.
• At the same time, Operation teams can in no time connect new published devices or upgrade the OS versions of the existing devices.

With every team getting job done quickly, that facilitates the workflow of app releasing for new devices and helps to capture news users of the latest flagships as early as possible.

If you are coming to IBC 2016 and would like to schedule a meeting with us, please contact jouko (dot) kassila (at) bitbar (dot) com or robert (dot) seege (at) bitbar (dot) com.

Happy testing

Dear Users and Customers,

We published our new website last week and I’d like to take this opportunity to thank everyone at Bitbar for working so hard on our new products, website and basically all things that we produce for you. Most importantly, I’d like to thank all our customers and especially those who provided feedback on our new branding and website implementation. Thank you! 

New Bitbar.com – All About Mobile DevOps

The new bitbar.com provides highly useful business-critical information and access to the world-class mobile devops tools that we build here at Bitbar. As there will be lots of more coming in the near future and we constantly keep adding more goodies for mobile developers, testers, and organizations that want to utilize optimal devops process our goal is bring more tools and technologies in this space.

From the company point of view, we now have two different product lines – one with mature and significant user base – and new one which is now in private beta. We’ll keep informing you about the new product, its capabilities and trial options as we go forward during next few weeks and months.

Testdroid Transforms to Bitbar Testing

The old ‘Testdroid’ is still our core technology for mobile app testing products. These products are used with different deployment options to make mobile apps, games and websites robust on real devices. Our public cloud (Testdroid Cloud) still provides the most largest device farm available online (with over 500+ unique models, 5000+ devices in total).

Our private cloud option (Testdroid Private Cloud) has been also extremely popular among developers that do not want to host their own devices. With this option we provide Infrastructure as a Service type of service where everything from hardware to software and devices are taken care and hosted for our customers.

In addition, the popular on-premise product (Testdroid Enterprise) enables our customers to build any size of mobile device labs for their own premises, with their own devices. The On-Premise solution has enabled our customers to build device labs with close 1000 devices (both Android and iOS). As there are no real limitation with scalability and number of devices that can be used with our On-Premise solution, any size of mobile device labs (internal device farms) can be build using the product.

The New Product – Bitbar Monitoring

We’ve been working extremely hard to build something really unique for mobile monitoring. Our mobile monitoring product focused on native Android and iOS apps, using real networks and user conditions, is soon available publicly.

Stay tuned for more! Soon, you’ll hear something amazing from our Bitbar labs…

An increasing number of apps require an email or SMS authentication code for user registration. The added layer of security helps to prove user’s authenticity and identity, decreasing the chance of fake users trying to exploit them. But what does it mean for mobile test automation, when regression tests should be able to continuously click through the registration process without user intervention?

There are few tried and tested workarounds to implement two-factor authentication to be a part of mobile test automation. In this blog, we’ll take a look of the most common one and how it can help testers and developers to overcome with authentication issues.

A generic two-factor registration process consists of three distinct steps:

1. User enters his or hers information and phone number and / or email address to the app
2. The app’s backend sends an authentication code via SMS or email
3. User reads and enters the code to the app

Entering user information using test automation is a relatively trivial task, but in the second step problems arise. Most apps do not have an integrated SMS or email app, so reading the authentication code would mean navigating to a different app on the mobile phone. Adding functionality for navigating different apps to read authentication codes should be avoided at all costs, because it would add unwanted dependencies and failure points. What alternatives are there for reading the codes, without unnecessary code overhead?

SMS APIs

Numerous online SMS services such as Twilio offer sending and receiving SMS messages using virtual numbers, allowing SMS access through clean REST APIs over the internet.

Many of these services, for obvious reasons, focus on outgoing messages, and allow incoming messages only to be rerouted as an HTTP request to any given address. Without any cloud based storage of messages, using these services would require setting up a dedicated web server for receiving and saving the messages, so they could be retrieved by the test executing on the phone later. In some cases this might be a valid solution, but setting up new hardware and maintaining it always costs money.

Another problem with online SMS services in our case of authentication is the nature of virtual numbers. Messages between virtual numbers are usually blocked by the providers for security reasons. As most authentication code SMS messages are sent by these online services (using virtual numbers), receiving the messages to virtual numbers is not possible. Sometimes the blocks can be disabled for example for development environments, but generally another approach is needed.

Virtual numbers backed by real SIM cards

To overcome the block of virtual to virtual messaging, a virtual phone number that appears to be a regular number is required. Fortunately there are services which provide an online number that is tied to a real SIM card’s number. One of these services is Google Voice, which allows SMS messages to be received directly to the user’s Gmail account.

To enable a Google Voice number, the account needs to be activated with a real phone number. However, after activation all messages to the Google Voice number can be read directly from the Gmail account without a real phone or a SIM card. Furthermore, the Google Voice number is not considered to be a virtual number by other services, so receiving authentication messages should not be blocked!

Authentication at Test Runtime

Using the tools discussed above full two-factor authentication can be achieved fully in test runtime using only the test’s code. A lightweight POP3 or an IMAP client can be easily integrated into the test framework, and when it’s time to receive the SMS or email authentication code it can be read directly from the Gmail account with no extra hardware and very little code overhead.

In addition, there are alternative two-factor authentication methods that can be used as part of the mobile app testing and test automation. Get in touch if you like to know more about those!

In the mobile development, Agile and DevOps are widely used terms and majority of mobile teams exercise either one of these ‘practices’. However, there are differences of how organizations have adopted both Agile and DevOps culture in their daily doings. Some may even see agile and devops similar manner.

We’ll take a look at how these two popular and well-adopted methodologies/cultures are different from mobile testing point of view. For starters, I’d recommend to read the definition of mobile devops.

This topic has been discussed numerous times here at Bitbar blog for mobile app testing but we’ve covered topic few times also in our monitoring blog. The new way of thinking Agile as part of the development flow can achieve various benefits for organizations to produce yet better and robust mobile apps and ensure the quality criteria during all phases of the development.

To build and manage operations efficiently and create those features that drive your app usage up and forward, plus provides a great user experience, testing with continuous integration and delivery is the core of the process.

Agile vs. DevOps – The Definition for Mobile Development

At the most simple definition, agile development and methods  adheres to the principles stated in The Agile Manifesto. This is where individuals and their interactions are done over processes, tools and the method is driving the organizational behaviour. This also includes various other interactions, such as creation of documentation and facing with customers and following the plan. And as said, agile methodology and its practices have been adopted differently by organizations and teams.

The DevOps and especially Mobile DevOps is a newer term, used to describe how two different roles, doers come together and change and improve doings by better communication and collaboration.

This is one way to look Mobile DevOps from organizational point of view. Another is to look how Mobile DevOps can bring a process change and how organizations can achieve better level of integration with their devops tools and processes. 

When comparing the communication between agile methodology and DevOps, the agile way of doing thing includes more scrum meetings, daily habits where everyone is kept on the same page. On the other hand, Mobile DevOps strives to involve more specifications and documents where practices, checkups and operational instructions are presented. Regardless of which methodology or culture we’re looking the process goes however the same way:

And there is a distinctive difference to waterfall as the process is iterative in each of its steps. Each and every sprint and operational procedure goes through these steps. This is where DevOps practice or culture has its strengths compared to strictly siloed and separated groups of doers.

Even with agile methodology in use, each step in the Mobile DevOps must be carefully evaluated and measured for efficiency and effectiveness. That eventually makes the process more productive, delivers value, and maximized the critical business advantages. If the process is too slow some things can be boosted up, paced up and even eliminated if those do not have any impact on the results. Mobile test automation can really help in enabling continuous, frequent and instant releases that can be then quickly integrated, deployed and testing in automation context, and in timely manner.

Agile vs. DevOps in Mobile App Testing

To gain benefits from adopted agile and devops practices in your organization, it’s always recommended to include testing as part of the process. Testing – and using test automation – as part of the sprint and daily even after every build, the problems arising from your mobile app can be quickly spotted, communicated back to engineering and quickly fixed. In addition, the recommended process for getting application thoroughly tested and verified in the real environment – not only on real devices but real user conditions (network, physical environment) mobile monitoring can be used as part of the process.

Testing in agile typically means that tests and test scripts with automation are executed as often as possible. With Mobile DevOps this typically means that continuous integration is utilized for helping to get this done, in automated and timely manner, with continuous testing. 

The level of automation is varies based on organization’s habits and adopted processes and tools. For instance, agile practice encourages to test as much as possible, but in mobile devops the testing is always seen as automated and tests are running against each and every build produced from the build system and continuous integration. The philosophy behind Mobile DevOps testing is that everything should be automated and that’s probably the most significant difference between these two.

Also, when looking at what tools are used in each practices, there is not that much difference. Continuous integration, delivery and deployment are seen slightly differently, as agile practice only considers used CI system as a step forward, advancement, but in mobile devops culture it is absolutely a must for organization to achieve full automation and seamless workflow.

The major difference is also when both practices approach the builds. In a common agile process every sprint ends with an app that should be deployable and used by the end-users. As we all know this isn’t always the case, but in theory each sprint should produce significant increment to the product. The Mobile DevOps approaches this with even more strict policy; every build should be deployable and able to be used by the end users.   

Finally, as the automation is praised and highlighted within both approaches, the scalability and parallelism is critical as it brings the real dimension to scale up the test automation, runs built app on any number of devices, get instant feedback of performance, behavior and the status of the app. 

On how many devices is your application tested during the development process? Let us know in the comment section below!

Reserve your seat now and mark your calendar! We’ll host a webinar together with Greenhouse CI that will provide you tons of great information about continuous integration combined with a real device testing on cloud. The webinar will include a Q&A session at the end so join us to ask our experts the latest and greatest tips and tricks for combining continuous integration efficiently from cloud with real devices. 

Hassle-Free Continuous Integration Combined with A Real Device Testing

Date: 11th of October 2016
Time: 11am PST / 7pm GMT

Target audience: Mobile developers, DevOps, Testers, QA, Managers

Thousands of mobile apps are developed and released every day. The fierce competition for attention of end-users has changed the way mobile developers build and test their apps – and how they utilize today’s mobile devops tools to gain advantage of productivity, effectiveness and speed.

Join this co-hosted webinar by Bitbar and Greenhouse CI to learn how cloud-based continuous integration is the perfect fit for real device testing in the cloud. The webinar will include a hands-on example and provide lots of tips, tricks and best practices on adopting cloud-based productivity tools for your mobile devops process.

In this webinar, we’ll cover:

• The top bottlenecks when building your apps, and how to tackle those
• How to adopt mobile test automation as part of mobile devops workflow
• Break-down of mobile devops processes, tools and methods
• Live demonstration of building, testing and deploying a mobile app using cloud services

At the end of the webinar there will be 15 minutes allocated for questions.

Presenters

Nikita Avvakumov – GreenhouseCI

As a back-end engineer, Nikita spends his days maintaining and improving Greenhouse’s CI service, implementing new features and keeping an eye on its infrastructure. Prior to joining Greenhouse, he worked as a web developer on both commercial and open-sourced non-profit projects.

Ville-Veikko Helppi – Bitbar

Ville-Veikko heads the marketing and ecosystem building operations at Bitbar. He has more than 15 years of experience in the embedded engineering, business and leadership roles. Ville-Veikko holds an MSc in Technology and an MSc in Economics and Business Administration from the University of Oulu Finland.

Dedicated mobile devices in cloud has been one of top feature that device farms have to offer for its customers. While majority of indie developers love to use public cloud for maximal device coverage with reasonable costs, many of larger companies want to dedicate mobile devices for their needs – and spread the use of those across their teams. This sort of product has been available for few years with our Bitbar Private Cloud (previously Testdroid Private Cloud) and it has provided the business-critical feature and tons of benefits for many top mobile app and game companies.

In this blog, we’ll take a look at why dedicated devices are a superior feature and how to quickly jump on board with it.

The internet is full of comparisons of public vs private clouds – and to add another dimension to this – on-premise clouds. Each and every one of these has its own characteristics, benefits and features tailored for certain types of users. While public cloud is extremely popular with companies that want to minimize the costs, but to gain device compatibility across all possible device variants used by end-users, private cloud and on-premise clouds can provide the same benefits – and yet more efficiently.

There are 8 facts about dedicated devices (compared to public, shared devices) that every developer and user of cloud-based devices should be aware of:

1. Unlimited Device Time and Unlimited Users

Dedicated devices provide the 24/7 timeslot for its users. When considering the costs versus available testing time, there is no better option that dedicated devices in terms of cost efficiency, accessibility and easiness of use. With dedicated devices – regardless of usage with any product deployment option – user only needs to select devices and testing on those can start in a blink.

With dedicated devices administrator can configure priorities and users that can access these devices in fashionable, modern UI with few clicks. The unlimited device time and unlimited number of users (inside the organization) is a great platform for distributed teams and all maintaining will be taken care for – as a devices-as-a-service.

2. Clear Separation for Usage, Accessibility and Priorities

Dedicated devices provide the ultimate scalability in terms of usage, access and priorities. When compared to shared devices, dedicated devices make it easy to allocate certain devices for manual testing, certain devices for test automation (either or with full access to Android and/or iOS) and mobile web testing. 

The major benefit of dedicated devices comes from the separation of how these devices can be used by its users. Manual testing occupies the devices and the test session can take whatever time so it won’t have any impact on test automation scripts as different dedicated devices would be used for that purpose. In addition, dedicated devices are easy to break down for each app team, QA team or even provide access to those externally.

3. No Interruptions or Delays for Mobile App Testing

Continuous integration is a heart of mobile app development and agile methodologies are widely used by app developers. When continuous integration gets its job done the test can instantly start on real, physical device on cloud. As there won’t be any other tests running on those dedicated devices test runs can start right away, without any delays or interruptions by higher priority customers or test runs.

In fact, it actually means like you’d have a device on your desk and as soon as CI has pushed test for that device, the test run will start on cloud. In addition, the superior queuing system with CI and cloud combination takes care of getting all test runs executed. Even the queue gets huge all test runs will be executed and results will be delivered back to cloud dashboard and even your CI environment.

4. Special Configurations Are Easy to Enable

Not every test automation adoption is the same and because of this dedicated devices can bridge the requirements and actual implementation together, and make it more productive for teams and users. Admin can quickly configure device access, details and permissions, as well as assign or remove access to specific type of user.

Furthermore, dedicated devices provide a way to tailor test automation, used framework and other test details for specific use case. For instance, Bitbar Private Cloud provides support for pretty much ANY test automation framework (open source, even custom and customer-specific ones) in the real device context.

5. All Required Software and Hardware Hosted for You

One of the most compelling reason to use devices on cloud is that everything is taken care for your needs. You don’ t have to setup any hardware, no need to install any software, and typically there is no need to configure any of those devices to match with your local environment. 

This speeds up the entire development and delivery flow as your devops time doesn’t get wasted with fire-fighting devices. 

6. Security with Data and Device-Governance

Access to dedicated devices can be bridged via VPN – and everything on cloud-end is SSL secured. This includes all data from test runs, applications and results achieved from those tests, are all encrypted. In addition, all login details are saved & allowed only under authorized access and specified networks. For instance, dedicated devices hosted on our premises are located in our secure lab premises with restricted access. 

When an user or a team gets access to dedicated devices, they are the only user for those specific devices. Unlike with Public Cloud that devices are used by variety of different types of developers and testers, these devices do not show up outside of its user. There won’t be any visibility to device, test runs, usage, results or data. This makes dedicated devices the super secure option for enterprises to ‘outsource’ their device-governance and hosting to external provider.

7. Feature That Scales Across Deployment Options

Dedicated devices is a feature that spans across different device farm deployment options. It basically enables getting started with Private Cloud or Public Cloud and then scale to On-Premise or even larger setup of devices around any of these deployment options.

8. Remote Access to Devices: Manual, Automation and API

The thing that really makes testing on cloud-based devices flexible is the remote access in three different levels: manual testing using a device on a browser, full test automation with frameworks and integration with API to remove all manual interactions for your device testing effort. 

With test automation and API users of dedicated devices can scale parallel test runs to cover any number of devices they have configured for their system. With our test automation practices there is no limitation on number of how many devices can be used simultaneously. We simply do not limit parallelism and concurrent test runs anyhow.

The recent launch of iOS 10 has sparked some serious discussion in test automation ecosystem and among framework developers. Several test automation frameworks have had issues with the latest iOS version and this have forced developers and testers to tweak their testing environments, scripts and tool setups.

We’ll take a look at the state of the art with iOS 10 and how to overcome with those issues with Calabash.

From UIAutomation to XCUITest

The most significant change in test automation between Xcode 8 and Xcode 7 is the UIAutomation support. Apple has now removed Instrument’s UIAutomation from Xcode 8 and replaced it with Xcode’s UI Testing Infrastructure (XCUITest).

As UIAutomation is not available anymore in Xcode 8, all uia_* calls now raise an error when tests are executed.

Calabash

Calabash also have had some issues with iOS 10 and basically running Calabash tests on devices with iOS 10 requires the following tools/setup:

• Calabash iOS server version 0.20.0
• Xcode 8 (the latest version)
• Ruby >= 2.2 (preferably the latest)
• run_loop gem >= 2.2

As Apple removed UIAutomation each and every framework have had impact with test script execution. Calabash’s bridge between Calabash-iOS and Xcode command-line tools is called DeviceAgent, and it is based on XCUITest framework.

To update your Calabash environment and version, just execute the following command:

$ gem install calabash-cucumber

If you need to update your Xcode version you can also use DEVELOPER_DIR environment variable, or simply install it over the older Xcode version. NOTE! If you want to run tests on iOS 8 (or older) you also need Xcode 7 installed. Because of this, we recommend having both versions installed and available.

If you are using MacOS Sierra you basically need to use Xcode 8 as the old Xcode 7 isn’t supported on it.

According to various studies, many of retailers are still prioritizing customer facing programs over strategic infrastructure improvements, and choosing to invest in data capture for marketing purposes and pricing programs over mobile platforms and improved delivery. This is really a surprising finding as mobile-first initiatives are spreading across verticals and domains – and soon a significant majority of consumers use mobile web as their first data point to make their purchasing decisions. 

How can brands ensure their quality criteria isn’t compromised and how to build better user experiences for customers? 

First of all, according to these studies retailers are still stuck with non-agile and non-devops – and that means the app development is way too slow to meet any sort of requirement for faster time to market. It’s just in line with this study that retailers looking to develop mobile applications may be facing a hurdle on the development side as a growing number of businesses have a backlog of mobile projects that can’t get out of the development phase.

Because of this, application development with the right approach, tools and environments, mobile marketing and electronic/mobile-commerce are the top initiatives and priorities among retailers.

In addition, there are significant differences between Agile methodology adoption and DevOps culture inside organizations. When it comes to mobile app testing and development organizations should consider only integrating their internal development and operational resources with real device testing on cloud.

Can Mobile App Monitoring Help?

Today’s mobile app developer organizations are increasingly investing in monitoring systems that can capture production, development, and bunch of other metrics all in real time. This growing trend is great, but if monitoring is done only for specific silos of the overall flow, results may not expose how to optimize the most important resource hogs and therefore monitoring only produces imperfect results.

The mobile app monitoring itself doesn’t help any mobile app to make its way to success – but the right tools and methods used in development and testing process will. Also, there are various different ways to use mobile monitoring, However, mobile app monitoring is extremely efficient way to improve visibility and transparency of resource problems, performance and user experience when native mobile application is still under the development.

Too Slow Mobile App Development is Hitting The Revenue

There are couple of important aspects why every company that produces apps for consumers should be looking to accelerate the delivery of their mobile apps and use variety of channels to provide those capabilities for consumer to buy – and among those, web experiences:

1. Very few consumers ACTUALLY USE mobile apps provided by retailers. Despite there is an app available for every need (or anticipated need that consumer might have) those apps aren’t used. Typically, those apps get in lost in the sea of apps.

2. When those apps aren’t used in volumes and for specific needs, retailers – or whoever that builds those apps – do NOT get feedback or cannot tweak functionality of those apps to be more suitable for consumers.

3. Furthermore, many of those retailer apps are getting SLAMMED due very bad functionality and quality. If the app gets 1.5 stars in any of those app markets, it is not likely to pop-up in search for consumers looking new apps.

4. One of the topics for retail apps has been also compatibility with the logistical back-end systems. This is naturally where raw testing with variety of different configurations will help to understand what sort of problems could there be and how to fix those.

With the average application Dev & Test & Delivery process taking between three and 12 months, this slow app development cycle is resulting in competitive disadvantage. Mobile teams must use tools and techniques that match the increase in mobile app needs within their organizations.

The Mobile DevOps Shift-Left Has Changed Things

This is why mobile devops culture and adoption has streamlined the entire development flow and made things getting out faster.

Instead of traditional 4 step process – plan, develop and test, deploy and operate – there are actually more flavors in the process that should be emphasized. Especially when it comes to considering what tools and techniques are used, how those fit together and enable seamless tool flow from planning to production. Mobile Test Automation is without shadow of doubt critical part of the flow and enables process to be highly iterative, repeatable, and robust. Automation is actually an enabler for continuous everything, in each step of the process.

Now, it’s worth of mentioning that agile methodologies and mobile devops culture doesn’t compete against each other, but they can significantly complement each others benefits. 

From Complexity to Agility – Dev & Test & Delivery

Mobile development and delivery are complex processes that require independent services to be delivered in a coordinated, reliable, and efficient manner. All enterprise mobile teams should aspire to continuous improvement in these ways:

1. Release planning – small, frequent releases

2. CI/CD — each team’s work is continuously integrated with that of the development team and IT Operations

3. Continuous Testing and all its flavours

4. Continuous Delivery — automated deployment of software

Naturally, we are in the business of embracing continuous integration, continuous testing, rapid feedback and continuous delivery. Automating this process increases reliability, quality and ensures that the app delivering what was required.

Continuous testing is the most difficult part of the to application lifecycle process but can yield the highest returns in terms of user time and perceived application stability. Bitbar Testing professional services can significantly help.

The rise of mobile brings major changes to traditional development methods. As Gartner explained at their recent Digital Workplace Summit, IT must adjust the way it manages product releases as the cadence of mobile release cycles speeds up.

In short, enterprises cannot approach a mobile development project in the same way that they approached traditional development projects in the past. Efficient and customer-satisfying mobile apps and especially development of those requires agility. But it also requires and delivers short release cycles – and delivers better return-on-investment.

Get in touch and we’ll be happy to help you out.

In competitive world of mobile games, gamers aren’t really shy nor they will turn their faces away from problems they confront when playing mobile games. Instead, they tend to provide pretty straight feedback on issues, games and sometimes even non-related things – because of frustration and emotions around the gaming experience. In addition, there are significant differences on how gamers take your product in different countries. 

The fact is that feedback from gamers DO get easily in front of hundreds of millions of other gamers – and that can impact on your revenue generation. How can mobile game developers tackle these challenges and how do Mobile DevOps approach come into play? And mobile game testing is done? We’ll scratch the surface of that in this blog.

What Drives Mobile Game to Become Popular

Gaming on mobile devices (phones, tablets) has been improving at a far greater rate than any other technology that came before it. Just few years ago there was no powerful and capable-enough hardware (devices) that would provide great gaming experiences. Simply, there were no software that would run on those low-end mobile devices that could bring photorealistic graphics (console-quality) into your handheld, and there were no ecosystem to support, build and foster sort of gaming boom we experience today.

And as this trend has been evolving significantly during the past years, all sizes of game studies have entered in to this extremely competitive landscape with their gaming products. 

As we’ve been discussing about the new Agile process that empowers game developers to build better and more robust mobile games, the quality metrics are something that are built when the actual game is developed. Therefore, the quality and building that up should be considered to be part of the actual development process, used tools, methods and frameworks.

The DevOps Process for Mobile Game Development and Testing

To build and test efficiently – and naturally let nothing to compromise those killer features – drive the mobile game up and forward, plus provides to offer a great user experience, testing in continuous manner is typically in the epicenter of the process. However, testing on real devices is just one thing that must be done right. Another important aspect is of how all that testing is operated and how results are measured during the development.

This is where Mobile DevOps can play a significant role ensuring testing environment, criteria and goals are meet. And as today’s mobile games are extending their life with new builds/releases/upgrades, testing all the new great features before it pushing it out for users is highly important. The process of testing looks somewhat like this:

While more traditional testing occurred close to the release, testing in mobile devops is done continuously, on daily-basis and it provides instant results on how issues are found, and how those can be fixed. The architecture for mobile game testing isn’t that different from regular mobile app testing but it includes bunch of real devices, combined with the continuous integration and used gaming development tools (dev tools, game engines etc.).

There is no room for manual testing in the mobile devops process as simply anything done manually will make the process much slower, less productive and badly inefficient. But that’s what we’re about to tackle here with full automation ready platform that can be used to automate all possible things affecting the productivity and bad overall performance by the developer organization.

The solution for faster development-testing-deployment iterations and instant error reporting results as significantly higher quality with a mobile game. The better quality comes from the test automation, efficient use of continuous integration, real devices testing, and with the use of advanced testing methods.

Will Mobile Monitoring Help Building Better Quality Games?

Mobile gaming has been booming for years and the same phenomena is going on for online gambling as well. During this time new ways to monitor mobile games on real locations, using real networks have showed up. However, doing that with native mobile game on real environment hasn’t been existed. The thing where mobile monitoring can make the difference is that it can instantly show if users are not being able to played with it, and if this has a relation on why they leave those games behind. 

With synthetic mobile monitoring not only performance aspects of products can be measured but also the health level of the system if more bandwidth needs to be added in the back-end implementation, more hardware or anything else to making the gameplay smoother for end-users.

How DevOps Will Help Building Better Mobile Games

Agile organizations are so used to having daily stand-ups and doing everything in agile way. That’s the process of how everyone is kept on the same page. In addition to this, Mobile DevOps strives to involve more specifications and documents where practices, checkups and operational instructions are laid out and everyone can follow the process. 

Sure, again: game ratings, performance/slickness of that game, and compatibility across all possible gaming devices matter and have a direct impact on installations: the better ratings, the better position in search. But those are the goals that both agile and devops adoption can bring to the table when used properly as part of the development, testing and deployment process. 

“Time-to-market can be improved with continuous integration, real mobile devices, the right test automation frameworks, AND the right mobile devops culture wrapped around it”

The sooner the mobile game gets tested, made robust and published for hundreds of millions of users, the sooner the revenue starts pouring in. Any mobile games’ time-to-market counts not only against competition but also when the revenue starts rolling in. Simply, time-to-market can be improved with continuous integration, real mobile devices, right test automation frameworks, AND the right mobile devops culture wrapped around it!

And it’s not all about competition. Some games are taking gamers by the storm way after their competition (other games) where published. Generally, the rule of thumb with mobile games is that “be the first, skim the cream” and that’s where mobile devops can help to get that mobile game published. 

There are lots of variations in end-user platforms (ie. devices) that mobile games should support (for sake of revenue generation) and ever increasing complexity in games that need to be tested on those end user devices in every release. If you come up with a bad release, you’ll drive big chunk of gamers out – and that has a direct impact on your revenue.

With larger game publishers it’s very common to build large internal game testing labs with full test automation, great devops practices around it – or alternatively – outsource all of that the expert. However, that expert still should be one that goes the mobile devops path, using test automation, real devices and making efficient process that cumulates as a higher quality game.

The Most Critical Mobile DevOps Tools for Mobile Games

Continuous Integration and Deployment

The use of automation is already in the core of how developers build their software. Companies rely on continuous integration tools (like Jenkins) that bring easiness, clarity and result-driven execution for all steps: development, testing and deployment.

In fact, continuous integration provides a workflow to help managing code and its regressions, and then enabling efficient testing of those builds. With this sort of Agile approach, continuous testing can also happen every time code changes in repository.

Game developer organizations should focus their efforts on building new features, versions, enhancements, and take care of bug fixing, while test automation produces results every day, and the entire test setup is managed by the devops. For instance, building a new version of app can take a place every day and then that application gets tested – before the next release takes a place. It provides efficient way to improve the overall game quality, find out and rub out the bugs instantly and not to drag those with the game, release after release.

Use of Real Mobile Devices with Test Automation

Despite of some believes mobile games can be tested with various test automation frameworks produced as open source. These frameworks provide a way to implement any sort of test scenarios, test scripts and cases around mobile games. When building test scripts for a mobile game it’s a no-brainer that developer organization strives to use as authentic environment for testing as possible – and this means the use of real mobile devices. 

After the game (in form of APK or IPA) gets built by the build system, it can be automatically send for its first test (the smoke test) to see how well it run on devices and is it even compatible. Each regression can be tested (regression testing) with the existing test scripts for certain specific function. For instance, if the mobile game relies on back-end services, this connection can be automated with sophisticated test scripts after each build.

In addition, many other aspects can be also automated: localization (testing different languages), different types of performance tests (load, spike, stress and others), connectivity and hermetic testing (isolation of game from network, other services etc.).

Now, are you using devops approach/do you have a devops culture when you build your mobile games? Let us know in the comment section below! 

The Mobile DevOps practices and Continuous Integration/Delivery together enable Agile development. Basically all these together are fully complimentary. As we’ve been discussing Mobile DevOps is a methodology and a set of (good) practices where Continuous Integration/Delivery is more like an implementation of that. And for mobile app testing it has lot of great things to offer.

The most important things in mobile app testing are the efficient use of test automation (using real devices), the right test automation frameworks, and efficient bug catching/handling and notifying developers about those. When it comes to parallelism and concurrent, simultaneous test runs on real devices, that’s what test automation together with CI/CD system is all about.

Mobile DevOps with Continuous Integration Enables Agile

The CI/CD systems and processes built around it push code out to testing as often as the developers need it to happen. Typically this is done several times per day or at least daily (or nightly) – and sometimes it can take days to get new build out for device testing.  In a nutshell, CI/CD enables efficient use of Agile methodologies.

The effort of maximizing the efficiency and effectiveness will easily improve the overall productivity. What this means in mobile DevOps is that test automation must be included in the process and connect it with continuous integration. Each and every build should be tested against the real environment. 

Then, what is the real environment? With help of test automation developers, testers and anyone behind the system can ensure the device compatibility is meet and defects are instantly spotted out. When the problem occurs, developers should get indication of that instantly, with all possible details and hints where the problem lays.

This speeds up of the delivery pipeline can push organizations to get instant results of their Mobile DevOps culture. The culture itself can reduce costs, as things get fixed earlier, and more importantly – make the outcome more robust and enable better quality.

How Mobile DevOps Can Enable Concurrent Mobile Test Runs

There are a myriad of testing targets when your consider how many different OS versions, hardware setups with all its diversity, physical networks, user conditions and experiences that must be delivered. Running something in simultaneously will get great details of how things actually perform.

We should start by defining what is meant by simultaneous test runs – and generally parallelism – in mobile app testing. As there are lots of discussion whether these are the same thing or not, in general, concurrency is a concept where two or more tasks are started, executed and finalized in overlapping periods of time. For example, having multiple threads running on a single-core chipset. The concept of parallelism is probably more straightforward as it is where two or more tasks literally run at the same time. For instance, having multiple threads running on a multicore chipset.

Is that all the same when looked from mobile app testing point of view: running test scripts simultaneously on variety of real mobile devices and getting results? The most distinctive difference would be when you get those results and whether devices start a test run as soon as test has been launched. 

However, the key point is that concurrent test runs do not necessarily run in parallel. For example, if the device picked up for a test run is not available, your test run maybe be in queue and once the device becomes available you’ll get tests finalized – and naturally all tests runs separately and only when test run is done.

“Nothing That Is Done Manually Is Hardly Agile”

Developing mobile apps is very different from developing desktop software and even embedded software. Mobile app development is meant to be agile, and lots of great tools and practices have been developed for that agility. However, doing something manually — such as testing an app — is never agile, which is why test automation has shown tremendous growth among app and game developers, speeding up their doings and yielding robust and better results. Manual testing is neither providing you the core benefit what comes with test automation – the parallelism. You can simply run one device at time.

NOTE! Despite of all this, sometimes for a sake of exploratory testing, the manual test session is required. The remote manual access to devices complements this offering and occasionally developers want to make sure their apps work just like they’ve planned – and this is where our remote manual access to mobile devices is extremely useful. But don’t get stuck on it – that will eat all your productivity!

The Process of Mobile DevOps and What’s Included

The actual development is closely related to continuous integration. The agile approach made continuous integration very popular as it enhanced the way developers work, collaborate and integrate their doings together, in natural way. The continuous integration with regular daily routines produce lots of data about the mobile application, regressions and compatibility of each developer’s contribute, and discovering issues early exposes integration flaws, even incompatibility of developer’s doings, and makes better time consumption and schedule estimating easier and more accurate.

Any tools and additional practices around the following process can enable simultaneous tests done on devices. The code is developed, maintained and built for test automation runs. And after all test iterations are finalized the deployment to production can happen. The deployment to production in mobile app context means publishing the release for its users. 

There are only few great DevOps tools mentioned in the picture above, but there are plenty of other great ones too. But let’s focus on test automation a bit more.

An Ideal Test Automation for Mobile DevOps

Mobile test automation utilizes real mobile devices and offers the possibility to test mobile apps instantly and effectively using test automation frameworks. The infrastructure should be build to support unlimited number of simultaneous test runs, meaning that users can select any number of devices for their test runs.

The actual benefit of test automation comes from simultaneous test runs, automated access, coverage, and runs on devices, repeatability nature of tests, and identical and accurate tests – that could probably never get to accurate with manual testing.

There are basically 4 requirements for organizations to meet the ideal flow with development, testing and deployment, and those are as follows:

1. Use Open Source and Standards 

Test automation should rely only on open standards that are widely used in the industry. Open standards mean there are no vendor lock-ins, source code is always open and available, and developers can incrementally develop new tests and focus on compelling features with their apps.

2. Only Automation Boosts Productivity 

Automated testing can increase the depth and scope of tests and significantly improve software quality. Lengthy and thorough tests that are often not doable with manual testing can be run automatically. Automated tests can easily execute thousands of complex test cases during every test run, providing coverage that is simply not possible with manual testing.

3. Cost Efficient and Effective Use 

Depending on the size of the project and the application, test automation will always provide a good return on investment. Once tests have been created, automated tests can be run over and over again at no additional cost. Test automation typically reduces the time required to run repetitive tests from weeks to hours. This is a significant time-saving that translates directly into cost-saving.

4. Integrate the Flow with Complimentary Tools 

To get the most out of your efforts and maximize testing coverage, select the most robust and most cross-platform method – and hook it up with your continuous integration, development and deployment system. Once you build an app, the system can automatically push it for testing on devices on cloud. This saves a lot of time, produces instant results and developers can fix any issues instantly.

In order to automate your mobile testing, you can use different types of procedures and methods to accomplish the real device coverage, compatibility and functional testing. 

The use of cloud-based devices have rapidly grown during the past years and there are plenty of different, great alternatives of how to deploy solution for your organization’s needs and practices. These deployment options provide flexibility to control device access and start any number of unmetered tests on those devices. The model of how much devices are used, what devices/models should be used for testing and many other things impact on decision which deployment options companies typically choose.

Today, we’re bringing something seriously useful and new to the solution roster – Dedicated Devices.

Introduction to Dedicated Devices on Cloud

Dedicated Devices is a compelling feature in Bitbar’s Public Cloud, Private Cloud and On-Premise that allows its users to dedicate (reserve) devices for their own use only. With the help of Dedicated Devices administrators who manage device access and allocation of those devices can quickly configure a set of dedicated devices for a specific user or group of users that fully control the device(s) and no other users have visibility of these.

Dedicated Devices will boost the productivity and enables companies to use more efficiently their resources, time and hardware with predictable monthly costs. And as a benefit there is significant time saving when certain devices are dedicated for only certain user groups or certain usage type like CI or manual testing.

Dedicated Devices Are Fully Configurable by Admins

The flexibility that devices-on-cloud offers for mobile app testing, game testing and development of both of these are great. Users can quickly select devices they want to test and develop on and change devices based on their needs and testing focus. Especially Public Cloud, which provides nearly 100% coverage of device volumes used globally provides an amazing testbed for app developers. 

So what do Dedicated Devices bring into this? Basically Dedicated Devices are available on Public and Private Cloud, and On-Premise solution. An administrator managing devices can quickly and easily assign a device for a user or group of users (under their organization) that makes the device(s) dedicated for that group. Changing the ownership and access is easy and quick from the admin panel – and just few clicks are required.

The Fastest Deployment and Device Management From Cloud

When building a private cloud with real mobile devices – or alternatively an internal test lab – hardware infrastructure is typically required, and frankly, lots of it in case of larger setups. Typically that’s not the problem when done properly and with the right instructions. If you are building an internal device testing lab with real mobile devices there are plenty of great resources to get into all details of how doing it right.

Dedicated Devices feature removes the need of acquiring  and statically allocating hardware for each team. They are quick to set up and reserve for any of your internal users. As Dedicated Devices can be allocated per user, per team or per organization just with few clicks and fully managed by admin users, you are always using your real device infrastructure with maximum efficiency.

Enterprise-Grade Security for Mobile App Testing

Security is always – and should be – the priority for any companies using external services. Not only that something is behind HTTPS but also how things are handled by the provider and what sort of methods are opened for users. 

As Dedicated Devices are a set of devices for only its users, those devices are not visible to anyone else. There is simply no trace of devices and those are not accessible by anyone else, except the admin and admin’s authorized users. Furthermore, no hardware details need to be configured and users can quickly get manual, automation and API access to Dedicated Devices. This provides the top security level for testing using Dedicated Devices and allows even direct VPN tunneling from dedicated devices to your behind firewall staging and development back-ends.

Centralized Device Management with Mobile DevOps

When it comes to running a device lab with any number of devices, this can be done with so many different practices. Running real mobile devices with Mobile DevOps is part of the DevOps responsibilities and making sure all devices are reachable, there are no jams, bottlenecks or failures to get test automation scripts started on those devices. 

If you use the On-Premise deployment option your Dedicated Devices can reside on certain locations and be allocated for your internal users when they need to have access on those devices. If you select to use Public Cloud deployment or Private Cloud deployment all the maintenance and keeping devices online is taken care for your organization. This complements the mobile devops best practices and makes accessing devices instant, flexible to configure and fully tailored for your organization’s needs.

Fastest Way to Automate Testing, Test Runs and Get Results

Manual testing has been known reason for slowdowns and making testing extremely inefficient. Well, that’s no-brainer as manual testers can really focus on one device at time. Running the mobile application on hundreds of devices simultaneously requires test automation and all test automation frameworks have been implemented to help their users to utilize the benefits of scale.

Manual testing session on remote device typically reserves a device and test automation scripts will end up queuing for their device time. With Dedicated Devices certain devices can be dedicated for manual testing and therefore manual testers can access those instantly, without need to wait a second. Devices can be easily separated for different user groups: manual testers and test automation users and the group compositions can be easily changed on the fly.

Clean Separation of Test Automation Usage and Frameworks

Have you ever compared e.g. Espresso vs. Appium client-side execution in terms of speed? One finalizes tests in seconds or few minutes and the other one takes 10 minutes or more – with comparable test pattern. Typically the speed is not the only attribute that should be compared as a framework that takes longer can do better job in terms of getting the app properly tested and provide a depth with test results.

Some teams and users prefer to use certain test automation framework. With Dedicated Devices admins can quickly reserve certain devices for those groups, with specific features, software and hardware available for those test runs. Espresso, Appium, Calabash, XCUITest or something else? Just dedicate different combinations of devices for users of different frameworks or different set of core devices just for Continuous Integration runs and the rest split between automation teams and manual testers.

Looking for using Dedicated Devices for mobile app testing? Look no further, this awesome feature has just arrived!

Have fun with Dedicated Devices!

Dedicated Devices for mobile app testing and development are allotted for a specific user or user group that can access the devices maintained and kept online on cloud. The device management aspect of dedicated devices on cloud is awesome: the device is online and available 24/7 – and it’s taken care by someone else. Also, dedicated devices on premises provide plenty of benefits as all devices can be centralized and mobile devops practices can make sure device is available for its users all the time.

What Dedicated Devices mean for their users, administrators and IT in general? We’ll take a look at these user groups and how they can get the best out of dedicated devices.

The fundamental difference of ‘dedicated devices’ and ‘dedicated mobile devices’ is that mobile devices contain more intelligence and they are typically regular mobile phones, tablets or wearables used by end-users. The old term of ‘dedicated devices’ has typically meant generic devices dedicated for one or few jobs: tape drives, printers or plotters.

The real value from Dedicated Mobile Devices come from the accessibility, maintenance, and easiness to adopt and use. Here are few examples of how users see Dedicated Devices differently and what they should know about when using those and managing devices.

All Bitbar solution deployment options provide its customers to host dedicated devices that are available and visible to only a group of users. This allows creating a device pool used only for continuous integration runs that need to be timely executed.

Below is explained how to setup using dedicated devices and what Dedicated Devices mean for different user groups.

Users – Developers, Testers, QA, Anyone with Device Access

Bitbar Testing users are typically developers, testers, QA folks and basically anyone who needs to either run test automation scripts on real devices on cloud or do manual testing with remote devices.

Users typically start with dashboard view. They see their projects on the left-hand side and can manage (create/remove/adjust) as they wish. 

Projects – the project management view provides a glance to all user’s projects, test runs and summary of results. This is the control panel for all test automation, test runs and getting tests executed in specific projects.
Reports – reports view provides capabilities to generate reports of projects and test runs, at any level.
Device Groups – this is where the user manages their devices that are used for test execution. Depending on which product deployment option they use there are all devices available devices view where users can create their own device groups.
Manual Testing – manual testing feature wherein user can manually access any dedicated device for a user.

Admins – How they control and assign devices

First of all, admins can assign and dedicate specific devices for users. To get started with this, admin should have administrative access to the solution (Public Cloud, Private Cloud or On-Premise) or alternatively authorized access to dedicate devices that are in their group of users.

First the method to identify devices needs to be updated to also include the ACCOUNT field.  This can be done from the Admin -> Devices -> Device Models table. On top right corner there is button () to edit criteria for device identification. New device model criteria should be as below.

This change can take a while as all devices in cloud need to be updated with this new criteria.

1. Enabling the feature. To enable the feature there MUST be ACCOUNT in device model criteria.

Once admin has add the device model there will be a few seconds delay as devices are getting processed and instantiated.’

Next a device can be assigned to a specific account. Find the new device from the Device Instances table and open device configuration window. There change device from Public to Dedicated and provide the main user account email that should be able to have access to the device. Device is then available to specified user and all of its sub-users and is not available anymore to other users.

2. Assigning device to a specific account. In admin devices one can assign device instance to specific account:

This action will create a new device model (if one doesn’t exist already). If no new device model is created, device model will be removed from all device groups in order to disallow use it for other users. Device is available to all users under the specified account (includes sub-users).

Furthermore, this device is no longer visible on public device list and this applies to all product deployment options: Public Cloud, Private Cloud and On-Premise.

3. Changing or removing device accounts. If admin selects changing or removing the device account, the specific device model will get removed from all device groups to in order to disallow use of it for previous owner. Device can be made visible on public device list for all users. There is no need to remove it from all device groups – previous owner can still use it as it’s public.

IT and Management – How Can They Follow the Progress

Dedicated Devices provide superior benefits over the public devices used widely with different users and user groups. Dedicated Devices provide ultimate flexibility to reserve certain devices for certain users – and that makes the test execution, and even manual testing, efficient using real devices from the cloud.

IT and management can quickly see the bottlenecks of testing effort, what issues arise with test automation and manual testing while the mobile app is getting tested in different phases of the development.

In addition to versatile test reports, the thing that really makes testing on cloud-based devices flexible is the remote access in three different levels: manual testing using a device on a browser, full test automation with frameworks and integration with API to remove all manual interactions for your device testing effort.  This provide IT and management fine-grained details of how testing is going and if there are any adjustments needed for devices, resources or frameworks.

Hello everyone,

Bitbar will be visiting GITEX 2016 throughout the whole event week. GITEX, annually held in October in Dubai World Trade Centre, is one of the largest technology exchange and trading events for Middle East and Africa. It is a perfect gateway for global brands to get a sense of ICT market, mobile ecosystem, software development, etc. in Middle East.

If you are also visiting the event, we’ll be excited to meet you there and talk about how our cutting edge mobile devops process have reshaped many of our top clients’ mobility strategy and how our software and devops solutions can strengthen your mobile team for the coming new year.

Send an email to Robert Seege at robert (dot) seege (at) bitbar (dot) com to schedule a face-to-face meeting.
See you in Dubai.