Factors to consider for Mobile App Testing

For Mobile app testing following are the success factors :

1. Connectivity-related testing: Connectivity-related testing Most mobile solutions depend on some kind of network connectivity. Solution design and testing are needed to address variable bandwidth, offline and flight mode scenarios, and validate user sessions moving between different network conditions. Both automated network simulations and manual testing in real network conditions are required to ensure consistent behavior.

2. Understanding Physical Characteristics: The physical characteristics in mobile solutions differ significantly from non-mobile solutions. Mobile testing needs to take several of these characteristics into consideration:

 • Screen size

 • Touch and gesture capabilities

 • Orientation (vertical or horizontal) and movements in three dimensions

  • Camera

  • GPS

3. Location Simulation: An ever-increasing number of solutions utilize location data and GPS integration. In these solutions, features are designed to depend on location or distance to other locations. Successful mobile testing ensures quality across different types of GPS implementations and needs to utilize efficient location simulation.

4. Dealing with Fragmentation: The market fragmentation for both operating systems and device types continues to challenge solution design and testing. Most solutions, both internal and external, need to support hundreds of device types and several versions of operating systems. A relevant mobile testing tool box includes: • Physical access to the major device type and operating systems combinations • Ability to run manual and automated tests across both physical devices and emulators • Access to cloud-based platforms to maximize testing a number of devices

5. User Experience: User experience is a main criterion for success. Testing user experience is an art that consists of both objective and subjective measurements. Key elements include: • Adherence to platform rules and guidelines, as dictated by app store review guidelines • User interaction – Level of intuitiveness and efficiency • Navigation – Adherence to platform rules and guidelines • Signup and login – Clarity and ease of use • Layout and user interface design – Look and feel, aesthetics, and precision in layout • Exception handling – User friendly messages and graceful exception handling

6. Engaging with Third Parties : Most apps are distributed through open and public app stores, each with its own set of guidelines. Apps that fail to adhere to guidelines may be rejected. And, given the third party review process, there is a bug fix latency inherent in updates. Also, when operating systems are updated, the new releases can break existing apps. This means that successful mobile testing must consider: 

 • Using common testing checklists based on the most recent app store rules and guidelines   • Using rapid testing cycles for updates in order to minimize impact of bug fix latencies

 • Testing existing apps on beta versions of operating systems

7. Performance: Mobile solutions can overload servers, through the rapid growth in the number of users and an increase in the average life span of transactions due to variations in bandwidth and latency. This can have a performance impact on all users. Also, there are significant performance differences across different types of devices. An app or responsive web may run well on a high-end device but may not be acceptable on a low-end device.

8. Security Mobile: solutions are implicit in an increasing number of scenarios where sensitive systems are accessed and private data is in transit or at rest, that is, stored in mobile devices. And given the physical nature of mobile devices, they are more easily forgotten, lost, or stolen. Mature mobile testing aims to:

• Secure confidentiality and integrity of data

 • Validate whether authentication and authorizing is secure

• Verify that systems are keeping records of events (non-repudiation)

In order to capitalize on the success factors outlined above, need to do following testing’s:

1. Mobile Functional Testing 
2. Mobile Compatibility Testing
3. Mobile Usability Testing 
4. Mobile Performance Testing
5. Mobile Security Testing

1. Mobile Functional Testing Mobile Functional Testing aims to validate the actual functionality of the mobile solution. The test approach is realized through a use case model in which each use case is documented, step by step, and run through to validate its actual results against the expected results. Use case-driven tests can be complemented by exploratory testing, in which the tester is given a more loosely defined set of goals to accomplish using the app.

Minimum set of areas to be covered during functional testing:

 • Functions 

  – Path testing

   – Boundary values (input)

   – Data lifecycle (using Create-Read-Update-Read-DeleteRead heuristic technique), onboard captors

• Application interoperability: 

verify interoperability between the most common applications embedded by the manufacturer and the most popular applications downloaded from stores:

  – Camera (movie/still) recorder, picture viewer, video player, music player, Internet browser, messaging manager, contacts manager, task manager, calendar, calculator, device setup

• Application lifecycle – Install by all means supported by device technology – Heuristic technique: Install-Use-Uninstall-Install-Use

• Network management

  – Flight mode, loss of connection, mode switching (GSM to Wi-Fi, Wi-Fi to GSM)

• Interruption management

 – Messaging, push notification, alarm, multi-display orientation, multi-tasking

• Display management – Multi-display orientation rendering – Multi-touch screen monkey test – Rendering and readability

2. Mobile Compatibility Testing:

At the onset of a Mobile Compatibility Testing effort, the range of platforms and specific devices is clearly defined. Once the scope is set, using our CTC tool we test on cloud based devices.

Mobile Compatibility Testing covers aspects of platform fragmentation, physical characteristics, and app store review guidelines

3. Mobile Usability Testing: The tangible deliverables from a Mobile Usability Testing effort include specific suggestions and advice on what to improve and change to enhance the overall usability of the mobile solution

4. Mobile Performance Testing : It’s important to maximize end-to-end performance by working through each layer of the mobile solution’s technology stack. Testing mobile solutions means measuring app performance, network availability and performance, along with system integration and back end performance.

 Mobile Performance Testing focuses exclusively on the mobile solution’s performance, disregarding functional and usability aspects which are addressed in other types of testing. We execute both manual and automated testing using market leading performance testing tools

5. Mobile Security Testing Mobile Security Testing is becoming increasingly important. Validating specified and implemented security measures often reveal critical security holes and threats. In a typical Mobile Security Testing effort need to check with tools to validate six common security aspects: 

 • Confidentiality: Does the app keep your private data private? We penetrate data storage locations looking for private data or data that should have been deleted during app exit. We analyze network traffic and validate whether or not sensitive information is appropriately encrypted.

 • Integrity: Can the data passed to and from the app be trusted and verified? We validate the integrity of the data being passed to and from the app by monitoring network traffic and, where relevant, validate whether or not the data is appropriately encrypted.

 • Authentication: Does the app verify the user’s identity to an appropriate degree of certainty? We test if the right level of authentication is implemented: For example, by validating the implementation of two factor authentication or by validating correct round tripping of mail- based confirmations. 

• Authorization: Does the app properly limit user privileges? We test whether or not server-based services are provided at the right level of privileges and only there, by trying to invoke functions or reach for data beyond the authenticated users’ privileges.

 • Availability: Can an attacker hurt the solution in any way? We apply common attacking methods on the server- based services by first monitoring open network traffic and then trying to either get to restricted functions or data or trying to halt the entire service.

 • Non-Repudiation: Does your app keep records of events? Where relevant, we can validate both client and server logs to make sure that it’s possible to use them to prove the user’s activities through the mobile solution.

The Bottom Line? Win At Mobile Application Testing With:

Mobile Device Selection: select devices with features that best fit your customer’s own usage

Emulators & Physical Devices: emulation helps with rapid development and automated testing

Mobile App Testing on Cloud: cloud-based mobile application testing reduces time to market (TTM)

Network Connectivity: test mobile applications under full, intermittent, and no network connectivity

Manual & Automated Testing: both are required to ensure its bug-free and usable

Performance Testing: required to check the load and stress verification of your app

Security Testing: check the app security and other vulnerabilities