Posted by
JORRIT WIT on Mon, Feb 20, 2012 @ 10:34 AM
Mobile Application Testing Tools 2
Keynote DeviceAnywhere vs. Perfecto Mobile Pricing Analysis and Summary

Written by Prateek Bhatnagar
Price being one of the key considerations for selecting a tool is further analysed. It will surely help you in selecting the right tool.
Price analysis has been divided into 3 cases (all calculations done using data from pricing given above):
1) Perfect for developers testing every now and then (Purchasing the exact hours)
Hours: 50
Devices: 100
Perfecto mobile: $25/hour -> 25 * 50= $1250
Keynote DeviceAnywhere: $16/hour + $100 access fees for devices-> (16*50) + 100= 900$
2) Perfect for companies with limited ongoing need for mobile testing.
Hours: 100/ month
Devices: Above 1500
Perfecto Mobile->
Monthly ($15/hour) -> 15 * 100= $1500
Quarterly ($13/hour) ->13* 300= $3900
Annually ($10/hour) -> 10 * 1200= $12000
Keynote DeviceAnywhere ->
Adding all packages = $250
Monthly ($16/hour) -> 250 + (16* 100) = $1850
Quarterly ($15/hour) -> 250*3 + (15*300) = $5250
Annually ($13/hour) -> 250*12 + (13* 1200) = $18600
3) Perfect for companies with growing ongoing need for mobile testing.
Hours: 100/user/ month
Devices: Above 1500
Users: 20
Perfecto Mobile (Single User) ->
Monthly ($0.96/hour/user) -> 0.96 * 100*20= $1920
Quarterly ($0.84/hour/user) ->0.84* 300*20= $5040
Annually ($0.67/hour/user) -> 0.67 * 1200*20= $16080
Keynote DeviceAnywhere (For any number of users) ->
Adding all packages = $250
Monthly ($16/hour) -> 250 + (16* 100) = $1850
Quarterly ($15/hour) -> 250*3 + (15*300) = $5250
Annually ($13/hour) -> 250*12 + (13* 1200) = $18600
For small scale testing where you need the tool for a known fixed number of hours Keynote Device anywhere is recommended.
Perfecto Mobile definitely has an edge over Keynote DeviceAnywhere when prices are compared for ongoing or larger needs. That said when we look at the customers page at http://www.keynotedeviceanywhere.com/customers.html we see many enterprise clients, who would be price conscious. Since we cannot deduce enterprise testing and the needs of the enterprise client is out of the scope of this paper, we can only suggest that these clients take a hard look at both options.
For the users of HP QTP the choice is easy since the Perfecto Mobile integration is seamless. However, every tool has its advantages and disadvantages and each company needs to select tool based on their requirements. Based on this analysis there are only minor differences between the two tools and both are a strong option.
For those that have made the choice or are doing the analysis please feel free to add comments and ideas.
Posted by
JORRIT WIT on Fri, Feb 17, 2012 @ 06:27 PM
Cross-Platform Mobile Development #4
Integrating innovative JavaScript web components in traditional web applications.
Post 4 of an ongoing series
Written by Pablo Elustondo
Our tasks was to place that component into the page and parameterize it appropriately for our tasks and then integrating it into the existing .NET webforms application by connecting it to the server via web services. The next figure tries to illustrate the integration.

The code to implement this “timeline” page is based, primarily on 3 files: a) an ASP.NET page that produces the html (timeline.aspx), the associated c# code (timeline.aspx.c) and a JavaScript file (timeline.js) with all the client side logic. In the ASP page, first we need to include all the JavaScript libraries and stylesheets corresponding to the Ext Js and Ext Gantt component. Then, to place the component into the page we just used an html placeholder. We did that with a code like this: <div id="GanttPlaceholder"></div>
That empty html div is just a placeholder where the Ext Gantt chart will appear after it is created. Our JavaScript code is responsible for initializing and configuring our Gantt chart and feeds it with the information about the timeline that it needs. This is a ‘model-view-controller’ implementation. So, one of the first things we need to create is a “store’ to keep the model information. For that, we use a code that looks like this: timelineModel = Ext.create('Gnt.data.TaskStore',...). This line creates the object to hold the “model” of our application which is a categorized set of tasks. All the information was included in the page during creation at the server side.
Then, we created the actual gantt panel with a sentence like ganttPanel = Ext.create('Gnt.panel.Gantt', { taskStore: taskStore, renderTo: 'GanttPlaceholder', ….}) . Here we are creating and configuring our Gantt chart view object and telling it which is the model that it must use and where is it that it should render itself. The rest of the configuration elements are meant to define how to set up colors, how to handle the menu options, print, etc.
Finally, we need to be able to send the information back to the server when he presses the save button after doing some changes. For that we use a JavaScript function to get the data out of the model and send it to the web service using the concepts described in the previous post in this the series.
Our JavaScript will pack that information into a string using a process known as “serialization” and send it to the server using an Ajax call provided by the underlying framework. That information will be un-coded (“de-serialized”) by the web service, converted into a model object in the server side and used to save it to the DB via the corresponding application layer. Making the Ajax from the client side is relatively standard and can be found in the documentation, but placing the web service on the server side may pose the question of where this should be placed in the code.
One possible solution, more suitable for the development of a complete SOA API, would be to create a complete web service interface for the web application independent on the existing pages. But in our case that would be an overkill and probably not too good from a modularity point of view, we just want to integrate with our existing code in a way that respects the existing architecture modularity. So, we used a very simple approach and placed the web service directly in the “code behind” of the timeline ASP page. The beauty of this from a maintenance point of view is that all this implementation is self-contained and not affecting or involving anything beyond this particular page. Our server side code in timeline.aspx.c file (normally called the ‘code behind’) looks like this:
[WebMethod(EnableSession = true)]
public static string saveTimelineService(string timelineString) {
…......
TimelineModel timeline = decodeTimelineString(timelineString);
saveTimelineModelInDB(timeline); };
The first sentence will decode the string (which could be json, xml or any other string representation) into a model object in the server and then it will save that model object into the database.
In our next post we will see how the same concept and almost same technology can be applied to produce a server based mobile app.
Posted by
JORRIT WIT on Tue, Feb 14, 2012 @ 03:20 PM
Cross-Platform Mobile Development #3
Integrating innovative JavaScript web components in traditional web applications.
Post 3 of an ongoing series
Written by Pablo Elustondo
In the previous post of this series (1 and 2) we presented a cross-platform architecture and methodology to develop user centric applications both for our browsers and mobile devices at a reasonable cost and speed. That was based on various principles including use of web services and modern and powerful JavaScript frameworks for the clients. Of course an application primarily or solely targeting iPad or iPhone would be better developed using Apple development platform and methodology, as we are discussing in our other series. The advantages of our cross-platform strategy becomes clearer when the application that we need to develop has to run on all browsers as well as at least two mobile platforms such as Apple and Android.
In this post we will show this strategy works for the browser. In future posts we will show the same for the Iphone, The framework and components that we describe here are a best-of-breed and we just finished to implement them in one of our current projects.
Our example is a simple page inside a bigger application made with .NET webforms technology aimed at managing projects. This particular page is intended to show a simplified Gantt chart that can be modified using drag and drop; as interactive as a desktop application. The next screen shot shows what the pages does. You can see a categorized and color coded set of tasks on a graphical timeline representation, you can drag and drop to change dates and you can also change the timeline resolution, zooming and collapse or expand a task. More functionality could be enabled but this page is meant to be simplistic on purpose.

This page was developed using the JavaScript framework Ext JS provided by Sencha and a specific component, called Ext Gantt, developed on top of that framework, provided by Bryntum. Both the framework and the component were developed following best practices in object oriented architecture, have very clear and well documented API and follow the model view controller paradigm.
Posted by
JORRIT WIT on Mon, Feb 13, 2012 @ 03:35 PM
Mobile Application Testing Tools
A Comparison Between The Leaders
KEYNOTE DEVICE ANYWHERE vs. PERFECTO MOBILE

Written by Prateek Bhatnagar
Mobile phones have heralded a new era named “The Mobile Age “. It has shifted the focus from computers to mobile phones for browsing, online transactions, listening music, watching videos etc. This shift has exponentially increased the development of mobile applications.
Types of mobile applications:
1. Data centric Applications (e.g. Sales Force Automation (SFA) , Customer Relationship Management (CRM), Enterprise applications)
2. Devises Capability based applications (e.g. Location Based Service(LBS), Camera , Multimedia, VOIP applications)
It has been observed that the better the mobile application is, the tougher is the task of developing and testing. The main challenge is to test the mobile applications on various platforms of mobile devices.
Major constraints for Mobile Automation Testing:
1. Increasing complexity of mobile applications
2. In depth functional testing
3. Testing device features used in applications i.e. camera , accelerometer
4. Bluetooth and Location Based Service (LBS)
5. Call and Voice mail scenario
6. Multimedia, Barcode , Near Field Communication (NFC) applications
7. Games and AI applications
Mobile automation testing can only be done for Data-centric applications.
To undertake mobile automation testing we must look at the tools that are specifically designed for mobile testing.
Mobile automation testing tools perform:
1. Application specific Test case writing
2. Conversion of test cases into scripts (record)
3. Execution of test scripts on devices
4. Unattended test cycles
There are various mobile automations testing tools in the industry but one of the first leaders is DEVICEANYWHERE. However the growth of this demand area has brought a fierce competitor in PERFECTO MOBILE.
In this article we intend to perform a comparison between these 2 leading mobile test automation tools: Keynote DeviceAnywhere and Perfecto Mobile. ( All data collected is obtained by direct interaction with both companies , demonstrations of their respective tools and performing intensive research online )
|
Factors
|

http://www.deviceanywhere.com/
|
http://www.perfectomobile.com/
|
|
|
They state: Keynote DeviceAnywhere provides an end-to-end platform for planning, testing, and monitoring of mobile apps to ensure the quality, performance
and uptime of your products and services as you extend your business to the mobile channel and
with its ongoing innovation and unmatched investment in product development and customer support. Keynote DeviceAnywhere continues to set the standard for Mobile App Lifecycle Management.
|
They state: Perfecto Mobile is a leading provider of remote access and automated testing solutions for mobile devices. It enable mobile operators, device manufacturers, application developers, testers and content providers to significantly increase the quality of their products and services while reducing time-to-market, costs and logistical efforts.
|
|
1.Image quality (for
taking screen shots,
videos )
|
High resolution device still image is pixel to pixel, exact and perfect.
For video the high frame rate depends on the internet connection.
|
Outstanding image quality no matter how high resolution the device is.
Again for video, frame rate depends on internet connection.
|
|
2.Coverage (countries)
|
Facility provided in USA, UK, France, Canada etc. and has the ability of giving access to a lot more countries.
|
Facility provided in USA, UK, Israel, Canada etc. and has the ability to provide access to a lot more countries.
|
|
3.History (Video frames
and Scripts)
|
Provides the facility of storing history of frames.
10 minutes history can be obtained.
History is deleted if system is closed for security purposes.
Also provides script history of all users.
|
It also provides the facility of storing history of video frames and scripts.
History is stored in a central server and can be retrieved anytime and it is private to each user.
|
|
4.Exporting History
|
History can be exported to .avi, .mpeg etc. formats and also to local files.
|
History can be exported to FLV formats or you can also extract and download it wherever you want.
|
|
5.Platform /OS Support
|
Supports every device ( Windows , Android,
Symbian, iphone, Blackberry etc.)
|
It also provides support for every device.
|
|
6.Ability to enter text
|
Provides ability to enter text in any field and in any device( device keyboard and the computer keyboard both can be used)
Also provides copy paste feature
|
Provides ability to enter text in any field in any device similar to device anywhere.
|
|
7.Download time
|
Time depends on internet speed and system
on which the application is being downloaded
|
Time depends again on internet speed and system on which application is being downloaded
|
|
8.Time required for the
application to open
|
2 to 5 seconds
|
2 to 5 seconds
|
|
9.Navigation of mouse
on Blackberry screen
(non touch)
|
2 types of keys are present.
1) Precision key - Normal mouse
2) Regular key – This one is for more easy navigation as it is provides easy movement of the mouse on screen
|
Normal movement can be performed using regular key but it allows for more easy navigation with the help of a click and drag feature of the mouse on the screen.
|
|
10.Pricing
|
The buyer must purchase a package plus an hourly commitment
1) Device packages: These are packages containing devices.
PACKAGES ACCESS FEE
First $100/Month
Second $75/Month
Third $50/Month
Fourth $25/Month
Fifth (and above) FREE
2) On the basis of hours:
HOURS HOURLY RATE $
120 16
300 15
600 14
1200 13
|
3 ways:
1) Pay Per Project:
10 hours - $250 ($25/hour)
2) Subscription:
Monthly:
Starting at $15/hour
Quarterly:
Starting at $13/hour
Annual:
Starting at $10/hour
3) Premium License:
Monthly: $690
Quarterly: $1,821
Annual: $5,796
|
|
11.Large selection of
devices
|
Have a large selection of devices
|
Have a large selection of devices
|
|
12.Connectivity
(Remote Location)
|
YES
|
YES
|
|
13.Smooth running of
device
|
YES
|
YES
|
|
14.Device interaction
|
YES
|
YES
|
|
15.Reporting
|
YES
|
YES
|
|
16.Scripting
|
Device agnostic (Keyword and Image based)
|
Device agnostic (Keyword and Image based)
|
|
17.Scripting (portability
across multiple
target platforms)
|
YES
|
YES
|
|
18.Flexibility – Devices
can be changed in
minutes + Request
new devices to be
connected
|
YES
|
YES
|
|
19. Additional Features
|
1) Application repository is present
2)FTP device log present
3)Freedom to build actions yourself or device
or device anywhere can populate it
(drag and drop to customize)
4)Checkpoints can be inserted
(Text, Image, Page)
5)It provides the facility of video compression
|
1)By far the best QTP integration
(Jointly developed by HP)
2)A new feature added to collect all
Vital information (e.g. Memory cache battery level , battery voltage , CPU usage)
3)Provides a secured private cloud
behind your firewall
4)Stronger image matching which
allow not only identical but also
similar
5)Provides log retrieval
|
Major source: http://www.slideshare.net/momobangalore/mdc2010-automated-mobile-testing
In the next blog we will compare pricing scenarios and make conclusions. Please feel free to add comments and ideas.
Posted by
JORRIT WIT on Thu, Feb 09, 2012 @ 01:55 PM
Objective-C Mobile Development
Best Practices based on pure Object Oriented Design
Post 1 of a Series
Written by Pablo Elustondo

The increasing importance of iOS apps for almost all business user-centric applications leads to the need of properly using Objective C and the XCode development environment. Even when some cross-platform mobile development tools like PhoneGap are starting to emerge, Objective C and Xcode are still the best way to develop high quality and high performance apps for iOS.
Unfortunately, Objective is not familiar to most programmers and the good practices can be confusing. It is a relatively old language based on C and ‘inspired’ by Smalltalk. At first glance it looks like a mix of two beautiful but radically different languages with a controversial conceptual integrity. Some old but existing practices like declaring internal variables in the interface are somehow inconsistent with the object oriented concept of encapsulation. Objective-C has evolved over the years and has various different options to implement almost similar things; some of them are rooted in C which should be avoided in the first place unless we really need to optimize performance. In areas like memory allocation, for example, various strategies as presented and some of them are already changing in iOS 5. Discussions around Objective-C in internet forums like StackOverflow are very popular.
Objective-C and Xcode are effective languages and a powerful development environment when used properly. In this series we will try to discuss the most simple and conceptual ways to look at Objective-C focusing on clarity before optimizing performance. We will look at Objective-C from the most purist object oriented view trying to capture the best practices and patterns while keeping it conceptual and overlooking the rough edges. Of course comments are very welcomed and may steer the direction of the blog series.
We will also introduce some of the latest advancement such as Objective-C automatic reference counting and Xcode UI storyboarding that make life easier. We want these posts to be useful to those who never used Objective-C before but understand programming and want to write new code; all concepts are introduced are assuming no previous knowledge of Objective-C.
To illustrate concepts we will use the well know object ‘bank account’. So, we want an iPhone app to access some basic aspects of our bank account remotely. Let assume all security and access aspect has been solved for simplicity.
First thing we will need is being able to manipulate and communicate with objects of type “BankAccount”. Objective-C is primarily, a statically typed language so we will write our variable with the statement BankAccount *bankAccount. We could also declare variables without a static type, but we would try to avoid that until is necessary. To tell our bank account that we need more cheques, we can say something like: [bankAccount orderCheques]. This is called in Objective-C “sending a message” (in other languages would be called a function call). If we want our bank account to deposit a cheque we have, we can write something like [bankAccount deposit: cheque];
Sometimes when we send a message to an object and we expect something in return. For example, the message “withdraw cash amount” would yield some form of cash (real or electronic). We could put that resulting money in the variable with the statement: Cash *cash = [bankAccount withdraw: amount];
The object can accept ‘messages’ and can also have ‘properties’. If we want to know the account balance, we can get his balance amount in the variable bankAccountBalance with a statement such as: float bankAccountBalance = bankAccount.balance. This is called the ‘dot’ notation, very popular in other OO languages but in Objective-C, this is only available on properties. We could also have written the same thing as float bankAccountBalance = [bankAccount balance] but for longer nested sequences the dot notation is more handy.
In Objective-C, as in other OO languages, classes are objects themselves and we can also send them messages. So, for example, one thing we can do with an object of class “BankAccount” is to create instances; create new account instances. If we want to create a new account instance, we can just do something like newBankAccount = [BankAccount new];
Much of object-oriented programming consists of writing the code for new objects and defining new classes. In Objective-C, following C style, classes are defined in two separated text files. A file with extension “.h” defines the interface and declares the methods, properties and super class. An implementation file with extension “.m” defines the class implementation. Most declarations in Objective-C start with the “@” symbol to differentiate this from the underlying C code. We use the following directives:
@interface to declare the class interface
@property to declare properties inside that interface
@class to import or include other classes that we have already defined.
To round up this post, this is how the interface for our BankAccount class will look like for now.
#import <Foundation/Foundation.h> //Includes necessary basic classes to make an app
@class Cash, Cheque; //We declare intention to use the classes we wrote
before
@interface BankAccount : NSObject //BankAccount just inherits from the generic Object
@property float balance;
- (void) orderCheques;
- (Cash *) withdraw: (float) amount;
- (void) depositCheque:(Cheque *) cheque;
@end
If we just want to use this object in our program, and somebody else is actually implementing it, this is the only thing we need to know. Of course, some documentation will help. But this is what we need from a compiler point of view. In the next post we go deeper showing how to implement and use this ATM sample app.
Posted by
JORRIT WIT on Tue, Feb 07, 2012 @ 08:26 AM
Defining a Security Test Strategy
Software Security Engineering Blog 4
Written by Maheshwar Kanitkar and Hemant Belorkar

Factors to define a security test strategy.
- Scope of security testing.
- Identify risks.
- Prioritization on risks.
- Regulatory Compliance.
- Define threat model to be used (can be based on Operating system specific security threat model, OSSTMM).
- Training requirements.
- Testing during Sustenance.
- Available tools, solutions, cost, time.
Security Testing Strategy Guidelines
Wide spectrums of applications following broad patterns pose many challenges for functional security testing. Largely due to lack of time, inadequate reporting and/or co-ordination deficiencies, testers find it difficult to complete their tasks. Integration of functional security testing into the testing phase of the SDLC is an important part of the solution. Additionally, spotlighting the following can help ease the process for testers:
- Functional pattern identification: Sometimes tedious, the pattern identification process is a one-time process of identifying the functional pattern of the application's behaviour for the input. These patterns could be consistent across the functional areas of the application, but testers should not progress forward with this assumption.
- Test case definition: Role matrix, data flows and the technologies used in the functional area help define the test cases. A good approach is to prioritize the test cases based on the impact by running them by the functional analysts and architects. Mapping of impact analysis to the security facet prioritization can help greatly.
- Parameters definition: Various tool kits are available that point out the parameters and their variations for each test case. Output behaviour testing and analysis leads to additional test cases and cross-functional test cases.
- Reporting results: Testing simplification, possible automation in reproducing the security flaws and impact analysis reporting empower management in reviewing and prioritizing remediation strategies.
Application security has been an uphill battle at many organizations, but this year's report on internal threats is a wakeup call that cannot be ignored. With a considerable number of the internal threats originating from applications, functional security testing is one of the most reliable ways to identify internal security vulnerabilities. Vulnerability assessments (VA) and penetration testing (PT) should be performed to determine the risk and attest to the strength of the software after it has been deployed. Although vulnerability assessments and penetration testing are used synonymously by many, they are not the same. Vulnerability assessment is a process of identifying known weaknesses of software. Penetration testing on the other hand is testing the security of the software, simulating a malicious attacker. A part of vulnerability assessment can be penetration testing.
Identification & Prioritization on Risks
- Assign severity levels for Availability, Integrity and Confidentiality to the information assets managed by the applications.
- Account for all the data, technologies (this includes any third-party toolkits/APIs) and user types defined in the application .
- Define a role matrix if not already developed for the data access .
- Identify the right information and technology owners.
- Organize a comfortable platform for the information owners to qualitatively or quantitatively define and assign priorities to the identified information and technologies.
Regulatory compliance
With the growing regulatory pressures and penalties for non-compliance being very huge, enterprises are finding it difficult to implement latest technologies that can pose a challenge for enterprise security and compliance. Most enterprises do not exactly understand all the regulations and worse is their perception that all the required controls are in place. It requires experienced professionals who thoroughly understand regulations and the risks involved in it, to ensure compliance within an enterprise. Quality Assurance and Testing from independent testing vendors will help organizations to address general regulatory as well as vertical-specific regulatory requirements. Apart from specialized testing services, compliance testing from independent testing specialists verifies whether a system is in compliance with required regulatory requirements and ensures that all applications geared around compliance do what they are supposed to do. On the whole, specialized testing experts with thorough industry knowledge will be able to help enterprises address all the country-specific and country-neutral compliance and regulatory requirements.
Posted by
JORRIT WIT on Thu, Feb 02, 2012 @ 11:44 AM
Selecting a Mobile Test Automation Tool Part 2
A Focus on Mobile Application Testing for the Web
Written by Paul Wu
Internet has made the world smaller, smartphones shortened it even more.
“There are almost twice as many mobile phones in the world than computers. Claimed by DeviceAtlas, the World’s leading device detection platform, there are as many as over 4,500 devices worldwide [3]”.
With that number in mind, you can understand it’s a huge market with users increasing daily. On the other hand, the challenges are, how are we supposed to test the website across all devices? What is the benefit of testing on simulators? What is the cost for mobile automating? What is easy-to-use method for testing automation? We will address the following questions for the following Smartphone Operating Systems:
Major Smartphone Operating System Review:
|
Operating System
|
Android
|
IOS
|
Bada
|
Windows Phone
|
Symbian
|
Blackberry
|
|
Manufacturer
|
Google
|
Apple
|
Samsung
|
Microsoft
|
Nokia
|
RIM
|
|
Browser
|
Chrome/Web Kit
|
Safari
|
Bada Browser
|
Default IE
|
Mozilla
|
Web Kit
|
Table1. Multiple platforms in mobile world
Functional Testing Automation Tools:
|
Features
|
Device Anywhere
|
Eggplant
|
◊ExperitTest
|
Jamo Solutions
|
Perfecto-Mobile
|
Robotium
|
Squish
|
Test Quest
|
|
Support of Platforms
|
Android, Blackberry
Brew, Symbian, iPhone, iPad
|
Android
iPhone
Symbian
Blackberry
Windows Mobile
|
iPhone, Android,
Blackberry
Symbian,
Windows Mobile
|
Android, Windows CE, Windows Mobile
|
Android, Blackberry
Brew, Symbian, iPhone
|
Android
|
GUI testing tool for all phones
|
Android, Blackberry
Brew, Symbian, Window Phone
|
|
System Requirement
|
Low
|
Medium
|
Low
|
Low
|
Low
|
Low
|
Medium
|
Medium
|
|
♦Cost
|
Cloud-Based
|
$6,999 $US for multiple clients
|
$2,499 US per
Year
|
Starting at $14,500 US per license
|
Cloud-Based
(e.g. $10 / hour)
|
Open Source
|
€2,499 ($3286 US) per license
|
Starting at $15,000 US
|
|
Test management Integration (QC, QAComplete, TestLink, etc.)
|
Yes
|
N/A
|
Yes
|
Yes
|
Yes
|
N/A
|
Yes
|
Yes
|
|
Scripting
Language
|
N/A
|
Eggplant Scripts
|
C#,
Junit,
Perl, Python
|
Vbscript
|
Keyword-Based ScriptOnce Scripts
|
Java
|
Javascript, Perl, Python, TCL
|
C#,
VB 2010, IronPython, IronRuby
|
|
Remote Control (VNC, rlogin, iSHH)
|
Yes
|
Yes
|
Yes
|
Yes
|
Yes
|
No
|
Yes
|
Yes
|
|
*Usability
|
2
|
3
|
3
|
3
|
2
|
3
|
3
|
3
|
|
Suggestions
|
Small Firm- Enterprise
|
Small
Firm - Enterprise
|
Small
Firm - Enterprise
|
Small
Firm - Enterprise
|
Small Firm - Enterprise
|
Small Firm
|
Small
Firm –
Enterprise
|
Small
Firm - Enterprise
|
Table 2. Automation tools and corresponding features
♦All the cost mentioned here is not recommended pricing or license. They are negotiable based on your challenges and requirements in your mobile testing. Also the cost will vary depending on the type of the licences. E.g. some licenses will allow you to develop any number of tests by any person in the development team and test all devices; whereas some licenses are charged by the quantity of implemented OS or Platforms.
*Usability is ranged from 1-5, 1 is the easiest
◊Experitest has no intension to replace current automation tool such as QTP, so this tool performs as a plug-in. In other words, test scripts you write in SeeTest can run in your regular testing environments such as JUnit, C#, Python, Perl, TestComplete-VB Script and QTP-VBScript. In order to do this, you need to export the code of the test you have written in SeeTest AND integrate to your regular testing environment.
Depending on your budget and requirement, choosing standalone IDE tools, such as Eggplant, and Squish will be a good option for those companies have no access to all the of necessary devices and will help you to resolve issues. Some strategies on how to select a small and common number of devices that will enable to identify issues across the mostly used mobile devices are introduced here.
With simulators in use for mobile testing, we will have quick access to a large number of devices and browsers. Most of the simulators are free, and manufacturers want you to develop on it. Mobiforge’s blog A Guide to Mobile Emulators addresses a lot of information about types of emulators you want to use and how to install and run on your system. Of course, everything has 2 sides, positive or negative. Some of the disadvantages are simulators have different heap and stack size from real handsets; and also, instead of low power ARM Processor used for the most of mobile device, the simulators are using the PC’s CPU, So there may be some problems for performance testing if using simulators.
For cloud-based solutions, for instance PerfectoMobile, and DeviceAnyWhere, are fast-growing and well publicized companies in recent years. They provide the most recent and frequently used web-based real handsets through online simulation, so you can easily test on any updated platforms promptly. One of the most exciting things for testers is that, with PerfectoMobile’ add-on (Figure 1), as well as Experitest’s plug-in techniques (Figure 2), HP QTP has extended its functional testing to MobileCloud. It’s especially recommended for the company who already has HP QTP 11 in use.
Figure1. PerfectoMobile Testing Extension for HP QTP [2]

Figure2. illustration for Experitest’s QTP plug-in.

In the next post we will compare some of these options as well as the list of available performance testing tools.

Posted by
JORRIT WIT on Thu, Feb 02, 2012 @ 08:42 AM
High Level Threat Model
Software Security Engineering Blog III
Written by Maheshwar Kanitkar and Hemant Belorkar

A threat model is essentially a structured representation of all the information that affects the security of an application. In essence, it is a view of the application and its environment with a security mindset.
Threat modeling is a process for capturing, organizing, and analyzing all of this information. Threat modeling enables informed decision-making about application security risk. In addition to producing a model, typical threat modeling efforts also produce a prioritized list of security improvements to the concept, requirements, design, or implementation.
Threat Modeling - Generic Steps
- Define the application requirements.
- Security-based risk assessments to identify areas of greatest risk to the business and the technology platform were completed.
- Identify user roles that will interact with the application.
- Identify the data the application will manipulate.
- Identify the use cases for operating on that data that the application will facilitate.
- Findings from the risk assessments were addressed in the security architecture and implementation.
- Model the components of the application.
- Model the service roles that the components will act under.
- Model any external dependencies.
- Model the calls from roles, to components and eventually to the data store for each use case
- Identify any threats to the confidentiality, availability and integrity of the data and the application based on the data access control matrix that your application should be enforcing.
- Assign risk values and determine the risk responses.
- Determine the counter measures to implement based on your chosen risk responses.
- Continually update the threat model based on the emerging security landscape.
One can build a threat model using STRIDE, an acronym for Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service, and Elevation of Privilege.
Benefits
Done right, threat modeling provides a clear "line of sight" across a project that justifies security efforts. The threat model allows security decisions to be made rationally, with all the information on the table. The alternative is to make knee-jerk security decisions with no support. The threat modeling process naturally produces an assurance argument that can be used to explain and defend the security of an application. An assurance argument starts with a few high level claims, and justifies them with either sub claims or evidence.
Posted by
JORRIT WIT on Sat, Jan 28, 2012 @ 08:15 AM
Memory Leak Testing
It can crash your application, can you afford to ignore it?
Written by Sunil Damle

Memory leak testing is an important aspect of a software testing process but often it is ignored during the various testing phases. A memory leak issue is difficult to debug or fix if identified in production.
Memory leak - Why it happens?
An ideal application program releases memory after use but if it cannot, it will result into a memory leak. Such a dynamically allocated memory could be not used by any other program or system and may result into issues like slowness in system performance or as critical as an application crash, with only remedy of rebooting the system. This is much critical issue in case of automotive applications or a business applications where it can not be easily restarted. So, it is important to plan and make sure that the application is free from such leakages.
How can Memory leaks be handled?
To detect a memory leak issue is very challenging. As such a defect is difficult to reproduce and isolate, it should be handled appropriately at different phases in the software development life cycle (SDLC). There can be different methodologies specific to an application.
Here is a suggested four step approach from QA's perspective:
1. Plan – A plan to minimize memory leak risks considering different factors
There are various factors to be considered for this issue but not limited to:
- Application Technology
- System Environment
- Programming practice
- Development approach
- Performance Testing
2. Design –Design test scripts and unit tests to detect memory issues, follow good design practices
3. Detect – Test for the presence of a memory leak in the system with functional and non-functional testing
4. Fix – Once detected, debug, isolate the issue and then fix the Memory leak. Once a Memory leak issue is fixed, the procedure to repeat steps 2 to 4
Memory Leak Detection Tools:
There are different tools available in market that can detect memory leaks mainly during unit testing. Examples: IBM Rational Purify, Compuware BoundsChecker; open Source tools like Valgrind.
Further, QA activities like static code analysis, following coding guidelines and thorough code reviews would help to minimize such issues.
Memory leak issues are often difficult to detect with just with functional testing, however specific performance testing helps to detect such issues.
Reference: http://msdn.microsoft.com/en-us/library/ms859415.aspx
Posted by
JORRIT WIT on Tue, Jan 24, 2012 @ 02:00 PM
Cross-Platform Mobile Development #2
Plain web services for Web and Mobile Application
Post 2 of an ongoing series
Written by Pablo Elustondo
In this post we will elaborate on the first principle from our previous post, ‘structuring the server side application around a clear API made with web services’. Here, we introduce web services for a Web and Mobile Application in an extremely simple, conceptual and practical manner using a realistic example. Some concepts in software development, like object orientation or web services, are sometimes explained in a complicated manner with unnecessary jargon and details. In this post, we go back to basics, focusing on conceptual integrity, best practices and simplicity.
Web services are a way to architect the communication between two applications that are connected via the internet/intranet. In Web and Mobile Application Development this two applications are normally classified as ‘the server (or service)” and ‘the client (or app)’. Both servers or clients normally run on a particular application containers: servers run on application servers like Tomcat or ASP.NET and clients run on top of things like browsers, mobile operating systems or PC operating systems. In any case, the concept is very similar, even a browser can be seen as an ‘operating system’ of some sort. The only thing that may change is the programming language, programming interfaces and capabilities that are available on different platforms. Understanding all those clients under the same conceptual umbrella allows us to architect cross platform applications. The next figure illustrates this:

When designing web services as described before, we are assuming that we are sending and receiving textual information, using a standard protocol like HTTP. Of course you can encrypt the information, but from a conceptual point is better to think of textual information that can be read by humans. Although the information that is exchanged can be arbitrary, we suggest here a best practice to structure this information. This practice is based on best practices for structuring applications using ‘model’ entities and ‘view’ entities. A ‘model’ entity is a way to represent a particular type of information, and a ‘view’ entity is a mechanism to show and interact with that information represented by the ‘model’ entity. For example, our ‘model’ entity can be constituted by an address, a street name, a city, etc and our ‘view’ entity can be Google Maps showing that address in the map. Both the client and the server should use well defined model entities to represent the information that is being manipulated. Using this practice (also called a pattern), the web service is used as a mechanism to send representations of these entities back and forth and to implement a particular operation that should be applied to them.
For example, the client may ask the server for an address, the server will send the address back to the client and then the client will show this address in the map combining that information with other information from other sources. These entities at the client or server side do not need to be identical, both servers and clients will represent entities in a way that is convenient for their role in the application. Clients and servers send and receive a version of those entities that make sense to both according to an established protocol. In other words, servers and clients exchange a ‘representation’ of the entities. The next figure illustrates this concept

Note: This idea of sending and receiving ‘representation of entities’ using a predefined protocol over HTTP is related with what is referred as REST (Representation State Transfer) protocol.