Reliability Testing of Mobile Applications

Reliability Testing of Mobile Applications

Challenges in Mobile Application Testing

Part 4 of 4

Written by Sarita Huddedar

Reliability testing deals with checking the ability of the software/application to function under given environmental conditions for a particular amount of time, by taking into account all workings of the software. In this type of testing, the problems are discovered regarding the software design and functionality and the assurance is given that the system meets all requirements. Software Reliability is the probability that software/application will work properly in a specified environment and for a given time.

Probability = Number of failure cases / Total number of cases under consideration

Software Reliability is measured in terms of Mean Time Between Failure (MTBF).

MTBF consisting of mean time to failure (MTTF) and mean time to repair (MTTR).  MTTF means difference of time in two consecutive failures and MTTR is the time required to fix the failure. Reliability for good software should be always between 0 to 1.Reliability increases when the errors or bugs from the programs are removed.

In case of mobile applications that involve internet connections, reliability testing is measured in terms of:

1. Network carriers: Applications that involve internet connectivity has to depend on the network carriers.
2. Data transfer: Which also involve dependency on network carrier speed, strength, the continued connectivity etc.
3. Security: Secure data transfer (sensitive data in encrypted format) and free of viruses/malicious software adversely affecting the application.

Along with the above mentioned points, mobile applications should be tested for specific reliability issues specified below:

 

1. Reliability in terms of Functionality: Testing for fragile areas of the application, maintaining application functionality consistency.
2. Reliability in terms of Compatibility : Testing of application on various configurations, devices, browsers
3. Reliability in terms of Performance: Testing the application for maximum user load etc.

 

The application should not hamper any device's data/functionality after installation.

 

References:

http://palpapers.plynt.com/issues/2009Apr/mobile-application-security-testing/

http://www.techrepublic.com/whitepapers/mobile-application-security-testing/1294429/post?tag=mantle_skin;content

http://en.wikipedia.org/wiki/Software_Reliability_Testing

 

Boundary Value Analysis

Introduction to Boundary Value Analysis

Written by Prashant Pujari 

 

One of our test engineers was creating test cases to verify a simple business rule in an income tax application.  The business rule said “The age eligibility criterion for a person to get employed and pay the income tax on the earned income is 18 years to 65 years (18 and 65 inclusive)”

 

Looking at this business rule, any test engineer would choose “Boundary Value Analysis” test design technique; commonly referred to as “BVA”, to arrive at various test scenarios or test cases. Boundary Value Analysis test design technique in its simplest form says –

“Generally, it is observed that defects tend to cluster around the boundary values. Hence boundary values include minimum, maximum, just inside, just outside values.”

Here the assumption is, if the system works “as expected” for these representative values, system will behave “as expected” for all values within the selected range. This assumption roots from the fact that Boundary Value Analysis falls under a Black Box test design technique. Here test engineers typically do not get to see the implementation details or how the code is written.

This tells us that for a valid boundary range, tests will be designed for total six values: three values to test lower boundary and three values to test higher boundary.

Going by this definition, our test engineer designed six test cases which must be tested.

  

The Problem with Boundary Value Analysis

Are these six values sufficient to catch the defect around boundary conditions within the application?

The answer is – Six values MAY NOT be sufficient to catch the probable defect.

Why?

Rene Tuinhout has discussed the reasons in detail in his article “The Boundary Value Fallacy” (http://www.testingexperience.com/testingexperience03_08.pdf).

For simplicity, let’s just go through the thought process of the test designer. Remember - Boundary Value Analysis falls under the Black Box test design technique and test engineers typically do not get to see the implementation details or how the code is written.

As most of the test engineers understand, it is observed that developers are prone to making errors in the boundary conditions and hence defects tend to cluster around the boundary values.

Conventional Boundary Value Analysis

Here is how a developer may implement a business rule “The age eligibility criterion for a person to get employed and pay the income tax on the earned income is 18 years to 65 years.”

To detect any possible programming fault made in the relational operator (<, <=, =, >, >=, <>), our test engineer has come up with six values that must be tested – 17, 18, 19 for lower boundary and 64, 65, 66 for upper boundary. For further simplicity let’s just consider the lower boundary for our discussions. With a little thinking, we may notice that developer may implement this code as:

IF iAge >= 18

Eligible

ELSE

Not eligible

ENDIF

What could possibly go wrong here? Just imagine if developer implements the code (Defects! and that’s what we want to catch… right?) as shown in table below by using different relational operators.

From the table above it is evident that 3 values are sufficient to detect any implementation defect related to relational operators. So far the claim made by Boundary Value Analysis test design technique holds good.

But Hold On!!!

Did we just assume that code might be written as

IF iAge >= 18

Eligible

ELSE

Not eligible

ENDIF

Think again! Is it possible that developer may write something like this?

IF iAge > 17

Eligible

ELSE

Not eligible

ENDIF

Looks like a valid implementation…

Let’s again walk through our earlier relational operator table and see if 3 values are still enough to catch the probable defect.

 

From the table above we just discovered that if the code gets implemented as iAge <> 17 (Wrong implementation), instead of iAge > 17 (might be a valid implementation), testing with 3 lower boundary values is just NOT enough! The three lower boundary values will not detect such implementation defect. 

Is there a solution for this problem?

Yes. There is a solution….  Enhanced boundary value analysis, which will be covered in Part 2 of this blog.

Security Testing of Mobile Applications

Security Testing of Mobile Applications  

 

Part 3 of 4 in a series on Challenges in Mobile Application Testing

 

 

Written by Sarita Huddedar

 

Development of mobile applications has accelerated at a tremendous rate recently. Mobile applications are used to store personal data, perform banking transactions, make travel arrangement, enhance social media and much more. Mobile applications should also be tested from security perspective.  Compared to web applications mobile applications are more complicated to test due to the factors mentioned previously in this series. In general, mobile applications are not as secure, compared with web applications.

Mobile applications should be tested for security in the following ways:

1. Installable mobile applications

Check for modifications in registry entries, configuration settings and the creation of new files/folders in the file system, after installing the application. This should be compared it with the structure before installing the application.  Application reversing and analyzing the components helps in penetration testing of the application.

2. Browser based mobile applications

Setting up web proxies for intercepting application traffic can help viewing application behavior and to modify the data for validation and authorization.  By intercepting client server data flow, one has complete control over application data flow and can thus perform a thorough test.

3. Certificates of Trusted authorities:

Most of the mobile applications use HTTPS. They rely on the device's certificate store to determine trusted certificate authorities. This connection is not established if the certificate is not a trusted one.

4. Application Permissions/User Permissions: For all the points mentioned below, user permissions are required.:

PIM: Personal Information Management: This includes address book, calendar events, to do, file system, key injections.

Protected APIs:  It includes Camera functions, Location data (GPS), Bluetooth functions, Phone functions, SMS/MMS functions, Network/data connections.

Cost-Sensitive APIs:  A cost sensitive API is any function that might generate a cost for the user or the network. The user will have to grant explicit permission to third-party applications requesting use of cost sensitive APIs. These APIs include Telephony, SMS/MMS, Network/Data, In-App Billing, and NFC Access.

In addition to this, if an application wants to know the user's location, the application requires a permission to access the user's location. Upon installation, the installer will prompt the user asking if the application can access the user's location. At any time, if the user does not want any application to access their location, then the user can disable location based services for all applications on the user's device.

 

For testing the Security of Mobile Applications, one needs to employ following methods:

Static analysis includes analysis of Source code, Source code scanning, Manual source code review, Binary Reverse engineering etc.

Dynamic Analysis includes Debugger execution, Traffic capture via proxy, Forensic Analysis, File permission analysis, File content analysis etc.

Testing the security of mobile applications is not only testing the application but also the third party web services, enterprise services as the weaknesses and vulnerabilities we find in mobile applications come mostly from interactions with supporting services.

In addition to above mentioned points, security testing involves:

Authentication and Authorization: Both these techniques should be used for restricted user access to the application.  Moreover, roles and rights management should be implemented within the different areas of the application.

Input Validation: Wherever user input comes into picture, validations must be done. All the user inputs should be verified and filtered based on the expected type. Restricted inputs should be used wherever user input is expected. The rewards of good input validation are resilience to dangerous attacks and a high level of information assurance.

Session Management: Maintaining sessions for users for specified time period helps in security of a particular user data. Requirement for timeouts of user logons must be in place for sessions. Identifying the maximum age of any given session ID as well as a timeout for sessions is essential. There is often the requirement to re-authenticate users during a session.  For example a net banking application would re-authenticate the user prior to transferring funds. This second authentication should also prompt the creation of a second session ID and the destruction of the original ID.

Encryption: All the sensitive data must be encrypted to make it secure. Encryption should be strong especially for sensitive data like passwords of user accounts, credit card numbers or other business critical information. Proper security measures must be adopted when flow of sensitive data or business critical data occurs. Whether this data floats between different modules of same application, or is transmitted to different applications it must be encrypted to make it safe.

SQL injection checks

SQL injection is an approach in which malicious script is used by the hackers in order to manipulate the application.  To check this, there should be some restriction to the input field. Also in such fields any html tags or script tag input must be prohibited. Also the application should not support anonymous access methods.

The last part of this series will deal with Reliability Testing of Mobile Applications.  

Fishbone Diagram for Employee Lateness

Fishbone Diagram for Employee Lateness

Employees not able to report to work place on time

 

Part 3 of a series of 3 -  Case Studies, Analysis and Fishbone Diagrams

Written by Girish Chincholikar

Steps to be followed to arrive to a detailed Fishbone diagram

a. Identify the Categories

b. Identify the effects

c. Identify the detailed effects

 

Following categories were identified

1. People - This was identified as People are involved
2. Logistics – This was identified as Logistics issues could be there
3. 3.       Infrastructure – This was identified as Infrastructure issues could be there
4. Process – This was identified as Process issues could be there

 

Following effects were identified for PEOPLE

1. Attitude issues
2. Lack of Professionalism
3. Less Committed
4. Improper planning

Following effects were identified for LOGISTICS

1. Logistics unavailability in certain areas
2. Poor local transport
3. Very expensive
4. Personal vehicle not available

Following effects were identified for INFRASTRUCTURE

1. Poor Infrastructure
2. No Infrastructure

Following effects were identified for PROCESS

1. Non adherence to process
2. No defined process
3. Loop holes in Process

 

 Translating to a Fishbone Diagram

References made: http://www.mycoted.com/Fishbone_Diagram

Outsourced Product Development - 5 Important Tips for success

Outsourced Product Development - 5 Important Tips for Success

By Anand Vidyasagar

In the last 13 years,  I have worked with many North American customers and outsourced some of my own products for development.   In today’s cost competitive world we cannot afford to have exclusively permanent staff on board throughout the lifecycle of a product.  It is best to with several types of resources right from the drawing board to launch a product into production. In a space where time is the variable, it is quite common to lose the business focus and get caught into local resourcing and staffing issues.   An alternative is to think globally and choose the right person for the right job. This can affect the end result and blur the original vision.

With this in mind here are 5 Important Tips for successful Outsourced Product Development

1. Build Integrated teams:

It is quite common to have your product management, development, and services teams working with your vendors.   At times you may find your teams (or yourself) stuck with a decision whether to share some information with your vendors. You should try to make conscious efforts towards becoming more transparent with your vendor so your extended teams can have the same sense of business urgency as you. (Just make sure you have a strong NDA in place when considering IT Outsourcing).

It always helps your community of Business Analysts, developers, service organization, and vendors to have the same information so that they can be more effective in delivering a solution that caters to your end objectives. They can make informed choices as your ideas start taking shape.

 2. Communication:

Invest in tools which bridge the gap between the teams.  When your teams could be spread across the globe it is quite critical that you have a good communication and collaboration tools in place.  Whether your company is outsourcing or not, communication is the key to success of any initiative.  This applies to whether it is a software development project or the Government making a decision on behalf of the entire nation. Communication failures can be disastrous for the entire initiative.

Here is one experience to understand how badly a communication failure can influence a project.

We had a customer that had two pieces of an application being built in two different geographies. They tried to isolate the teams and make sure they were co-located to help them with better/faster development. It turned out to be very challenging as they worked to  integrate the solution. Along the way it was just assumed that the teams were collaborating enough to ensure success.

It was short project but when we reviewed it with the client we discovered a disconnect and found for the next project we needed to monitor that existing systems were more closely followed.    

We also set up an internal wiki site for the developers to help with collaboration. The wiki site was continuously monitored to make sure no one was going off course.

Always remember when you are working with teams around the globe, they are not going to read your mind. You have to state the obvious to keep things crystal clear. DO NOT leave anything for speculation or assumption. Make sure your internal meetings are captured well and posted on a wiki site for people to read and understand the rationale behind it.

 3.       Pair the Partners

It is quite common these days in a Global Delivery Model that your project teams will be spread in different parts of the globe. When you are dealing with such situations it is quite a common temptation to create teams that are co-located.

You are better off if you create a team in which some people sit in one part of the world while the other sits in a different part of the globe. With this approach, if you have a good communication and a collaboration model setup, you would be able to derive more value and time reductions.

People working in the same time zone may have afternoon exhaustion where they are likely to push a problem to the next day. With this “Pairing” model, you will reap the benefits of having the second partner take it from where the first partner left off.  Some organizations have an “Onsite coordinator” model where one person acts as a liason between two teams. This point does not exclude this important role but adds another line of communication to give each team a face.

Every large software application is built up of smaller pieces which fit into the bigger entity. Educate your Program Managers/Product Managers/Architects to keep things simple, let the building blocks be small enough to be developed by NOT more than 3 people. Once you figure out the work assignment try to distribute the team across the globe. You will get the added advantage of the work being done round the clock. It is quite important for each of the paired partners to hand-off their work at the end of their day and let the other partner take it forward.

 4.       Think Global: Find the right people for the Job

It is common to hear that a job cannot be done from a different locations.  Here is a scenario from our past experience.  We had a fantastic network guy in our implementation team who was also the in charge of installations of the software at a customer location. He was in the field, when he became stuck with one piece of an application server installation. He got very frustrated, trying to install the application server, going through the installation manual, time and again.

We were back at another location, trying to figure out what could have gone wrong. Finally comparing the issue across our global team we found there was a person in India who had experienced these issues. The problem was he was half way across the globe so the issues were:

1. How to get him to see the problem first hand.
2. How to get access for him to solve the problem.

Our networking guy proposed a simple yet powerful solution. While he was at the site he made sure the machine was the network and he used a combination of WebEx and Net Meeting tools to allow access for our India based Expert to get onto the box and take control.

This problem was resolved in a matter of couple hours. The mantra here is “Think Global”.  You or your partners, may have people on the other side of the world, who may have the perfect solution to your problem.

 5.       Celebrate Success together

Every milestone past, every achievement has taken a lot of effort and your entire global team was part of it. Make sure you celebrate the success with your own employees as well as your outsourcing partners. DO NOT leave anything to assumption that the management at the vendor will do it for you.   Besides it really boost morale that the client directly commends a team.

Celebrating successes with your IT Outsourcing vendors will ensure a greater bonding and sense of being a part of your success. This will help build the momentum, so your vendor will be there for you when you need them the most.

Orthogonal Array Testing Tools

Orthogonal Array Testing 2

Orthogonal Array Testing Tools

Part 2 of 2 

Written by Prashant Pujari

 

If we consider the volume of test cases that are required to exhaustively test all possible combinations, manually generating such a volume of test cases is not possible. Orthogonal Array Testing Tools help in generating test cases; eliminating the duplicate sets of test scenarios (based on pairs of variables), thereby providing test designer a valid subset of test cases.

 

However it is important to note that the tools statistically arrive and suggest scenarios to be tested. Based on business rules and application logic, actual scenarios which one can test with, might be much less than what tool provides.  The test designer should share the test cases generated from Orthogonal Array Testing Tools with business analysts and confirm the validity of test cases.

 

Most of the tools help to generate test cases automatically based on the Orthogonal Array Testing technique. One can choose best suited tool which satisfies the requirements and serves purpose.

The detailed list of available tools is available at http://www.pairwise.org/tools.asp

 Case study

Time range	Counters	Values
Past day	Counter 1	Checked
Past week	Counter 2	Unchecked
Past month	Counter 3
Past year	Counter 4
Custom	Counter 5
Real time	Counter 6
	Counter 7
	Counter 8
	Counter 9
	Counter 10
	Counter 11
	Counter 12
	Counter 13
	Counter 14
	Counter 15
	Counter 16
	Counter 17
	Counter 18
	Counter 19

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

In one of the projects we were supposed to test combination of 3 variables – “Time range”, “Counters” and “Status flag”. Factors = 3

“Time range” variable had 6 possible values. Levels = 6

There were 19 possible values for “Counters” variable. Levels = 19

And “Status flag” had 2 possible values. Levels = 2

In order to test all possible combinations, we would have required 6 x 19 x 2 = 228 test cases.

We used free utility “AllPairs” developed by James Bach, james@satisfice.com <mailto:james@satisfice.com>, www.satisfice.com to generate test cases based on Orthogonal Array testing technique.

The utility produced 114 uniform combinations - ^[1] Test caseswhere [“Counters” and “Time range”] or [“Counters” and “Values”] or [“Time range” and “Values”] pairs are getting covered at least once.

The utility also generated ^[2] Pairing detailsshowing map of pairs getting covered in one or more test cases

 

^[1] Test cases

~ Symbol in table below represents “Don’t care” values. Such values are already covered in one or the other pairs

 

 

^[2] Pairing details

 

Orthogonal Array Testing

 

Introduction

In its simplest form Orthogonal Array testing is a Statistical way of testing “Pairwise” interactions. Invented by C. R. Rao (Calyampudi Radhakrishna Rao), this technique provides uniformly distributed coverage of all variable pair combinations. This technique falls under Black Box testing techniques. Many times it is also referred as “Pairwise” testing or “Taguchi methods” – since Dr. Genichi Taguchi was one of the first proponents of orthogonal arrays in test design.

 

Test	A	B	C
1	0	0	0
2	0	0	1
3	0	1	0
4	0	1	1
5	1	0	0
6	1	0	1
7	1	1	0
8	1	1	1

Consider simple example where there are 3 variables – A, B and C. Let us assume that each of these variables can have value 0 or 1.

 

What would be the number of tests we will need to test all possible combinations?

In order to test all possible combinations of A, B and C we will need

2 x 2 x 2 = 8 tests

 

The number of variables is termed as “Factors” and the maximum number of values each variable can hold is termed as “Levels”. In this example we have Factors = 3 and Levels = 2

 

Now if we expand the same concept to a scenario where we have to test 5 variables with each variable having values ranging from 1 to 5, our number of exhaustive tests would be

5 x 5 x 5 x 5 x 5 = 3125 tests (Factors = 5, Levels = 5)

 

There must be some way to reduce the number of tests into something that we can handle without compromising on the quality.

 

Orthogonal Array Testing

When there is a need to create efficient test cases in a short time, test multiple combinations of variables and to get maximum coverage where exhaustive testing is not possible, the Orthogonal Array Testing technique can be used. This technique makes sure that for a variable pair, unique combination of possible values gets tested at least once.

 

Orthogonal Array Testing technique is used in the test design phase which helps test designer to come up with possible scenarios or test cases. Orthogonal Array testing does not guarantee 100% extensive test coverage. Still it is the best statistical and scientific technique to reduce number of test cases than using random test case selection.

 

The fundamental reasons for recommending usage of Orthogonal Array testing are:

• The simplest bugs in a software program are generally triggered by a single input parameter. The next simplest category of bugs consists of those dependent on interactions between pairs of parameters - These can be caught with Orthogonal Array testing
• This technique gives effective coverage with a limited number of test cases
• This technique provides a small set of test cases – a practical alternative to testing all combinations
• It is a scientific, more efficient technique than random test case selection. It is a popular way to achieve a desired quality with reduced efforts and cost
• Test cases designed using such techniques are more likely to find defects efficiently by testing variables and values in combination

Fishbone Diagram for Shared Development Environment

Fishbone Diagram for Shared Development Environment

Case Studies from a Software Testing Department and General Business

Part 2 of a series of 3 -  Case Studies, Analysis and Fishbone Diagrams

Written by Girish Chincholikar

 

Case Study 2: Development environment is shared between Development and Testing members

The following categories were identified that might affect the problem

1. People - This was identified as People that are involved in the work activities
2. Environment – This was identified as Common environment was used across the project
3. Process – The process might not be in place
4. Infrastructure

Following effects were identified for PEOPLE

Testers were logging more defects

• due to unstable environment
• due to shared environment

Development people were rejecting defects

• as they were not reproducible
• due to shared environment

Rework for Testers

• Due to rejection
• Demotivation

Testers were demotivated

• As separate environment was not available
• No processes were available
• Defects getting rejected

 

Following effects were identified for ENVIRONMENT

Shared environment

• Business decision
• Budget constraint
• Infrastructure constraint

Unstable environment

• Increase in defect count
• Rework
• Productivity loss
• Defect rejections
• More test cycles

Following effects were identified for PROCESS

No defined Process
Process failure

Following effects were identified for INFRASTRUCTURE

1. Infrastructure constraint
2. Infrastructure under budgeted

Translating to a Fishbone Diagram

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Software Security Breach Notification Laws

Software Security Breach Notification Laws

 

Software Security Engineering Blog 6

Written by Maheshwar Kanitkar and Hemant Belorkar

Security breach notification laws have been enacted in most U.S. states since 2002. These laws were enacted in response to an escalating number of breaches of consumer databases containing personally identifiable information. The National Conference of State Legislatures maintains a list of enacted and proposed security breach notification laws.

[For more information, please refer - http://www.ncsl.org/IssuesResearch/TelecommunicationsInformationTechnology/OverviewSecurityBreaches/tabid/13481/Default.aspx]

BASEL III

BASEL III is a global regulatory standard on bank capital adequacy, stress testing and market liquidity risk agreed upon by the members of the Basel Committee on Banking Supervision in 2010-11

[For more information, please refer - http://www.bis.org/publ/bcbs188.pdf]

FISMA

The Federal Information Security Management Act of 2002 ("FISMA", 44 U.S.C. § 3541, et seq.) is a United States federal law enacted in 2002 as Title III of the E-Government Act of 2002 (Pub.L. 107-347, 116 Stat. 2899). The act recognized the importance of information security to the economic and national security interests of the United States. The act requires each federal agency to develop, document, and implement an agency-wide program to provide information security for the information and information systems that support the operations and assets of the agency, including those provided or managed by another agency, contractor, or other source.

[For more information, please refer - http://csrc.nist.gov/groups/SMA/fisma/index.html]

EU Data Protection Directive

The Data Protection Directive (officially Directive 95/46/EC on the protection of individuals with regard to the processing of personal data and on the free movement of such data) is a European Union directive which regulates the processing of personal data within the European Union. It is an important component of EU privacy and human rights law.

[For more information, please refer - http://ec.europa.eu/justice/policies/privacy/index_en.htm]

FFIEC

FFIEC compliance is conformance to a set of standards for online banking issued in October 2005 by the Federal Financial Institutions Examination Council (FFIEC).

[For more information, please refer - http://www.ffiec.gov/default.htm]

OWASP Top 10 / Guides

The OWASP (Open Web Application Security Project) Top Ten provides a powerful awareness document for web application security. The OWASP Top Ten represents a broad consensus about what the most critical web application security flaws are.

[For more information, please refer - https://www.owasp.org/index.php/Category:OWASP_Top_Ten_Project]

ISO17799

ISO/IEC 17799:2005 establishes guidelines and general principles for initiating, implementing, maintaining, and improving information security management in an organization. The objectives outlined provide general guidance on the commonly accepted goals of information security management

[For more information, please refer - http://www.iso.org/iso/catalogue_detail?csnumber=39612]

SCADA Security

SCADA systems were designed around reliability and safety, not security. Now SCADA systems are becoming increasingly interconnected with IP networks and have become vulnerable to Internet threats. The National Information Assurance Partnership, a partnership between the National Security Agency and NIST that administers the Common Criteria Evaluation and Validation Scheme for trusted systems.

[For more information, please refer - http://www.computerworld.com/s/article/97606/New_security_standards_to_strengthen_SCADA?taxonomyId=17&pageNumber=1]

OASIS

The Organization for the Advancement of Structured Information Standards (OASIS) is a global consortium that drives the development, convergence, and adoption of e-business and web service standards.

[For more information, please refer - http://www.oasis-open.org/]

Organizations intending to develop secure software, must define their security engineering practice. There are very specific requirements for building secure applications. The security engineering process starts right in the software design phase through to software testing, deployment phase and ends with the maintenance phase.

Software security is the ongoing process of exercising due care and due diligence to protect information, and information systems, from unauthorized access, use, disclosure, destruction, modification, or disruption or distribution. The never ending process of information security involves ongoing training, assessment, protection, monitoring & detection, incident response & repair, documentation, and review. This makes software security an indispensable part of all the business operations across different domains.

Outsourcing to vendors who are operating as an independent testing practice may be a viable option to manage the specific needs of security engineering and testing of applications for security and regulatory compliance.

Fishbone Diagram

Fishbone Diagram  

Case Studies from a Software Testing Department and General Business

Part 1 of a series of 3 Case Studies, Analysis and Fishbone Diagrams

Written by Girish Chincholikar

What is a Fishbone Diagram?

The fishbone diagram was originally developed by Professor Kaoru Ishikawa is often referred to as an Ishikawa Diagram.  The technique can help structure the process of identifying possible causes of a problem.  The diagram encourages the development of an in-depth and objective representation ensuring all participants keep on track. It discourages partial or premature solutions, and shows the relative importance and inter-relationships between different causes of a problem.

The method is ideally used over a number of meetings, enabling the team to become deeply immersed in the problem. Fresh suggestions regarding possible causes can arise during the break and members are more likely to forget who originated every idea, thus making subsequent discussions less inhibited.

Causes can be derived from brainstorming sessions. These groups can then be labeled as categories of the fishbone. They will typically be one of the traditional categories mentioned above but may be something unique to the application in a specific case.  In this series we look a several business situations, analyze possible factors to the problem and then develop the appropriate Fishbone Diagram.

Case Study 1:  Why do we get Software Defects in an Application

Steps to develop a detailed Fishbone diagram

Identify the Categories that affect the problem

Identify the effects from each category

Identify the detailed effects

 Following categories were identified

1. User Error
2. Transmission of Data
3. Coding of Application
4. Databases
5. Environmental Changes
6. Security breach

 Following effects were identified for USER ERROR

1. Faulty Assumptions by the user
2. Users perception
3. Not knowledgeable
4. Over confident

Following effects were identified for TRANSMISSION OF DATA

1. Modules not integrated
2. Failure in Integration testing
3. Flaws in the transmission design
4. Integrated incorrectly

 Following effects were identified for CODING OF APPLICATION

1. Coding standards not followed
2. Unit testing not performed
3. Bad code written
4. Requirement not covered

Following effects were identified for DATABASES

1. Functions written incorrectly
2. Triggers written incorrectly
3. Data redundancy
4. Data inconsistency
5. Problems in Database versions

Following effects were identified for CODING OF APPLICATION

1. Coding standards not followed
2. Unit testing not performed
3. Bad code written
4. Requirement not covered

Following effects were identified for ENVIRONMENTAL CHANGES

1. Data inconsistency
2. Incorrect environmental setup
3. Managed by 3^rd party vendor

Following effects were identified for SECURITY BREACH

1. Technical flaw in the system
2. Logical flaw in the system
3. Security testing not carried out

The next Step would be looking at the above list of effects and then developing a more detailed list of effects resulting from these.  This is beyond the scope of this blog but must considered to get full benefit of the Fishbone Diagram.

Fishbone Diagram for Software Defects