Metamorphic Relation Prioritization for effective testing

Posted on by admin

Writing Guides

 

This is a paperfor thesisfor my master computer science. It’s represent a research in Software Testing field. This paper should be45 pages single space. Using the provided Latex template fromin this website following their format and layouthttp://www.montana.edu/etd/

 

Just write the chapters below to fill the provided template

Contents:

  1. Abstract
  2. Introduction
  3. Background
  4. Related work
  5. My Approach
  6. Results and analyze
  7. Threats of Validities
  8. Future work
  9. Conclusion
  10. Reference

 

 

 

 

 

 

 

 

 

 

 


 

 

  1. Abstract

 

I copied what fits my workfrom abstractsofreference 3 andreference 5to give you some understanding about what my thesis about– create a new abstract

The choice of test oracle—the artifact that determines whether an application under test executes correctly—can significantly impact the effectiveness of the testing process. However, despite the prevalence of tools that support test input selection, little work exists for supporting oracle creation. (coped from reference 3)

 

We proposed a method that supports test oracle creation, which is, going through using effective Metamorphic Relations from another paper (I used Metamorphic Relations of reference 4) paired with mutation analysis to generate a prioritized Metamorphic Relations order in terms of fault-finding effectiveness.

 

Experimental results obtained by employing our method overmachine learning classification algorithms which support core functionality to many application domains – such as bioinformatics, computational linguistics, etc.

reveal that our method has an effective approach for producing effective oracle data sets, increasing the testing efficiency by maximizing early fault detection. Improvements over best practice of up to 80% percent were observed. . (coped from reference 4 and me)

 

 

some program cant test no test oracle . metamorophic testing solve that. Applying metamorophic relation on program can catch faults in a program.

 

Our approach uses a set of features developed using the control flow graph of a function for predicting likely metamorphic relations. We show the effectiveness of our method using a set of real world functions often used in scientific applications.

  1. Introduction
READ ALSO :   Continued Professional Development

 

 

  1. Background

5.1Metamorphic testing,

5.2Mutants testing,

5.3Machine learning (similar to reference 5)

 

  1. Related work

6.1 reference 3,

6.2 reference 4,

6.3 reference 5

 

 

 

 

 

 

 

 

  1. My Approach

 

This approach need Mutants and Metamorphic relations.All my mutants generated using Mujava tool using existing programs. I did two experiments:

 

My approach illustrated in Figure

 

 

 

My approach Steps:

 

First, we generate a collection of Mutants(MTs) from the Program under Test (PUT) and divided Mutants into two sets: evaluation set (ESet) size 2/3 f and training set.(TSet) size 1/3

Second, finding the appropriate Metamorphic Relations and pick a random order(MRs) and run it against the training set using initial and follow-up test and logs of the results.

Third, based on log information,we generate the ranked order by prioritize metamorphic relation (MRs) effectiveness.

Next, we run the ideal, rankedand random orders against the evaluation set (ESet)then logs of the results.

Finally, start estimating based on the three approaches,

 

 

We hypothesize that this oracle creation will produce an oracle data set that is more effective at earlier fault finding than an oracle data set selected with a standard (Random) approach. Also, faults in the mutants will also be likely to reveal faults in the actual system, the tester defines expected initial and follow-up values for each metamorphic relation of the oracle data sets. Testing then commences with a small, and potentially highly effective, oracle.

 

  1. Experiments Setup:

7.1.1 Program Under Tests(PU1).

  1. Experement1

PUT1: using program in reference 4.

And These two imagesgive more information

READ ALSO :   Academic help online

(u can resize)

b.Experement2:

PUT2: using Weka library similar to reference 5. (weka IBK and wekaKStar libraries)

7.1.2Mutants (MTs):

I used Mu java tool to generate mutants for the two experiments more information about how this mutants generated in reference 8 and what is mutants operation types

7.1.3 Metamorphic Relations(MR)

In both experiment I used existing MRs from reference 4 for the second one from reference 5

  1. Experement1(initial case study):

Metamorphic Relation:using MR in reference 4.

b.Experement2:

Metamorphic Relation:using MR in reference 5.

 

  1. Testing Mutants against Metamorphic relations:

This experiment is done 10 time and took the avg

  1. Take Program Under test (PUT) create mutants(MTs) using MuJava
  2. Loop through all the mutants (MTs)
  3. Create randomly initial test cases pass it to the Mutant and record the result (Result A)

Ex.

Array= [1,2,3] , avg = 2 (result A)

  1. Loop through all Metamorphic relation (MRs)
    • create follow up test case from initial/MR[i] and pass it to the Mutant and record the result (result B) ex. MR is additive , x=3 Array= [1+x, 2+x, 3+x] follow Array= [4, 5, 6] result B = 5
    • compare result A against result B using MR[i] propertya(resultA< = result B) (2 <= 5)
    • if not satisfied MR[i] properties call it a catch faults else it’s pass and we record the result
  1. Remove week Mutants from the Set (more information about week mutants in reference 3)

 

  1. Prioritization:

Input to this step the Set results of step 5.2. I used greedy approach to generate the results:

Greedy Approach:

Input (set)

Get MR with largest number of kill

Record the MR

Remove mutants killed by this MR

Output(order)

 

Prioritization Step:

  1. Divide the Set into 2/3 to Training Set (Tset) and 1/3 to Evaluation set (ESet)
  2. Create Random order for MRs (MR_Random) (MR1, MR2, MR3)
  3. Apply Greedy To generate Ranked Order (MR_Ranked)
    • Loop through Training Set
READ ALSO :   The Arab-Israeli Conflict

Get MR with largest number ofkilling mutants

Record the MR

Remove mutants killed by this MR

  1. Apply Greedy To generate Ideal Order (MR_Ideal)
    • Loop through Evaluation Set

Get MR with largest number of kill

Record the MR

Remove mutants killed by this MR

  1. Use Greedy approach to test Orders ( Random, Ranked, Ideal) against Evaluation Set
  2. Recorded the results

 

  1. Evaluation:
    1. my questions

 

  1. Mutants:
  2. What is ideal number of Mutants that a program should have to apply it in our approach?
  3. How many times does this approach need to be trained to get the best results?
  4. Metamorphic Relation:
  5. What is the ideal size of the MR collections?
  6. Sets:
  7. What is best sizes for the division of Training and Evaluation sets?
  8. Approach:
  9. For a given set size what MR ordering approach performs the best?

 

  1. when to log

Similar to reference 3

 

  • oracle evaluation (random, ranked, ideal)

Similar to reference 3 and to reference 5

 

 

  1. Results and analyze
  2. Statically analysis

Similar to reference 3 and to reference 5

 

  1. Answer my question

 

  1. Threats of Validity

 

  1. Future work

-Create a new order from clustering the mutants according to their types

– pick another approach for creating sets rather than greedy

-using cross validation on sets

  1. Conclusion

 

  1. Reference:

This is the main references for my approach but I will need to have more use as much reference as you can add to the thesis

  1. Metamorphic Testing (Background)

http://www.isa.us.es/sites/default/files/segura16-tr.pdf

  1. Mutants Testing (background)

http://ieeexplore.ieee.org/document/6802989/

  1. Automated Oracle Data Selection Support (method)

http://ieeexplore.ieee.org/document/7112189/?tp=&arnumber=7112189

  1. MR for Math’s Programs (MR + case Study experiment 1)

http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6698899

  1. MRs for Machine learning (real experiment with MR)

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3082144/

  1. mu java (tool)

https://cs.gmu.edu/~offutt/mujava/

  1. weka(tool)

 

  1. Mutation Operation

https://cs.gmu.edu/~offutt/rsrch/papers/oomuts2011.pdf