Presentation & Conclusion

Preparation for the presentation

Today, all of the members of the group met in the labs and we decided on how to format and deliver our presentation in a timely manner.

We each individually worked on our designated information for the slides and as a group reviewed the presentation together. We timed and practiced test runs of our presentation in order to make sure that we had enough time to deliver all of our information within the time limit that was set for us. We also came up with questions that might be asked after our presentation and had a mock question-answer session.

Our conclusions from our separate testing technique showed us the main flaws in the program code which came to light during our individual testing. The principal errors within the code are as follows:

• Accepts and uses negative numbers to calculate answers
• Entering a 0 input leaves the user unprompted while the computer waits for a valid input
• Entering the inputs one after the other in a horizontal line gives a different calculation to those entered along with a carriage return after each input.
• Accepts inputs for 'locations' where the values are not increasing consecutively, i.e. 2 4 3 6 is accepted and gives the incorrect answer

The most important conclusion that can be drawn from these problems is that the original program needs to be rewritten with the above flaws in mind. The main function of software testing is to improve the program by "breaking" it which we feel our test cases thoroughly did.

BVT Test Cases & Table

Test cases

                                                              BVA on Steps Inputs

     Test case 1 - Expected outcome - Error Msg: Invalid input (# steps must > 1),Prompt user

                          Actual outcome     -The program as seen above fails to prompt the user for a valid                                                                  input which would be values between 2 and 120. 

                          

     

     Test case 2 - Expected outcome - valid output 

                           Actual outcome    - valid output

    Test case 3 - Expected outcome - valid output 

                         Actual outcome    - valid output 

x

      Test case 4 - Expected outcome - valid output

                           Actual outcome    - valid output 

     

       Test case 5 - Expected outcome - valid output 

                           Actual outcome    - valid output 

     Test case 6 -  Expected outcome - Error Msg: Invalid input (1 < #steps < 121),Prompt user.

                           Actual outcome    - The program accepts 121 steps and calculates the largest                                                                           difference as seen above.

BVA on Location Inputs

     Test case 7 -  Expected outcome - Error Msg: Invalid input (# steps must > 1),Prompt user

                           Actual outcome    - The program should not accept values < 1 but as seen above it                                                                outputs a value.

     Test case 8 - Expected outcome - Error Message: Invalid Input (must be greater than previous), Prompt User

                           Actual outcome    - outputs a value

     Test case 9- Expected outcome - valid output

                           Actual outcome    - valid output

     Test case 10 -  Expected outcome - valid output

                           Actual outcome    -  valid output

     

      Test case 11 - Expected outcome - valid output

                           Actual outcome    - valid output

     Test case 12 -  Expected outcome - Error Message: Invalid Input (must be greater than previous), Prompt User

                           Actual outcome    - The program outputs a value ignoring the repeated no.

BVT Table

Test cases	Steps	Locations	Expected output
Boundary values on steps Robustness Test case - test case 1
1	1	-	Error Msg: Invalid input (# steps must > 1), Prompt user
2	2	2, 5	Valid output - 5
3	3	12, 6, 15	Valid output - 5
4	60	115, 151,195......	Valid output
5	120	7,10,15…………	Valid output
6	121	15,19,35............	Please re-enter a valid input
Boundary values on locations, Robustness Test case - test case 14
7	2	-1, 2	Error Message: Invalid Input (must be positive), Prompt User
8	3	2, , 5 - below min	Error Message: Invalid Input (must be greater than previous)
9	3	2, 3, 6 - min	Valid output
10	3	2,  4, 6   - nominal	Valid output
11	3	2,  5, 6   - just below max	Valid output
13	3	2,  5, 6   - max	Valid output
14	3	2,  6, 6   - above max	Error Message: Invalid Input (must be greater than previous)

• I first did the boundary values tests on the steps input then the location inputs
•  According to the project specification the safety officer is only interested in the gaps between the steps and so is not interested in the height of the first step from the ground. Therefore the first location value doesn't require any boundary value tests. 
•  Boundary values for the subsequent values is evaluated using min, above min, nominal, max, above val.

DD Path Testing: Final Steps

My last post concerning DD Path Testing will include a few more test cases beyond the 16 I originally laid out in my previous posts. I will also detail a few flaws I discovered within the program and attempt to give possible solutions to these problems whilst explaining my further test cases.

The following test cases concerns what occurs when we input negative and decimal numbers. Obviously we can't allow a negative number of 'steps' or negative 'locations' as this doesn't makes sense for obvious reasons. Concerning decimal numbers, while we cannot have a fraction of a step , we can have decimal numbers as distances. However, in our assumptions relating to the schematics of the program given to us, we have decided that since the schematics define the gaps as being units, we are taking this to mean numbers of the natural set, i.e. positive, whole numbers.

Interestingly if we use a negative number as the number of 'steps', it does accept it. Clearly, this is a fault within the program. Besides this, it also only allows one 'location' to be entered although this has no bearing on the answer as it is 0 each time. This is most likely due to the logic of the program never allowing it to enter the third loop as the number of steps is already less than 0.



This is also another violation as it both accepts a negative number as a valid 'location' which it shouldn't and as a result, it remains a reason why the largest gap is 4 where it should technically be 1 in the first example and why it is 8 instead of 2. This kind of flaw would lead to incorrect results and may cause some consequences in the real world. As such, ensuring the program does not accept negative numbers at any stage in the program is integral.



This test case of decimal numbers covers both decimal or double numbers being entered in as both the number of 'steps' and the 'locations'. In both cases, the exception is caught and we are prompted to enter a valid input, even for negative decimal numbers.

The next two test cases I have conducted were simply random and I am still not entirely sure what the results relayed to me means. However, I do think both are a result of a logical or structural flaw in the program, something possibly overlooked. In any case, it is a good sign that we have stumbled upon something that is hard to understand how it occurred as that is confirmation we are testing correctly and thoroughly.



While at first, I was unsure how this answer was calculated, it does make sense when I looked over it again. First there is the issue of having all of the values on a single line. It seems to be causing trouble in this case, although it works for others. The largest gap it initially comes across is 11 which is also the correct answer. The program only requires one more valid input for its computation however due to the inputs being on a single line, it does not seem to process these as they contain a string. Although this seems to be untrue for the last line of input as it terminates the program. While it causes some confusion, it seems most of the issues would be resolved by entering each value and then entering a carriage return to make the process easier to read. However, there could be some changes made to the original program so it could process it either way the inputs are entered.



Although I am not totally sure what is happening in this test, I know that it is more than likely a programming error within the program code causing this unusual test result. Also it is possibly that all of the inputs on the same line is affecting the computer's ability to calculate here, as the above example. If I were to enter each input after a carriage return, this problem would cease. However, software testing is about breaking the program and finding faults in the code in order to create a better program.

With the last examples of my tests, I have come to the end of my testing process concerning DD Paths. While it took some time to understand the technique, I feel like I can test programs more confidently now and spot errors in code a little better. While this program we have is flawed, I believe that in keeping it fairly basic, we could understand all the different inputs we have to account for and why testing is important to make sure all of our statements of code are working correctly.

As I have pointed out in all of my test results, a rather substantial amount of changes to the program is necessary to handle all of the errors we have found. Of course, there may be other errors that DD Path Testing has missed and other testing techniques would find easier, however I feel like if most of the errors I pointed out in my posts were taken care of in the program, it would run the way it is supposed to. Until we find more errors with another batch of testing perhaps. 

Decision Tables

In this blog post, the test cases for the decision tables will be executed and explained. From the previous tables wherein the conditions resulting in which actions were detailed, we created the test cases below to facilitate such behaviours and determine if the actual output of the program reflects that of the expected output. It they match, then the program works for those specific conditions. Otherwise a fault is found.

TEST CASES	STEPS	LOCATIONS	EXPECTED OUTPUT	ACTUAL OUTPUT
1	1	-	Error Msg: Invalid input (# steps must > 1), Prompt user	Enter locations: prompt
2	s	-	Error Message: Invalid input (not an integer) Prompt user	Please re-enter a valid input
3	-1	-	Error Message: Invalid Input (must be positive) Prompt user	Enter locations: prompt
4	3	5, 10	Prompt user	Prompt User
5	5	2, 3, 6, 4, 10	Error Message: Invalid Input (must be greater than previous), Prompt User	6 (Wrong Output)
6	5	-1, 2, 4, 6, 10	Error Message: Invalid Input (must be positive), Prompt User	4 (Wrong Output)
7	5	2, 5, 8, l, 12	Error Message: Invalid input (not an integer) Prompt user	Please re-enter a valid input
8	5	2, 3, 6, 4, e	Error Message: Invalid input (not an integer) Error Message: Invalid Input (must be greater than previous) Prompt User	Please re-enter valid input
9	5	-2, 4, 6, 5, 10	Error Message: Invalid Input (must be positive) Error Message: Invalid Input (must be greater than previous) Prompt User	6 (Wrong Output)
10	5	-10, 3, 5, 6	Error Message: Invalid Input (must be positive) Prompt User	Prompt User
11	5	2, 4, 5, 3	Error Message: Invalid Input (must be greater than previous) Prompt User	Prompt User
12	5	2, 4, 5, e	Error Message: Invalid Input (must be greater than previous), Error Message: (must be an integer) Prompt User	Prompt user
13	5	-10, 2, 4, 3	Error Message: Invalid Input (must be positive) Prompt User	Prompt User
14	5	2, 4, 6, 8, 12	Print Valid Output	4

1. Steps is not greater than 1

As per the condition s must be greater than 1, the program's output/action is not reflective of the expected output in that it continues to ask for the locations even though the input for steps is not greater than one. Logically, for there to be a "difference" there must be two or more locations. However this program does not seem to factor that in.

2. Steps is not a whole, positive numerical number.

Here the input for steps is not an integer and therefore should not be accepted. As expected, the program rejects the letter 's' and asks the user for a valid input. It passes this test.

As a sub-section of that test, if a negative number is entered however the program accepts it and proceeds to ask for the locations. This is clearly a fault as it should only accept integers greater than 1 for the variable steps. Furthermore, logically, there is no such thing as a negative one step. When continued, if the user does enter a valid integer, the program simply prints "The largest difference is 0" which does not mean anything. It fails in this sense.

3. The number of locations entered is not the same as the value of steps

The program does as expected and waits for the user to input more locations until the number of locations entered is the same as the value of the steps. It does this in both entering the values one by one (each value entered after a carriage return) as well as in one line. It passes this test.

4. A location entered is less than a previous one

Here the program accepts all of the locations entered, even though the fourth location (4) is less than the previous one (6). This does not make sense logically as a subsequent step cannot be lower/closer to the ground than the previous one in a conventional ladder (wherein subsequent steps are higher than the last/previous). In turn, the program outputs the wrong answer (6 from 10 - 4), wherein it should have warned the user to re-enter that value or organise the values in such a way that they are in order.

5. A location entered is negative

Again the program simply accepts a negative number even though it does not make sense in the context. The distance of a location from the ground up cannot be measured in a negative sense. Therefore the program should warn the user of this fact and ask them to re-enter the value. Here it provides the wrong answer once more (4).

6. A location entered is not a whole, positive numerical number.

The locations entered in this example are 2, 5, 8, 'l' (the letter l).  When each value is entered using a carriage return, the program catches the error as an 'l' is not a numerical number and asks the user for a valid input, reflecting the expected action in this case.

The same happens when the locations are entered in one line. The first value entered replaces the 'l' and the user has to input the remaining values again as well. However, even with the hindrance of inputting the value again, it does return the right answer. 

7. A location entered is less than a previous one and it is not a whole, positive numerical number.

There are two rules not being fulfilled in this case. The fourth location (4) is less than the previous one (6) and an 'e' is entered as the last value. The program catches the error of the latter and asks the user to re-enter a valid input but does not do anything about the first problem. This poses as a problem as it will try to calculate the largest difference given the these values once the user inputs a replacement for the 'e'. This is clearly a fault.

8. A location entered is less than a previous one and a location is negative

Again, two rules are being violated in this case as the first location is a negative and should not be accepted and the fourth location (5) is not greater than the previous one (6) and should also not be accepted. The program does not catch either of these faults and proceeds to calculate the wrong largest difference between the values entered, leading to the wrong output of 6 (from 4 -- 2).

9. A location entered is negative and the number of locations entered does not equal the value of steps.

The program accepts the first value entered (-10) which it should not have (for reasons stated previously). However it waits for the user to keep entering values for locations until it reaches the number of steps. The only error here is that it accepts a negative value when it should not

10. A location entered is less than a previous one and there are not enough locations entered

The fourth location entered is 3 which is less than the previous (5). For reasons stated above, the program should not accept this as a valid input and should ask the user for a value greater than the previous. Clearly, this is a fault in the system. However, it does still keep prompting the user for more values until the number of locations filled equal the steps.

11. The number of locations entered is not enough, a location entered is less than a previous one and a location entered is not a whole numerical number.

Again, the program does not catch that a location entered is less than the previous one (i.e. the third input 3 is less than 4) and allows for that to be valid inputs. However it does catch that a letter 'e' is entered and asks the user to re-enter a valid input for that location. It also waits for the user to input 5 steps.

12. The number of locations entered is not enough, a location entered is less than a previous one and a location entered is a negative number.

In this case the program still accepts the negative number, which would lead to a wrong output in the end if the user manages to enter enough valid values. Here the program is still waiting for enough values to be entered until the number of locations equal the steps. As well as that, the fourth location (3) is less than the previous one (4), which the program does not warn against/catch, and will lead to the wrong output.

13. The steps entered is greater than 1 and is a whole, positive numerical number and if the locations entered equal the value of steps and each subsequent location entered is greater than the last and they are all whole, positive numerical numbers.

In this case, the user enters all of the inputs perfectly that is each subsequent number is greater than the one previous, they are all whole positive numbers and the user entered enough to reach the required number of steps. It produces the correct output.

CONCLUSION:

Through these tests, the program in question only managed to pass 3/13 tests (2, 3, 13). It shows that there are many faults within the code as it does not cater for many cases of invalid inputs such as:

• negative numbers
• where a location entered is less than the previous
• a combination of the two above.

As well as that, when the program does encounter an error it does not specify what a valid input is. Even if it catches certain faults such as when an input is a string or a character, it overlooks the other ones even if it happens in the same sequence. This could be solved by continually rejecting negative inputs and continually compare if the current input is greater than the previous one, before performing calculations on the set of data.