First of all we like to thank everybody for joining our talk in such big numbers. If you joined the talk, please rate it at https://agiletestingdays.com/2017/session/regression-testing-1/ (we already have some great idea´s for next year, so we like to speak again)
In this wrapup we will share with you the remarks that were made after the talk on the boards in our sponsor-booth and answer some questions that we could not answer during the session. I also reveal the new name for the test formally know as the regression test.
On the board ´Action without a thought is waste, so stop Regression Testing´
- just experiment
- continuous delivery
- stop with waste
- saves a lot of time
- why test when we do not get new information
- apps with a single purpose that do not require many changes do not need a regression test
- user story testing over regression testing
- action without thought is waste
- like for testing in general, depends on context
- let it fail in production, it will keep us occupied
On the board ´I Love Regression Testing´
- just experiment without and see what happens
- love to have them automatically done in my pipeline for each changed feature
- just to be sure that everythings good
- good, but not 100% proof
- regression test first …. bugs second
- worst idea since ISTQB
- feeling secure about your code while implementing many user stories
- it´s called code coverage
- just try and see
Our goal was to get you thinking about your daily work and wheater you are doing the right things. It is not always a good idea to not have a regression test: only if there is a real risk and you need this kind of expensive big test to uncover new problems. Otherwise, you can do without. We did actually test our idea at one of our customers (a big government program, so not a small test), and you can actually do without a regression test without having all kinds of issues.
We could not answer all questions immediately, so here are the missing answers:
- what do you mean by RT (on what level) ==> at first our idea was that it should be anything after the Unit Test, however after some discussion in the team we decided that we want to include the UT as well. What is the use of running a unit test on an unchanged unit. You know for sure it will not fail and it does not say anything about the quality of the product. The time should be invested in checking wheater the UT is a good UT and only rerun a UT(by hand is fine) if the unit is changed. We did however not test this situation is real life, so if any of you did and it worked, please let us know.
- how about 1 code base with multiple teams working on the same product ==> this one took some more thinking, however in the end we drilled it down to the same point: do you know what changed? If you do, then you don´t need a RT for sure. If you don´t know what changed and when and by who, then you are not in control and you will need something to get in control (rather then an expensive RT that doesn´t bring real value).
- how about multiple teams working on connecting applications in a chain ==> if your team is changing an application without knowing it´s surroundings, that is definitely a risk. Probably you should consider to implement LESS, to help the teams communicate (you should be able to communicate without, but if it helps, it helps).
New name for the test formally know as the Regression Test
This the test we described as ´i am not in control of my changes test´, the test that you should keep:
- with variable test data
The new name we liked best is: Consistency Check