Test automation has been adopted and used effectively in the IT industry for over a
decade now. Some of the core objectives of using test automation, along with conventional testing, is to repeatedly test certain actions, logics, and business functionalities with the end goal of increasing the effectiveness, efficiency, and coverage of the software in test. However, as the use of test automation has become more and more extensive over the last decade, some myths associated with automation testing have arisen and are in need of addressing. Online’s team of QA experts have banded together to dispel some of these myths and to ensure that anyone looking to utilize test automation knows that it is about far more than just looking for software defects.
- “Using automated software testing means that no further manual testing is required”
Both automated and manual testing are equally important and one is simply not a replacement of the other. Using automated scripts, we can rapidly go through the pre-existing test cases and validate functionalities and hot-fixes. This gives the QA team ample time to investigate more, discover new information, and exercise the newly-added functions of the application/software. Also, it is important to diligently identify the test cases that would require to be automated to provide the best ROI. In my experience, I have always tried to prioritize test cases by communicating with leaders/stakeholders so that the essential business requirements or testing goals are met.
A good example of when manual testing is needed is when there are hot-fixes for a release that were not integrated in the automated tests. In those cases, manual testing is essential to verify that the defects are no longer existent in the software or application. In conclusion, no matter how complete the automation tests are it does not negate the fact that manual testing is also needed.
- “Test automation can only be implemented at the Graphical User Interface (GUI) level”
Today’s project managers and QA’s are moving towards Agile based methodologies, which allows for testing to no longer be limited to the User Interface layer. Early feedback is brilliant for any agile project, so if we’re concentrating only on the UI layer, we are left waiting until the UI is developed and ready to test. Now we can test even before the UI is developed and we can directly test the APIs using tools like Cucumber, FitNesse or SOAP UI. Following this approach tests not only small cosmetic changes or validations on the GUI, but also gives developers more time to fix any bugs.
- Test automation can also be implemented at:
- The data level using Ruby or Java to perform a query from Data Base
- Backend systems using Bash, Shell, Expect, Python, and Java
- Desktop applications with tools like AutoIT or HP UFT
- Mobile applications using Appium
In conclusion, we can use test automation at many different levels in various platforms or environments.
- “Test automation will always help improve the quality of the software or application under test”
Simply put, test automation is not guaranteed to improve the quality of software because the test automation steps are pre-defined, leaving room for human error. If the test cases are not written diligently and with proper understanding of functionality and business requirements, then the automated test may not help the quality of the software/application at all. A full understanding of the software’s important functionalities will allow the appropriate test automation cases to be made, ultimately improving the quality of the software. In addition, the QA person developing the automated scripts needs to have the aptitude and curiosity to be able to understand the main goals/objective of the test case and add his/her experience into it to make it more robust and efficient.
- “Test automation development is independent of the Scrum team in an agile project”
The name ‘test automation’ may lead you to believe it can work as an independent entity, especially given the pre-requisites for the development are essentially the documented test-cases, automation related tools, and test environments. However, opting to utilize such a methodology could have a negative impact on the project. Leaving it to work independently would likely lead to a lack of communication or flow of information, causing the automation team to not be fully aware of the changes applied to the current functionalities or the new ones that will be added in the sprint. Therefore, more time is invested in the development and maintenance of the tests. To get the maximum benefit, the scrum team should work collaboratively with test automation in terms of identifying the automated functional and regression tests, confirming current sprint goals are aligned with automated functional tests and strategizing the development of the tests that are more stable. It is also important in an agile based project for test cases to be updated on a regular basis to better assist the test automation team in terms of robustness and maintainability. Therefore, the cooperation between both teams is essential to make the project successful.
- “Test automation is a short-term investment that yields immediate results and reduces testing effort instantly”
It is true that one of the main objectives of test automation is to reduce testing effort, but the reality is it may take some time to gain this benefit, especially when the application or software is changing rapidly. The first few development cycles are crucial and require the most attention because there is an initial need to investigate the product, select different tools, prepare POC, compare different options, and develop basic infrastructure based on existing/new framework to support testing layer. Therefore, the development has high initial investment, but once the application becomes stable and the automation personal is also accustomed with the application, then the maintenance becomes much easier and we start to witness the true benefits of test automation. That’s when the ROI would start to become more visible.
In short, test automation is an incredible tool that can go lengths to ensure the software being developed is of the utmost quality and meets the needs of the users. It is not the be-all and end-all of testing however and a manual, human touch will likely still be required.
Did we miss any myths? Feel free to leave a comment below! If you would like to learn more about Online’s Business Consulting practice, click here.
Submit a Comment