Too many options
Yes, this space is more crowded than ever before. It’s easy to get lost with most tools talking about “fancy” capabilities but failing to get the basics right. The success rate from test automation still remains way below expectations. It is important that a tool is capable of handling the complexities of a modern application landscape and at the same time, help you build sustainable and low-maintenance automation.
Here is a simple framework you can use to help you maximize the value of your time and focus on the critical aspects you expect from the tool.
Narrow your list down
Sometimes when doing comparisons, it’s easier to strike out some options based on obvious factors and work with a shorter list. It helps maximize your time on trial and evaluation and really ensure it works for your business and app landscape.
Easy to get mislead by traditional “record and play”. It’s marketed and comes across as magically fast, this technology is a decade older. You can put lipstick on a pig, but it’s still a pig. The link to the blog goes into more details of why record & play just doesn’t work in the real world.
Stay away from tools whose core expertise is not automation, like model-based testing tools that “integrate” with automation tools. Or tools that started out as desktop application testing tools and added capability for web and API.
If the automation tool does not offer core functionality such as managing automation assets, reporting, and tracking, and service automation – don’t waste your time. And don’t get into the “integration” trap. Core functionality should be native and easy to achieve in a simple manner. Breaking the core lifecycle of automation into multiple tools only brings inefficiencies.
Codeless automation tools can be challenging. Moving all the test logic into some invisible executable is not really helping the purpose. Tools that drive test logic from excel spreadsheets suffer from obvious shortcomings.
Stay away from the tools that claim to do everything under the sky. For example, performance testing is better handled by a specialist tool.
Capability assessment framework
This framework will help you score on capabilities that are critical to making automation fast and sustainable. To start with just a simple Yes/No checklist could go a long way to ensure a 360° evaluation. The capabilities are divided into technical-functional, accessibility, adoption and the ROI to do benefit analysis.
Technical & Functional
Business process focused
Automated test design
Reusability / Modularity – Need of custom frameworks
Robust element identification
Seamless Data-driven testing
Test asset management & inbuilt traceability
Change Impact management & Automated Change management
Execution tracking and cycle management
Service and API automation
Accessibility
Ease of use for the broader test and project teams
Centralized access to test automation assets
Ability to execute tests and analyze results at a functional level
Ability to set up governance
Adoption
Easy to adopt
Need of specific programming skills and resource dependency
Vendor locking
Hidden costs (like execution agents, add-on modules, protocols, accelerators costs, maintenance and support add-on costs..)
ROI
What are the key components to analyze ROI? There are many things here, but I am only listing the ones that are the usual suspects. They are very basic, but these are critical to automation that can give you returns on your investment.
Need for framework development
Element setup
Data definitions
Runtime & executions
Change Management
Get the most from your trial and evaluation
It’s tough with our day jobs to give time and energy to evaluating a new tool. Here are some quick tips to help you thru the process:
Don’t get caught up by exception use cases. Consider what serves 80% of your Automation needs
Carefully observe the learning curve
Emulate situations for future change adaption and maintenance
Consider how easy/difficult will it be to move away if needed in future
Review pricing model for hidden add-on costs. You want to make sure ink does not cost more than a printer!
Finally…
If I had to really pick the Top-5 things, here are critical ones that you shouldn’t compromise on:
Ability to automate UI and API simply and in the same flow.
A tool that could potentially handle the complexities of today’s technology stack and robust element handling without any custom framework and programming overhead
Something that can keep up with the continuous nature of changes and where maintenance doesn’t kill the ROI and outcome is more sustainable
Don’t get locked into automation assets that would be proprietary to a specific tool
An automation tool that’s simple to adopt and has test management, reporting, and CI integrations built-in.
If you are thinking about going with or sticking to selenium, this is a comprehensive summary of how you can potentially leverage all the power of selenium without any of its framework and programming overheads
Do check out how ACCELQ scores and compares and sign up for a trial to experience the difference.
View & Download Original Source @ https://www.accelq.com/blog/how-to-choose-test-automation-tool/