Test Automation pyramid and related discussions offers enough education as to why GUI tests could become a beast of burden over time. It could risk one’s neck if GUI tests were written without considering the user intentions. Heard situations where regression tests were passing but the system isn’t fit for use because it doesn’t fulfil a meaningful workflow.. tests were barely checking whether UI elements were populated and populated with values. Isn’t the job of UI unit tests? There are plenty of guidance around BDD, designing and writing meaningful GUI tests.. and in addition to that, what I think will add more value and minify the burden of brittle GUI tests is more emphasis on API/service tests. These days, most of the products are built around service oriented architecture, offers RESTful or some sort of web services. Most of the UI is modern, developed for the web, decoupled from the business logic and talks to the web service to perform business logic. In these kind of situations, API tests are absolutely necessary. Even though, web services are private, known to my GUI only , it would add a lot of value testing thru web services, will reduce considerable overhead and heavy lifting that the GUI tests carry.
While there are plenty of tools around to help with API testing, I’m very impressed with Visual Studio Web Testing. Exploring other tools like Frisby Js and Yadda. More tooling specific write up will be out sooner, but for now, staying away from tooling and stressing more around the intentions. Tests with good intentions will yield better results, so, what needs be tested at the API layer that can offer more confidence?
Assume for every button click or tab out or equivalent action from GUI, your application calls out an API/web service to perform some business logic. GUI is updated based on web service call response. So, GUI needs to be tested for user interactions and GUI representation …data driven verification, thorough business logic verification can be offloaded to API testing. Also, API tests can be written while writing API code and need not wait for GUI to be ready., short list in my opinion, of course, there are more..
- How do we send data – Headers, Query string parameters, payloads?
- What headers are mandatory?
- What parameters are mandatory? What is optional?
- What if expected header information is not sent?
- How payload is validated?
- What is the content type for payload?
- Is payload validated for types, null, or correct values?
One of my recent experience reminds the last point. One of the steps from the API test workflow made a call to an endpoint with appropriate headers and payload. In the business logic that call serializes the data and calls out to rules engine. We observed Rules engine crash and after 2 hours of debugging with the developer, we found that data serialization returned null and a call to rules engine with null caused all the trouble. Root cause was the API call did carry payload but not content-type for the payload.. although this may not happen as our GUI always calls out with appropriate headers, this leaves an easy whole to crash our system for intruders. Also, surfaces real code quality issues. It was evident that the entire class/controller implementation had no null, type check what so ever.. forgotten hero “Defensive coding” ..
“Defensive Coding in C#”.
Especially, API testing will help elevating confidence around Validation and Exception handling
- Does input validation work?
- Do we evaluate input data schema?
- How do we handle missing parameters?
- How do we handle wrong input?
- Is API giving right error codes?
- Does it issue appropriate error if wrong content type is requested?
- Are these errors logged?
- What if whole system or some part of the system is unavailable, how would it affect user?
- What if system crash during transaction, how would it recover? What error would it give?’
Invariably, some of these checks will happen during some transaction from GUI and we would end up spending hours in analyzing and finding API issues.
Almost the time spent in proper API testing vs bug fix may be the same, however,
It’s about bug prevention vs bug fix,
It’s about Developer mindset, pressure while developing vs bug fixing under the gun,
It’s about more stable API tests vs brittle GUI tests,
It’s about test within the sprint and gain confidence vs defer GUI tests for subsequent sprints and building Tech Debt
API testing is certainly valuable and worthy investment