Posts

Perceived quality level of a software may have dropped, but testing is not the answer.

"Modern software is of a lower quality that it was in a past".

Maybe. Perceived quality of software may have decreased, but I don't think that "more testing" is a proper solution.
More testing does not mean more qualityMore testing may find more issues, but not necessarily. Simply spending more time on the activity does not mean results would be better. And somebody needs to fix bugs, test bug fixes.
So we can't tell that more testing means more quality, but certainly means bigger costs.
Software quality != No bugsBugs matter, but that's hardly the only factor to measure quality level. There are a whole bunch of other things that matters: UX, number of features, documentation, price, delivery model, cute logos...
Spending more time on testing may mean spending less time on these things.

Adequate quality levelSo it is obvious that one can't spend 100 years testing every possible case. I think that each product has some Adequate level of qualit…

Agile is not the goal, but means

Image
So we sit in the room and discuss the transformation plan for a company which was working in a waterfall manner for nearly a decade. The final goal is to switch to iterative development with somewhat small iteration (let's say, 3 weeks). The "only" problem we had is that we have 3 weeks if manual regression, and it does not really fit to the plan. We have options:
Long waterfall-like sprint, where first half of the sprint we develop features, second half - test themSeries of short sprints and one "hardening"/"release" sprint dedicated to regression when we finally decide to release things.  There's third obvious option, which is decrease regression time, but no-one seems to have knowledge how it can be done in a next 5 years. So this gets dropped.
"We need to have potentially shippable increment each sprint, "hardening" sprints are not Agile!" - says one man in a room.


Well, that may be true. May be it not as cool as Spotify Eng…

Two different views on Test Automation

Image
I have published several posts with the aim to deliver one message - sometimes it is more efficient (fast and convenient) to change the application under test (make it more testable or eliminate the need for testing at all) then invent or employ complicated test automation techniques to check the same functionality. Even though there were a lot of misunderstanding caused by badly forming those posts, I still think I stroke something deeper.

For instance, this twitter post made me think that we speak two different languages:

That's like asking a pharma company to self-certify that their drugs are safe without any independent approval! #softwaretesting#CIO — Ayush Trivedi (@ayushtrivedi) 4 September 2017
Now it started to seem to me that there there two different views on what test automation is. First, probably prevailing point of view, is that test automation is a part of ages-old traditional QA process, where test automation specialists is just a test specialist using tools to test

The broken concept of a Page object, or Why Developers Should Be Responsible For Test Automation

Preface: I am in the middle of writing a series of posts about test automation frameworks architecture. I am still going to continue that series, even though this posts kind of devaluate the whole test automation framework concept a bit. Sorry for that, just can't stop ranting.

Looking though the internet I spotted a couple of posts where some test automation specialists were talking about "page object" "pattern"/"model", as it was something special they had invented.
Well, I also have something to say about "page object".

http://classicprogrammerpaintings.com/post/147044648074/gentle-technical-discussion-on-irc-channel

"Page object" may be described as pattern that allows us to decouple thing you can do with web page (external interface describing test/business logic) from the real implementation code you will have to use to actually get things done with this bloody web page. If one heard about GoF patterns, she may think about &…

Test automation framework architecture. Part 2 - Layered architecture

Image
Probably the most popular architecture pattern used for test automation frameworks (TAF) is layered architecture. This pattern is so well known that on job interviews for some companies when they ask you about TAF architecture you are supposed to describe this one. If you don't - they think you know nothing about the architecture altogether.

I suggest first reading brilliant description of the pattern at the oreilly web page, cause in this post I am going to describe the pattern in a way it is usually applied to build test automation solution.

Usually, there're three distinct layers, which may have different names, but follow the same logic mostly. Sometimes those layers called test layer, business-layer and core layer, but there're no standard names really. Key rules for layered architecture are the dependency direction (each level depend on the level below) and call direction (no level can call/reference code described in the level above).

The rough structure of such…

Test automation framework architecture. Part 1 - No architecture

Image
Most of the "UI test automation" tutorials I have seen describe the Test Automation Solution where Selenium Web Driver is used directly from test methods and no additional abstraction layer exists. This "architectural pattern" is so ubiquities, that I decided to describe this as well.

I think, we can call this "architectural pattern" as No architecture. The structure of test automation solution created with No architecturepattern presented (in a very rough way) on the picture below:


Such test automation solution usually consists of some amount of test classes, each containing some number of test methods. All orchestration of interaction with System Under Test (SUT) is done either (using JUnit terms) in setUp and tearDown methods, or test methods themelves. Tool, whatever it is - Selenium, RestAssured, Selenide - called directly in test methods.

Such approach is industy well known anti-pattern. However, in some cases, it may be ok to use. There're some…

Test automation framework architecture. Preface.

Sometime ago I wrote this post describing my understanding (at that time) of the architectures used to create test automation solution (framework). While there's some information I still agree with, my understanding has evolved and I want to share this understanding with others.

First, lets probably coin the terminology I use (which is not necessarily would be the right one - feel free to suggest something different)

Test automation framework
It is a framework that allows one to write automated tests. Usually one means the specialized framework, i.e. framework that is specialized for one or several related applications under test. Framework does not include tests themselves

Test automation solution
It isis a complete solution used for test automation. It includes everything needed to perform automated test, including tests themselves. Solution may be based on a framework, however it is not mandatory.

Architecture
May be described as an imaginary model that dictates how code is of the…