Posts

Test automation framework architecture. Part 2.1 - Layered architecture example

Image
Let's consider an example of what Layered (Tiered) architecture may look like for a test automation framework. As a system under test we're going to use this simple and neat "Todo list" application.

First let's do a brief analysis of the application. From my brief exploration I came up with the following check list:

  Area/Group    Things to check    Comments  Todos CRUD1 Create itemBasic scenario, UI-level case1.1 Create item with same nameCase is not valid. There's no validation for that - nothing to check really.1.2 Create item with empty/not valid/malicious nameWhile it would be nice thing to test manually, from test automation stand point I would expect this to be filtered on service or repository level, so either integration or unit test.2 Mark item completeBasic scenario, UI-level case2.1 Un-mark item completeBasic scenario, UI-level case3 Delete itemBasic scenario, UI-level case3.1 Delete item when there're multiple items with the same nameKnowing …

Why I would prefer writing 100 unit test to one UI test

Choosing between writing one UI tests and 100 Unit test I would always prefer second option. Why?

Well, I am actually lame with UI testing, but that not the main reason.

The only thing I worry about is speed. How fast we can change a software? Quality - I don't care about it much cause quality standards are contextual and can change. The only thing that is certain - there're going to be changes and we mostlikely don't know what those changes are about. So there's only one thing that matters - how fast we can adapt to this change.

Internal quality is something that actually makes us able to change fast. UI tests do not contribute to internal quality. UI tests may even make things worse.


That is the reason.


And I invite you to Testcon Moscow to talk about this more - I am going to present some more thoughts on this topic there.

Manual and automated testing confusion

I am getting tired of post about how "automated testing" going to replace "manual testing".

Let me offer you a simple analogy. I have a car that have lots of useful self-check lights, like "check engine", "low gas", "low battery", etc. Automated tests are similar to those lights - if light is on, most likely something is wrong and I need mechanic to look at my car. Does having those self-check lights let me not to visit mechanic for human-driven check yearly? They don't. Fact that self-check light is not on does not mean my car is OK, does it?

Areas covered lots with self-check may require less time to check (as there lower risk they would be broken). However there are lots of areas that is not feasible to cover with self-checks. There're some car parts I am not even aware of, while mechanic knows they weak points and can find issue in a couple of minutes.

It is possible to have only automated checks and not have mechanics to loo…

Perceived quality level of a software may have dropped, but testing is not the answer.

"Modern software is of a lower quality that it was in a past".

Maybe. Perceived quality of software may have decreased, but I don't think that "more testing" is a proper solution.
More testing does not mean more qualityMore testing may find more issues, but not necessarily. Simply spending more time on the activity does not mean results would be better. And somebody needs to fix bugs, test bug fixes.
So we can't tell that more testing means more quality, but certainly means bigger costs.
Software quality != No bugsBugs matter, but that's hardly the only factor to measure quality level. There are a whole bunch of other things that matters: UX, number of features, documentation, price, delivery model, cute logos...
Spending more time on testing may mean spending less time on these things.

Adequate quality levelSo it is obvious that one can't spend 100 years testing every possible case. I think that each product has some Adequate level of qualit…

Agile is not the goal, but means

Image
So we sit in the room and discuss the transformation plan for a company which was working in a waterfall manner for nearly a decade. The final goal is to switch to iterative development with somewhat small iteration (let's say, 3 weeks). The "only" problem we had is that we have 3 weeks if manual regression, and it does not really fit to the plan. We have options:
Long waterfall-like sprint, where first half of the sprint we develop features, second half - test themSeries of short sprints and one "hardening"/"release" sprint dedicated to regression when we finally decide to release things.  There's third obvious option, which is decrease regression time, but no-one seems to have knowledge how it can be done in a next 5 years. So this gets dropped.
"We need to have potentially shippable increment each sprint, "hardening" sprints are not Agile!" - says one man in a room.


Well, that may be true. May be it not as cool as Spotify Eng…

Two different views on Test Automation

Image
I have published several posts with the aim to deliver one message - sometimes it is more efficient (fast and convenient) to change the application under test (make it more testable or eliminate the need for testing at all) then invent or employ complicated test automation techniques to check the same functionality. Even though there were a lot of misunderstanding caused by badly forming those posts, I still think I stroke something deeper.

For instance, this twitter post made me think that we speak two different languages:

That's like asking a pharma company to self-certify that their drugs are safe without any independent approval! #softwaretesting#CIO — Ayush Trivedi (@ayushtrivedi) 4 September 2017
Now it started to seem to me that there there two different views on what test automation is. First, probably prevailing point of view, is that test automation is a part of ages-old traditional QA process, where test automation specialists is just a test specialist using tools to test

The broken concept of a Page object, or Why Developers Should Be Responsible For Test Automation

Preface: I am in the middle of writing a series of posts about test automation frameworks architecture. I am still going to continue that series, even though this posts kind of devaluate the whole test automation framework concept a bit. Sorry for that, just can't stop ranting.

Looking though the internet I spotted a couple of posts where some test automation specialists were talking about "page object" "pattern"/"model", as it was something special they had invented.
Well, I also have something to say about "page object".

http://classicprogrammerpaintings.com/post/147044648074/gentle-technical-discussion-on-irc-channel

"Page object" may be described as pattern that allows us to decouple thing you can do with web page (external interface describing test/business logic) from the real implementation code you will have to use to actually get things done with this bloody web page. If one heard about GoF patterns, she may think about &…