Treten Sie ein in Ilari’s Spektakulären Testing Zirkus. Unsere Pforten sind offen für sowohl Tester als auch Entwickler. Die Löwenbändiger und Clowns heissen Sie willkommen zur Show. Nehmen Sie Platz. Die Lichter werden gedämpft. Die Bühne ist frei für Spektakel und Wunder. Geniessen Sie die Show!

Dieser Eintages-Workshop beleuchtet einige der wichtigsten Dimensionen des Software Testings. Jede Sektion beinhaltet eine kurze Einführung, gefolgt von praktischen Übungen oder Spielen. Aktive Teilnahme ist nicht zwingend aber sehr empfohlen. Wir werden folgende Themen durchspielen:

• Was genau ist Software Testing?
  Ja, was ist es denn genau? Was ist der Wert, der unsere Arbeit als Software Tester in das Produkt bringt? Wir reflektieren über unsere Tätigkeit und was genau unser “Produkt” ist. Es gibt dazu ein Spiel in sechs Akten.
• Testing Missionen
  Die Teilnehmer werden etwas testen und dabei herausfinden, wie wichtig es ist, die Mission genau zu klären. Natürlich sind ein paar Fallen eingebaut. Ein wichtiges Instrument des Software Testers ist “Fragen stellen”.
• Checking & Testing
  Es ist ein äusserst wichtiges semantisches Instrument, bei unserer Arbeit zwischen Testen und Checken zu unterscheiden. Wir ergründen die Tiefen und diskutieren über Automatisierung, nicht-technisches Testing und welche Unterscheidung sinnvoll sind.
• “Safety Language”
  Die Worte, die wir wählen, beeinflussen unser Denken. Indem wir Aussagen mit Vorsicht machen und uns immer wieder vergewissern, was wir wissen und was wir vermuten, erschliesst sich uns das Testobjekt in grösserer Tiefe. Wir benützen dazu “Safety Language”. Dieses Modul beinhaltet ein kurzes Spiel und eine Evaluierung davon.
• Modellieren
  Um die Software zu verstehen, anerbietet es sich, darüber ein Modell aufzustellen. Auch hier wird es wieder ein unterhaltsames Spiel geben, bei welchem die Teilnehmer herausfinden, wie effektives Modelling funktioniert.
• Beobachten
  Etwas zu beobachten hat physiologische und psychologische Komponenten. Wie bringt man sich dazu, das Relevante zu beobachten? Dieses Modul beinhaltet eine Einführung in Aufmerksamkeit und in die Regeln, mit welchen unser Gehirn Daten verarbeitet. Natürlich gibt es auch hier eine unterhaltsame Komponente.
• Fokusieren/Defokusieren und die Generierung von Testideen
  Was tun, wenn man stecken bleibt? Nicht mehr weiter weiss mit der Arbeit? Welche Möglichkeiten bestehen, damit man weiter testen kann. Eine Einführung in diverse Heuristiken, die helfen können, das Testing effektiver zu gestalten.

Es wird Rätsel und Gelächter geben und eine auffällige Abwesenheit von Powerpoint Folien. Also, treten Sie ein und lassen Sie sich unterhalten (Sie werden vermutlich sogar etwas Brauchbares lernen)

Wir haben fünf Sinne, mit denen wir unsere Umgebung beobachten können. Tasten, Schmecken, Riechen, Hören und Sehen. Diese fünf Sinne sind die Eingangspforten in unser Hirn. Weil die Menge der auf uns auftreffenden Information zu gross ist, haben wir starke Filter installiert. Manchmal verpassen wir dadurch das Offensichtliche. Nun, wie können Sie ihre Beobachtungsgabe verbessern? Was heisst es, ein guter Beobachter zu sein? Können Sie diese Fähigkeit trainieren?

Sobald Sie eine relevante Beobachtung gemacht haben, möchten Sie diese vielleicht beschreiben. Das geschieht üblicherweise mündlich oder schriftlich. Da natürliche Sprache anfällig ist für Fehler und Missdeutungen, muss man da grosse Vorsicht walten lassen. Nun, wie können Sie sich bei Ihren Beschreibungen verbessern? Was ist eine gute Beschreibung? Können Sie das trainieren?

Da beides, Beobachtung und Beschreibung, integrale Bestandteile von fast jeder Testing Aktivität sind, haben Sie hier die Gelegenheit, sich in diese wichtigen Aspekte des Software Testings zu vertiefen. Für Software Tester gibt es eine breite Palette von Fähigkeiten, die Sie beherrschen müssen, um Ihren Job gut zu machen. Das Studium von theoretischen Modellen und praktischen Anwendungen können Ihnen helfen, um besser zu werden.

Dieser Halbtages-Workshop wird eine Mischung von theoretischer Einführung  und praktischen Übungen in den verschiedenen Domänen sein. Wir werden die Psychologie der Aufmerksamkeit behandeln, uns über linguistische Modelle unterhalten und gemeinsam Übungen durchführen. Ich plane auch, Peer Reviews über die Übungen durchzuführen und es wird eine lebhafte Diskussion unter den Teilnehmern geben.

Stellen Sie sich vor, Ihr Team sitzt nicht gemütlich zusammen im gleichen Raum oder Gebäude, sondern verteilt über verschiedene Länder. Wie halten Sie die Kommunikation aufrecht? Wie gehen Sie mit einer solchen Situation um als Manager? Wie stellen Sie sicher, dass wichtige Information an die richtigen Leute fliesst?

Ilari Henrik Aegerter war in der Vergangenheit der Manager von Productivity & Test Engineering beim weltweit grössten Marktplatz eBay und hatte dort ein Team, welches verteilt war auf Zürich, Berlin, Paris, London und Moskau. Dieser Workshop ist eine Verbindung von einem Erfahrungsreport über die Schwierigkeiten und auch eine “Experiential Session” über die Kommunikation mit Einschränkungen.

Die Teilnehmer werden eine kurze Übersicht über die Erfahrungen von Ilari bekommen und dann in eine Simulation gebracht, in der sie in einem Mini-Projekt kollaborieren müssen, um es erfolgreich durchzuführen. Natürlich wird es Störungen geben, welche das Unterfangen noch etwas schwieriger machen werden.

Körper und Geist sind unseparierbare Einheiten und die alten Griechen wussten dies bereits. Diese Session startet mit einer Einführung in die Peripatetische Schule des Denkens. Die Teilnehmer werden eine Einsicht erlangen, warum körperliche Bewegung und Denken gegenseitig Unterstützende Tätigkeiten sind.

Dann gehen wir auf einen Spaziergang.

Wir werden gemütlich zu Fuss gehen und sehen, wohin uns unser Weg führt. Die ganze Session wird draussen stattfinden, und wir spazieren in einer gemütlichen Geschwindigkeit draussen in der Natur. Es wird sowohl eine Übung in Lösungsfindung sein, als auch eine “Experiential Session”. Nicht viel ist bekannt darüber, wohin uns diese Session führt. Vielleicht treffen sich die alten Griechen auch mit 21. Jahrhundert Geekery und es könnte eine Aufzeichnung über Teile der Session geben.

Kommen Sie ausgestattet mit ihren Testing Problemen, Gedanken und Rätseln und alle Teilnehmer werden fröhlich einstimmen in gemeinsames Lösungsfinden. Sie werde nach dieser unvergesslichen Session sowohl glücklicher als auch weiser sein.

Four months ago my first proposal for a conference was accepted. It’s been an interesting journey in unknown territory. The conference is in two days.

Lately on Twitter there’s a lot of talk on the Speak Easy program and on gender inequality in tech conference speakers. To this I’d like to pass on some points to others doing their first conference presentation. “Preparation is your friend”, a friend told me. But what does that really mean? Here is a preparation heuristic for you:

• Throw away the ideas that don’t work for you. Don’t succumb to the sunk cost fallacy. If you can’t get a hang on the topic or the angle doesn’t work for you, just let it go. Start over. Go with whatever you feel is interesting.

• Accept feedback and criticism. People are trying to help you, so listen to their feedback and points. They are probably right.

• Pause. Take a break. Start preparing early enough so you have time to leave your preparation alone for days or weeks. A critical distance will make it better.

• Isolate yourself. Stop writing and talking to other people. Start talking to yourself, the mirror or the wall. You think you are going to “have a conversation” or a dialogue with the conference audience, but really it’s a monologue until the talk is over and discussion begins. So you need to practice talking without expecting or relying on feedback.

• Research and review. Get to know your topic in depth. Who has written and said things about it? What theories are you using and how does that affect your talk? Are your sources all Wikipedia, blogs or academic papers?

• Flow. Every time you practice and for every week that goes by, the focus and the presentation will change a little bit. Or a lot. Let it change because every audience and every context is different, so you need to be able to adapt.

• Embrace creativity. Write everything down and then throw it away. I have no text in my presentation slides, but that doesn’t mean I haven’t written anything. I have written what could be a short novel, because I am most creative and innovative when I write. If your creative mode isn’t writing, then find out what it is and start doing that.

• Visuals. Pictures, graphics and layout take time to find or create. I made simple drawings of my topics main elements and a friend (and cartoonist) drew the whole slide deck. I paid her with a good bottle of wine. So find someone who is good at layout design or drawing.

• Environment and resources. Look around you. I’ve used my friends and surroundings as much as possible. We all know people that are specialized in some area: My sister is a designer (nice layout), my big sister coaches people in public speaking (yay!), my step mother is a consultant in organizational change (organizational theory), my husband is a developer (general discussions on test/dev relationships) and I have friends that are sociologists and philosophers (just smart people). And last but not least: my colleagues and friends at House of Test (test discussions)

• Rehearse your talk. Find someone to practice with. When you think you know your presentation, find a colleague or a friend who will listen. The first 2-3 times are horrid but that’s okay. Now you can improve the presentation.

The last bullet didn’t fit in the catchy name of the heuristic. But here it is anyway:

• Panic. When you panic and decide to quit because the mere thought of presenting to others freaks you out: Call a friend. Get a perspective on things and accept your fear as a normal feeling for any human being.

Good luck and I am looking forward to seeing you!

B.

In a previous assignment we were introducing a continuous integration type of process to allow us to deliver code faster and in smaller parts to the main codebase. The idea was that each code delivery should be extensively checked by our test-automation resulting in the main codebase being in an always releasable state. The theory was that we should always be able to pick the latest build and be able to use it and release it. However my manager at the time, Mattias Göransson, made a very insightful comment about what always releasable actually meant in our context:

“We might have an always releasable state of our main code base but that does not mean it is release worthy”.

During the project we reached a stage where we could always send out the latest executable and we could with confidence lean on our automation to say that the product could be installed and used (to some degree). But to make sure it was actually release worthy, that it solved the intended problems for our customers, we needed eyes-on testing. For example; in this particular context it was important to have a user friendly interface and our automation could not make a judgment and provide us with an answer if our product lived up to that requirement or not. What the automation could tell us was that the product was technically working (to some degree).
The distinction between something that can technically be used and being worthy to release gave me a much better language to use when discussing and planning what testing was needed in what part of the process. I believe the distinction also helped us keep focus on our the testing by better understanding in the organization about what values the automation could and could not provide.

In the beginning of my career I was assigned to test a part of a very complex automotive system. The project was behind schedule and a fixed release date was closing fast. In the automotive industry there are fixed release dates and patching is very expensive since it requires the car to be brought to a workshop. The system developed was much more complex than the ones currently in production; unfortunately it was in terrible condition. The system crashed on a regular basis, features were missing or not working and there were a lot of re-design and requirement updates.

My approach was to divide the system on the different GUI views. I listed these in Excel. Then I added the subparts of the views into my Excel spreadsheet. By reading the different specifications and talking to the system owners I came up with a lot of things to test and added them to the items. The reporting was done by color marking the items based on my opinion.

• Green: No problem found
• Yellow: Small issue
• Red: Major issue or many small issues (or that feature was untestable because of other defects)
• Grey: Not tested

I reported issues with the system or specifications into a bug tracking tool. The severities of these issues were discussed with system owners and the project manager. If needed, I updated the colors of my excel items to reflect any new information regarding severity. I also added references to the bug reports next to the colored items. When they were retested ok I crossed them out but kept them in my spreadsheet for regression testing. Both system owner and project manager were very satisfied with my work.

The next step

Eager to learn more about testing I signed up for a course. At this point an international known course like ISTQB Foundation sounded as a good idea. It wasn’t… at least not for me…

The course was packed with text-heavy PowerPoint slides. There were almost no discussion around concepts and no actual practice. Since the goal of the training was to pass an exam there was a big focus on memorizing answers to questions. When I challenged some of the “facts” in the training material the teacher told me to just remember it and take it with a grain of salt.

The importance of salt

Unfortunately, as a rookie in software testing I did not bring enough salt. There was very little discussion about in which situation use certain approach would be recommendable. Context was not considered at all. When I got back to my assignment I tried to implement some of the stuff we had learned at the training.

I started to make detailed test case specifications and mapping these to requirements. I also spent time thinking on how to measure requirement coverage. After a while I realized that even if I did test all the requirements they would have changed when I was done. And updating the test specifications would take a lot of the time I could use for doing testing. And the requirements were nowhere near a full description of the system. So I stopped, and went back to my former way of testing and reporting. All good? Unfortunately not…

Everyone’s happy, except me

Although the project manager, system owners and suppliers thought I was doing a great job, and there was really good progress, I couldn’t lose the feeling that I was cheating. I did not do it the “right” way… How could they be happy about my work? Was it because they didn’t know anything about testing? For a couple of months I got nothing but praise for my excellent work but still felt bad. Then I went on a seminar called “Beyond Test Cases” by James Bach… He made me realize that there was no right way of doing testing and that testing is not about presenting a number of failed or passed test cases. Nor is it about producing requirement coverage metrics. It’s about questioning the product to reveal information about it which is useful for the stakeholders. And the way you approach this challenging task is different in every situation. When the assignment ended and the customer made a performance review of me I got 4,7/5 with the motivation ” I don’t want to give you all 5’s because you might get cocky”. That was great, but the best part was that I finally felt that I had done a great job and that I deserved the praise.

Thank you James for showing me a better path!

Two recent interviews with Henrik Andersson have been published.

“To me, punk rock is the freedom to create, freedom to be successful, freedom to not be successful, freedom to be who you are. It’s freedom.”

Patti Smith

First of all I must say that I’m sad that such an infected debate between various factions continue to rage in the testing community. It seems to entrench the various views and making it harder for to build bridges between different points of views.

I also think it’s important to point out that the contents of ISO 29119 is not all bad per se. I have read several comments in discussions where people find support in the material presented in ISO 29119 and rightly claim that we cannot all be thought leaders in the test community and come up with new ideas and thus, finding inspiration and guidelines in the work from other people is essential to become a better tester. the ISO 29119 does contain material and information that some testers will find valuable in some context, no doubt about it.

So far, so good, but I nonetheless signed the petition against ISO 29119. Why did I do that? I’m actually a latecomer to the petition and I had the opportunity to read the response to the petition, published here: http://www.softwaretestingstandard.org/29119petitionresponse.php before signing up. In there, I think Dr Stuart Reid does post some valid responses to some of the objections put forward in the petition and in conjunction with the petition; for example I can live with the material not being free and I’m certain that it has been applied to several organizations while the ISO 29119 was in draft state – probably also successfully.

However, then some details in the response that bugged me started to stand out. The first thing is that the response is posted on a site that does not seem to allow any comments or open discussion. Neither could I find any direct link to the response on discussion forums on e.g. LinkedIn. This does counter the argument that everyone is invited to discuss the material in an open and transparent way.

I’ll start a bit out of order, with the following passage from the response:

According to ISO, standards are “Guideline documentation that reflects agreements on products, practices, or operations by nationally or internationally recognized industrial, professional, trade associations or governmental bodies”.

They are guideline documents therefore they are not compulsory unless mandated by an individual or an organization…

Fair, but why not call them ISO guidelines, then? One of the advantages the Dr Reid mentions with the ISO 29119 is that is builds a common vocabulary and that it should not re-invent the wheel. I think standard English already has a perfectly functioning definition of “standard” – and it’s not the same as “guideline”. My fear, and I think many with me, is that many stakeholders in companies will not be able to make this very important distinction. They’ll just read “standard” and read their own interpretation into that.

Dr Reid also writes [emphasis added by me]:

There are some useful IEEE testing standards (e.g. IEEE 829, IEEE 1028) and national standards (e.g. BS 7925-1/-2) but there were large gaps in the standards relating to software testing, such as organizational-level testing, test management and non-functional testing, where no useful standards existed at all. This means that consumers of software testing services (and testers themselves) had no single source of information on good testing practice.

I believe that the single most important factor for the evolution/revolution in testing has been the fact that there has been a multitude of different sources of good information. Many of them conflicting with each other, but full of ideas that help me get different points of views which I can then bring back with me to my projects and apply as appropriate. Admittedly, some sources can be hard to find, and if there were a single portal with all testing related information on the web, then it would probably be helpful, but this is not what the ISO is aiming for. As far as I can tell ISO wants to create a single source of ISO controlled information, which again makes it looks much more like a standard than a guideline.

Moving on to the question about agile and exploratory testing. From Dr. Reid:

The scope of this initial work (ISO/IEC/IEEE 29119 parts 1, 2, 3 & 4) was largely defined by the existing IEEE and BSI standards (which they would replace), although it was clear from the onset that a completely new ‘Test Processes’ standard would be required, in particular to ensure that agile life cycles and exploratory testing were considered, as well as more traditional approaches to software projects and to testing.

Actually, the agile life cycles and exploratory testing are not my main concern; they are already out there and more or less widely accepted in the testing community. What worries me more is what we might miss out on in the future. If ISO 29119 requires “a completely new ‘Test Process'” to accommodate agile and ET, then that also means that agile and ET came about despite material from IEEE, BSI, ISO, ISTQB, etc, not because of it. How does ISO ensure that 29119 does not curtail the next advancement in software testing – something that would require the ISO 29119 test process to be completely rewritten?

The concern about future needs for the test process (or other parts of the ISO 29119) ties into the tailoring of the ISO 29119. It has been stated several places that an organization can basically tailor it any way they like, to fit any context, but if that is so, then it seems not so helpful. It’s like making scaffolding out of play-doh – I can form it into whatever I like, but i won’t really support me much. If, on the other hand, the scaffolding is more rigid, so that it supports me, then it will also be less possible to tailor to anything. Which one is the ISO 29119 – steel or play-doh?

Also, even if ET is covered in the ISO 29119, it might not bee on equal footing. I think it’s telling the Dr. Reid chose the following paragraph in his response to the petition:

“When deciding whether to use scripted testing, unscripted testing or a hybrid of both, the primary consideration is the risk profile of the test item. For example, a hybrid practice might use scripted testing to test high risk test items and unscripted testing to test low risk test items on the same project.”

And stepping back a bit to “standards” and “best practices”. Dr Reid writes [emphasis added by me]:

• Definition of good practice in the testing industry – a guideline for testing professionals, a benchmark for those buying testing services and a basis for future improvements to testing practices. Note that we do not claim that these standards define ‘best practice’, which will change based on the specific context.
• A baseline for the testing discipline – for instance, the standard on test techniques and measures provides an ideal baseline for comparison of the effectiveness of test design techniques (for instance, by academics performing research) and a means of ensuring consistency of test coverage measurement (which is useful for tool developers).

Isn’t “ideal” synonymous to “best”? Maybe splitting hairs a bit, but I find it interesting that the word “ideal” is used in the very next paragraph after stating that ISO 29119 does not define “best practices”.

As stated before, I’ve not really been involved in this debate before and I have not tried to get involved in the ISO work with 29119, but I can’t help having a few reflections on the following from Dr. Reid:

However, as a Working Group (WG) we can only gain consensus when those with substantial objections raise them via the ISO/IEC or IEEE processes. The petition talks of sustained opposition. A petition initiated a year after the publication of the first three standards (after over 6 years’ development) represents input to the standards after the fact and inputs can now only be included in future maintenance versions of the standards as they evolve

1. The following quote from “The Hitchhiker’s Guide to the Galaxy” comes to mind:
   • “What do you mean you’ve never been to Alpha Centauri? Oh, for heaven’s sake, mankind! It’s only four light years away, you know! I’m sorry, but if you can’t be bothered to take an interest in local affairs, that’s your own lookout! Energize the demolition beam!”
2. Does it really matter when feedback comes? It does worry me somewhat that ISO basically says “It doesn’t matter whether we have published something good or bad – since it has been published, then we must stick to our predefined change request process”.

After all of this, I must still say that I kind of wished that I could endorse ISO 29119; a lot of talented and experienced people have put a lot of effort and thoughts into this and it does contain some valuable information that I very well might end up using in some project some day.

I would not have signed the petition if I thought ISO really sees the 29119 as just another asset in the great library of thoughts and ideas out there in the testing community. However, the more I read the discussions around ISO 29119 and the responses from proponents like Dr. Reid, the more I feel that there is a disconnect between what e.g. Dr. Reid writes about standards, guidelines, best practices, tailoring, etc. and what he actually means. For something that has the potential impact like the ISO 29119, I believe the risk is just too great and it would need more testing before release – that’s why I have signed the petition to stop ISO 29119.

I came across this article http://konsultbolag1.se/tour-testing (in Swedish) about how Exploratory Test (ET) and Tour testing can be a good complement to scripted/manual testing. It covers problems and benefits associated to the approach.

When I first skimmed through this article I thought “there are some things I don’t really agree with”. After reading it more thoroughly it more became “there are a lot of things I don’t agree with at all”.
So since there is no way to respond to the article directly I will do it here.

First we probably need to talk about definitions.

I will assume that when we talk manual tests we mean tests written down in detailed step by step instructions then executed by a human tester with an expected result, sometimes also called scripted testing or checking (more about checking can be found here: http://www.satisfice.com/blog/archives/856). Which by the way is a great way to kill a tester’s spirit.

My interpretation of ET is in line with what James Bach says here (http://www.satisfice.com/articles/what_is_et.shtml).

When I talk about session based testing (a way to manage ET) I mean Session Based Test Management (http://www.satisfice.com/sbtm/).

There can of course be other interpretations for ET and Session Based but this is the original source.

Ok so let’s get on with the comments (Swedish quote from the article then Translated to English).

“Under tiden dokumenteras testförfarandet. Detta för att det ska gå att återupprepa scenariot vid avvikelser, följar upp testet samt återupprepa testet senare.”
“Meanwhile the test procedure is documented. This is so that the scenario can be reproduced on anomalies, do follow up on the test and to reproduce the test later”

• How much you should document depends on a number of things, e.g. what kind of product you are testing, how long time you have, what the stakeholders need and so on. The documentation can be in rigid step by step (for e.g. regulatory or law conformance), session notes, screen recordings or simply kept in the testers mind. You should always think about what the purpose of the documentation is and if the worth of it outweighs the cost of documenting.

• Although if you file a bug then it probably is important to have clear step by step instructions so developers, the future you and possible other stakeholders can understand what you did, how you did it and the impact.

• If you during your ET testing find a scenario you think is worth repeating you can of course note down the steps and possibly automate it but one of the benefits is that you often don’t repeat a test since there is often more value in doing a new test (can be rather similar but not exact the same) rather than repeating it.

“Exploratory testing eller Utforskande tester (UT) kan vara ett bra komplement till din testning och dessutom kan denna typ av tester vara en mycket effektiv metod för att hitta de där retsamma buggarna som annars brukar slinka igenom – om dessa tester görs på rätt sätt vill säga! Det är en allmän uppfattning att UT hittar fler kritiska buggar än traditionell testfallsbaserad testning eftersom testaren kan reagera och undersöka mjukvaran utifrån dess beteende istället för att strikt följa ett testfall. Frågan är bara hur man ska göra för att lyckas med sina utforskande tester och undvika riskerna?”
“Exploratory test can be a good complement to your testing and also this type of test can be a very efficient method for finding those pesky bugs that otherwise slip through – if these tests are done in the right way that is! It is general belief that ET finds more critical bugs than traditional test case based testing since the tester can act and investigate the software based on its behavior instead of strictly following a test case. Question is what you should do to succeed with your exploratory tests and avoid the risks?”

• First the article talks about that ET can be a good complement and then that it finds more critical bugs, in that case I would say it should be more than a complement. Although going by James Bach “To the extent that the next test we do is influenced by the result of the last test we did, we are doing exploratory testing.” it is hard to understand what the other testing (that ET should be a complement to) is.

“Det ska sedan en lång tid tillbaka sitta i ryggmärgen hos testaren att kunna använda sig av alla olika teststrategier och tekniker för att garantera mjukvarans kvalitet.”
“It should since a long time back be second nature for the tester to use different test strategies and techniques to guarantee the quality of the software”

• This needs to be said, we can never guarantee the quality of the software, we can however give information about what we have seen/not seen and our impression about the software.

• And yes, it takes time to become a great tester. As with everything else you need to practice and learn continuously to become better and better but this is not only for ET but for all testing.

“Jag tycker att man med en metod som kallas Tour testing slår hål på den myten till viss del eftersom testerna här kan anpassas till testarens erfarenhet och systemkännedom.”
“I think that you with a method called Tour testing can puncture the myth to a certain degree since these tests can be adapted to the testers experience and system knowledge”

• You don’t need Tour testing to adapt the testing to the tester’s experience and system knowledge, you can do that with ET and the right guidance and education. Tour testing is just one way to approach the product in different ways. You can also use personas where you take on the role of a fictive person. Maybe it is someone that is impatient, someone that is lazy and use all shortcuts or someone who want to exploit the program (think security). You can also use the different hats technique and so on.

• Tour testing isn’t a replacement for testing knowledge. Tour testing is a technique and hence you can with practice become better at it. It is not something that magically makes testing simple.

“Min uppfattning är att UT framförallt kräver styrning, mål och dokumentation för att vara effektiv och inte erfarenhet.”
“My view is that ET above all requires control, goal and documentation to be efficient and not experience.”

• First off I would say that then it is not ET. Of course we might want some kind of control (e.g. by using SBTM) but if control and a clear goal is the important parts then it sound more like checking. And that control should be more important than experience seems to me as a lack of trust in the testers. I would translate it to “I’m the only one who know what to test so do exactly as I say”. This is for me as a tester a horrible place to work in. As a test lead, trust in your team is very important. Of course trust needs to be earned and new testers might need some guidance in the beginning before they have shown that they can be trusted fully. But to rank control above experience and skills is to me just plain wrong.

“För att belysa riskerna med UT kan man dra en parallell mellan testaren och en turist som ska åka till London för första gången utan att göra någon som helst research om staden innan. För att upptäcka staden vandrar turisten planlöst omkring på gatorna i hopp om att stöta på roliga och intressanta saker. Troligtvis kommer turisten att stöta på en och annan intressant sak, men utan någon förkunskap blir det svårt att förstå vad det är och betydelsen av upptäckterna som gjorts. Det här gör inte upplevelsen speciellt rik eller kvalitativ och framförallt är risken stor att turisten missar massa intressanta sevärdheter eftersom tiden rinner iväg när denne planlöst går omkring. Turisten vet inte heller hur stor staden är så det blir svårt att veta hur mycket som finns att utforska. Om detta översätts till test av mjukvara så förstår vi snart att stora risker uppstår. Visst kommer turisten ändå uppleva delar av London och troligt är att denne hittar några bakgator och genuina ställen som man annars kanske inte hade stött på.”
“To highlight the risks with ET you can make a parallel between the tester and a tourist that is going to London for the first time without doing any research at all about the city. In order to discover the city the tourist aimlessly wanders around the streets hopeing to bump into fun and interesting things. The tourist will likely bump into one or another interesting thing, but without any pre-knowledge it will be hard to understand what it is and the meaning of the discoveries made. This doesn’t make the experience very rich or qualitative and above all the risk is large great that the tourist misses a lot of interesting sights since the time flies when he aimlessly wanders. The tourist also doesn’t know how big the city is so it will hard to know how much there is to explore. If this is translated to test of the software you will quickly understand that big risks appears. Sure the tourist will still experience part of London and probably find some back alleys and genuine places that you otherwise might not have bumped into.”

• So many things to say about this metaphor. First, it is very few testers that when they start testing have zero knowledge of the product they are working on and have no idea how it should behave. Even when non testers are brought in to test the product they usually have used some similar product that can act as an oracle (a reference to tell you if the product misbehaves). E.g. if it is a Windows program there are some parts that all Windows programs share (like how to close it, normal shortcuts …) and if it is an app there are other apps out there to compare to. Also feelings is a good oracle, if you feel confused, angry, bored, intrigued or excited can tell you a lot of the product.

• A tourist still knows what preferences she have and what she likes. Is this pretty, was this far from the hotel, did it taste good and so on. For all those questions we have a gut feeling answer. She can usually understand some signs (at least those with images) to help her navigate and if she can’t that is also good information (think usability). She can find interesting areas that she doesn’t have time to investigate now but if she returns she can spend some time on it (future tests). She will understand if she ends up in a dead end and need to circle back. And day by day she will learn more and more about the product.

• You can get a lot of information from someone that have no knowledge about the product, everything they find strange is a potential issue from a usability/help documentation view. And if you are a tester hired to test this product you will have more information than Mr. random on the street. Pairing a new tester or Mr. random with an experienced tester is a great way to learn about these issues that the more senior tester maybe have grown blind to after testing the product for a long time.

• For this to be a real problem we need change the human tourist to an alien that has landed in London and this is the first time seeing earth. It will not know if the air is breathable, won’t understand any signs at all, maybe won’t realize if it hit a wall and won’t understand if it is humans or cars that is the dominating species. Although this would translate to a brain dead tester and not a new tester.

• Risk of missing interesting attractions is always the problem. But we can of course do some risk analysis or similar to try and find the areas we should visit, but also experienced testers need to do that. When a risk analysis have been done you can easily assign a new tester to one of those areas instead of letting her loose on the whole product (if that isn’t what you want to do).

• The risk as I see it here is not with the tester doing the ET session but rather the Test Lead (or team) planning the sessions and guiding/teaching the new tester.

• And by the way it is the back alleys that can be the interesting parts. The interesting tourist attractions (Big Ben, London Eye) can probably be covered by automated checks since we already know where they are, how they look and some have been there for a couple of hundreds or years.

“Riskerna med manuella testfallsbaserade tester kan istället likställas med en väl förberedd turist som innan resan till London läser alla möjliga guideböcker och antecknar vad man vill se och göra under sin vistelse. Turisten kollar upp kartan, valuta, restauranger, transportmedel, väder, evenemang, ja allt man kan tänka sig”
“The risks with manual test case based tests can instead be equaled with a well prepared tourist which before the trip to London reads all different kinds of guide books and notes what to see and do during the stay. The tourist checks the map, currency, restaurants, transportation, weather, events, everything you can imagine”

• This is a really good thing to do as a tester regardless if you are to create automated checks or run ET. You check old bugs, specifications, similar products, help pages, older versions of the product, expectations and so on to get better understanding what to expect (get more oracles). This is part of the daily work to learn more about the product so you can make better calls what and where to test.

• The weather part here is a good example since it as with many things in a software project is impossible to predict. A human running a ET session is probably much better in noticing these differences than an automated check or when running a predefined script (since people tend to become zombies when following detailed instructions).

“En stor risk med UT är att fokus endast hamnar på positiva, funktionella tester då testarna inte får någon styrning. Man utgår ifrån det man vet, vilket brukar vara krav eller sina egna erfarenheter om hur mjukvaran ska fungera och vad den ska klara av. Detta är vanligt bland juniora och ovana testare och dessa positiva tester kan i mångt och mycket likställas med ”checking”. Genom Tour testing kan man på ett tydligare sätt få juniora testare att hitta infallsvinklar och angreppssätt samtidigt som man hela tiden bygger på kompetensen och utökar verktygslådan.”
“A big risk with ET is that focus only is put on positive, functional tests since the tester is not controlled. You start with what you know, which usually is requirements or your own experience about how the software should work and what it can do. This is common among junior and novel testers and these positive tests can basically be equalled with “checking”. With Tour testing you can in a clearer way get junior testers to find angle of approach and ways to attack at the same time as you build competence and expand the toolbox.”

• Why should ET lead to a tendency to only test positive tests? If this is the mindset of the tester how would it differ when creating automated checks?

• Why shouldn’t we be able to control what we test with ET? It is just a matter of deciding and communicating what parts and with what focus we want to have. This is where the debrief and charter part of SBTM comes into play. In debrief the test lead have a chance to hear what has been tested and can feedback and guide the tester if she is off track by changing the charters.
  There are many ways to handle this and many test conference sessions are about this. You can e.g. use mind maps to plan, heuristics (a fallible shortcut to a solution of a problem) to help remember what different things to test and dashboards to visualize what has and are to be tested.

• Again here is very little trust in the testers and feels more like an “I know best” mentality. Is this really the case with all junior testers? I would say it depends on the person, how they are educated and guided and not number of years in the testing trade.

• I don’t agree that if we only run positive tests we can compare it to checking. Checking can be done on negative tests as well (here I assume that negative tests are some test where we expect the product to fail) and if you run ET it is not that you follow predefined steps, you go where it is important to go (depending on current information and priorities) even if it is only positive flows. This is the core of ET and something a check can’t do. And as pointed out, a junior tester doesn’t know that much about the product so if she wants to try something she will not know if it is a positive or negative test.

“En väldigt stor risk med UT som växer med komplexiteten på mjukvaran är att man får en relativt låg test-täckning (coverage) mot vad man borde kunna få med UT. Eftersom människan i sin natur är snäll och gärna vill hitta vägar runt problem så finns risk att testare går runt komplexa delar med knepiga funktioner och testar i de områden/programflöden där de känner sig hemma. Det kan också vara av ren vana som i att ”Jag sparar alltid mina värden genom att använda Arkiv-menyn” istället för att använda de snabbkommandon eller andra mekanismer som finns för spara i applikationen.”
“A very big risk with ET which increase with the complexity of the software is that you get a relatively low test-coverage compared to what you should get with ET. Since man is by nature good and rather find ways around the problem there is a risk that tester avoids complex parts and tricky functions and tests the areas/program flows where they feel at home. It can also be due to habit like “I always save my values trough Archive menu” instead of using the shortcuts or other mechanisms that exists to save the application.”

• Again with the condescending tone towards testers. People I have worked with that I think of as good testers have never had this approach. This to me is comparable with not reporting a critical bug because it is too much “paperwork”. People that think like this is not testers (at least not good ones). If you have these in your organization you have either made a mistake in your hiring procedure, you have not guided them properly or you have killed their motivation.

• The issue with defaults and bias is on the other hand a real problem. Although in my mind it is maybe more related to testers that have worked with the product for a long while and have gotten some habits, not for new hungry testers. But it is a real problem and something to be aware of. Rotating testers between different areas can be a good way to minimize it.

“Det kan vara väldigt lätt att grotta ned sig i en viss del av en applikation om man hittar något som verkar avvikande eller helt enkelt drar uppmärksamhet till sig. När man kombinerar sessionsbaserad testning med UT och 80 % av tiden ägnas till denna ”intressanta del” så säger det sig själv att resterande delar inte kommer få lika mycket uppmärksamhet. Hade testledaren avsikten att applikationen skulle testas igenom på en mer övergripande nivå eller med annat fokus kan testet bli missvisande om inte tiden rapporteras på en väldigt låg detaljnivå.”
“It can be very easy to dig down in a certain part of the application if you find something that seems divergent or simply draws attention towards itself. When you combine session based testing with ET and 80% of the time is spent on this “interesting part” it is self explanatory that the rest of the parts won’t get as much attention. If the test lead had the intention that the application should be tested in a more general level or with a different focus the test can be misleading of the time isn’t reported on a very low detail level.”

• I don’t understand this part. One of the reasons you run Session Based with time boxed sessions is to have a better view of where you should be spending your time and where you actually are spending your time. So what Session Based can do for you is to catch this problem when it occurs since it will be caught in the debriefs and hence give you a tool to deal with it. Without Session Based you might end up with spending too much time on one area but if you have a good test lead it shouldn’t happen.

• If the test lead had the intention to test more broadly, the test lead should have said so. I guess the test lead have regular interactions with the testers and don’t just hand them the test instructions on stone tablets once a month.

“Som en lösning på problematiken med riskerna inom UT och manuella testfallsbaserade tester finns metoden Tour testing som utnyttjar fördelarna i de båda metoderna. Med UT hittas generellt fler kritiska avvikelser och lite udda saker, medan man i de manuella testerna minimerar riskerna att missa viktiga områden och centrala funktioner.”
“As a solution to the problems with risk within ET and manual test case based tests there is the method Tour testing which use the benefits in both methods. With ET more critical anomalies and some odd things are generally found, while you in the manual tests minimize the risks that you miss important areas and central functions.”

• How does the manual test minimize the risks? The information and risk analysis you are basing your checks on should also be used when doing ET to decide where and what to test.

The Tour approach is one good way but don’t forget that this is only one way. There are many others, and different approach suits some products or people better and worse.

Drawing the system can be a really good exercise to find unclarities, questions and so on. Although don’t forget that it is often interesting to find out what happens between the districts/areas since that might also be between different responsible teams.

And just to make it clear, I have nothing against Tour testing. I agree with many of the points made in this article (especially in the benefits and conclusion part) that it is a good heuristic you can use to try and avoid missing parts of the product and that it can be a good way to visualize different areas of the product and create some interest in what the testers do. But it is also important to note that Tour testing is not something that is easy and anyone can do well. This also requires that the tester practice and constantly improves in order to become good at it.

Yesterday I ran in to a bit of a debugger bashing with a couple of people disowning the debugger with statements like “I noticed as soon as my students got access to a debugger [rather than just ‘print’ statements], their design very quickly deteriorated” and “I often feel that it’s much faster to reason around potential causes for bugs than to search for them in a debugger”. Now, these statements are undoubtedly true, but I am a huge fan of the debugger and will use it extensively in just about any programming task that I do. Admittedly, I do no longer do programing as my main profession, but I still code small tool in various languages, so this dialog got me thinking on why I find it useful.

I tried to think about how I use the debugger when I’m coding and then in struck me; I use it as a tool for unscripted unit testing. I have the feeling that many people, when they think of unit testing, they think of small pieces of code that check some other code. I also do this and find if very useful, especially as a way to constantly refactor and improve my design. However, these small snippets of code can be seen as test script and although there is value in these test scripts, there is much more to testing. Also, the code snippets do not really test the code, just check it – I want to test it.

What I do when I code is after every so few lines of new code, I have a micro test-session where I explore the new code where I can really test it beyond the (simple) checks of my scripted unit tests. For me, this adds invaluable information about the code that I have just implemented, that I feel I could not get otherwise.

As with most tools, their value depend on how you use them and with the debugger, it might not be a good tool for software design or for investigating bugs, but as a tool for unscripted unit testing, I find it to be great.

First off I want to thank everyone who attended Let’s Test and made it an awesome experience. This was my first test conference and all of you helped to make it a fantastic one.

There were so many great talks to attend that choosing between them was an impossible task, or at least a very hard one.

One thing all presenters had in common was that the information they were sharing was mixed with a great sense of humor. This made time fly and all the sessions felt way too short, even the full day with James Bach. As a colleague said when we attended a presentation James had in Malmö. “It is like a test related stand up”.

Below are my sum-up of the sessions I attended. Have a look at http://lets-test.com for the full program and the slides (coming soon). And I apologize if any quotes are wrong.

James Bach opened the conference with his keynote “How do I know I’m Context driven?”. He talked about the problem of shallow agreement where we might think we are talking about the same thing but really we are not. He also pointed out that context driven can mean different things such as.

• A paradigm – The model by which I understand, experience value, explain and categorize good testing.
• A community – People I influence and am influenced by who expose the principles of context driven testing.
• An approach – A heuristic I apply to my projects.

In depth look at the art of Test Reporting – James Bach

I was one of the early registers so I got a seat at James full day tutorial. The day was composed of presentation and exercises where one exercise was to answer the unicorn question “‘Between 1-100, how completely did you test?”. Amazing how many answers you can get to that question. Here are some:

• 1
• 80
• Silence
• 100, I tested all 4 minutes
• 0, I know 0 about how much I have covered.

He also talked about the 3 levels of test reporting:

• 1. What is the status of the product? – Is the product any good?
• 2. What did you do to get to that fact? – How do you know?
• 3. Was that testing good? – Should I be pleased with your work?
• 3+. What is the quality of the test report? – Is the report good enough?

These levels are good to keep in mind since it will help you create a thorough report fast.

There were a lot of topics covered such as tacit vs. explicit, the beer game, the low tech dashboard and safety language among other things. One topic was ways to increase your credibility as a tester by:

• How you dress
• Using correct language / words
• Using the right tone
• Growing a beard

Second day started off with Johanna Rothmans keynote “Kick-Ass Manager”.

I’ll give you some of my favorite quotes.

• “Hire people who are smarter than you”
• “Multitasking has never worked. It is the fastest way to get nothing done”
• “Adapt / evolve or die”
• “Bugs are not the interesting point to business, the effect of the bugs are.”

The leading edge of testing mobile apps – Julian Harty

I was one of the lucky few who got the “Mobile developers guide to the galaxy” book (which you can get for free here http://www.enough.de/products/mobile-developers-guide/).

One of the topics where that feedback is public and very important for the app. If you get bad feedback (and it can be enough with a couple of one star ratings among a bunch of five star) it is often easier to scrap and re-do. Another thing is that it takes roughly 2 weeks to get your app approved by Apple and around 6 weeks to get an update (fix) in.

Linguistics – On how to keep the dialogue Constructive – Leo Hepis

For this workshop we got two brave volunteers (Erik Brickarp and Martin Nilsson) to act out a phone screening for a job interview. Observers where to watch out for uncooperative answers e.g. “Will I get a build on Friday? – Our integrator is on vacation”. The workshop focused on the Meaning part of Satir interaction model (Intake, Meaning, Significance and Response). The workshop got me thinking on how I usually respond and will make me more careful in my responses from now on.

The evil testers Guide to Technical Web Testing – Allan Richardson

“Technical testing is a reminder to keep going deeper”. You can use MORIM (Model through Observation, Reflection, Interrogation and Manipulation) as a reminder to keep going deeper.

Allan uses a version of Chomsky’s Transformational Grammar to model his thinking in multiple surface structures to form one deep structure. Every new surface structure will either strengthen or re-form the deep structure.

And then a quote about how he work:

“Developers work in the backend so I work from the frontend and work myself back”.

Observation Ninjas & Description Superheroes – Ilari Henrik Aegerter

It’s amazing how easy it is to fool our brain. For this Ilari used pictures, mind-reader exercises and card tricks to show us. I especially liked the mind-reader one which I have amazed all my friends with (you can see it in the presentation http://vimeo.com/66929939 @29:34). Hopefully this will make me more aware of the things that might fool me and hence avoid being fooled.

The Art of Understanding – Carsten Feilberg

This was a workshop where we were divided in 3 teams and all the teams got a number of words that could have a lot of different meaning such as Project, Developer, Tester, Manager, Data, Transport, Delivery, Shop, Shop owner and so on. Our task was to arrange these words in relation to each other. The thing I found interesting here was that even though we didn’t have enough information to make any real meaning of this we still struggled to find meaning. Another interesting thing was that even though we didn’t know our solution was the right one and we only had worked with it for 10 minutes, most of us still defended our own solution when comparing to the other groups and wouldn’t change anything.

Bad Metrics and What You Can Do About It – Paul Holland

This is a really interesting topic and Paul attacked it with humor and passion. A lot of the bad metrics he mentioned I have unfortunately seen myself. Among others the salary based “you need to find 3 bugs a week”.

Goodhart’s Law which says “When a measure becomes a target, it ceases to be a good measure” is something we all should be aware of.

Paul also listed some elements of Bad Metrics:

• Measure and / or compare elements that are inconsistent in size or composition
• Create competition between individuals and / or teams.
• Easy to “game” or circumvent the desired intention
• Containing misleading information or gives a false sense of completeness.

A number of bugs found really don’t say anything. There is no way of knowing the status of a product by just a bug number. What kind of bugs is it, how easy can we fix them, are they critical and so on.

We know more than we can tell – Michael Bolton

There are a lot of things we know but we don’t know why and how. We learn all the time by imitations and trial and error.

Michael talked among other things about the 3 flavors of tacit knowledge:

• Relational – (Head) “All possible chess games”.
• Somatic – (Body) “Can we explain how to keep your balance when riding bike explicit?”
• Collective – (Society) “Biking in Amsterdam, follow the “flow” of the traffic”

A good quote from Michael when talking about if a tester needs to be able to program:

“Either be a programmer or be charming”.

There were a lot of good sessions which have triggered a lot of thoughts.

But Let’s Test is not only about the sessions. The interaction during lunch, between the sessions and during the evenings are equally important if not more so. Meeting and talking to all of the participants is what really made an impression. Add good food and free beer to the equation and you have a recipe for success. So all in all this was a really good conference and I’m definitely coming back next year.

See you all next year at Let’s Test 2014, if not sooner.

I recently had an interesting experience that gave me an introduction to cross site scripting (XSS). I have not dealt with this kind of security testing before but I have learned that there are some really fun things than can be done, and those things can be used to do really bad things.

It started when I was discussing with a colleague about a couple of text fields on the web-page I was testing. What I had found was that the text fields accepted any text given even though you were only supposed to enter a number from 0 to 100. Overhearing our conversation my desk neighbour and developer suggested we could try to enter a script into the text box and see what happens. On the first attempt nothing more happened than the script being accepted as text. But when trying the same script on the next field the script was suddenly executed and I got a popup with a friendly “Hello!”. The text field was supposed to accept a name entered as a text-string but took my small script and executed it. The script was not only executed once, it was executed every time any user entered the same area on the web page.
A friendly hello might not be so dangerous but if it instead said “Your Outlook password has expired, please enter your old password and enter a new.” then the users might be exposed to a severe risk. Even though my knowledge in security testing has been limited to this point, copy-pasting a small script was enough to uncover a potentially severe issue.
This experience also showed me once again how powerful the combined effort of testers and developers can be: I uncovered a potential place of an error and the developer showed how that error could be exploited.
Below is the small java script I used. If this one is entered somewhere in a website and executed then the site is potentially vulnerable to XSS and needs attention:

<script>alert(‘Hello XSS!’)</script>

For more learnings about security issues check out the CWE/SANS Top 25 Most Dangerous Software Errors. At place four for 2011 we have XSS and there is a short section about prevention and mitigation.
Another page I found very interesting to read is OWASP and their dictionary on different attacks.

Happy cross site scripting!

Team spirit can be a good thing but when teams starts to compete internally despite having the same goal then it can be a problem.
During a startup meeting for a new project I suggested to the developers that they could bring code to me for testing in an earlier stage. The developers stood up as one entity and shouted “No! We don´t want you or anyone else to file bug reports on our unfinished code!”. The developers were used to having the development and test teams separate and in a kind of competition; the testers were handed the code after it was “done” and then pointed out the errors the developers had made. To calm the developers down and to stress that I was not looking for improving my statistics I told them: “I will not file any bugs before OR after you have delivered your code unless you first agree to it. I can provide help and value by being involved in an earlier stage supporting you.”. And then there was silence…
Once the confusion has settled the developers decided they were ok with the idea. Shortly after that meeting I started to get frequent visits from the developers with suggestions and requests for help with testing of different parts of the software. After four months I was leaving for another assignment and got the question who would be taking over as test lead after me and I replied: “I’m not test lead, that´s the guy over there.”. The answer was followed by the question “Who is that?”.
If you spot teams building walls against each other putting more effort into preparing for long sieges than working together an intervention might be in place. If the people in the teams are not working well together because they are in different teams then Unteaming the teams might help. By unteaming the development and test team and creating a bigger team the cooperation towards our common goal greatly improved. The actual test lead still had the mindset of his team being separate from the other teams and because of this he became less involved in the combined effort. Since he waited for communication via mail or management he missed out on the opportunity to work more proactive.
My conclusion from this experience is that team building can be good thing but if it is done at the expense of less collaboration between teams then unteaming can make a positive difference.

And I want to send a thank you to Michael Bolton for pushing me to write this post and who helped me with feedback!

After the PSL course I started to be more active in building rapport with my colleagues. On the first day of being assigned to a new project I took a different approach than I had done before: Instead of sitting down starting to play around with the development software I took a big cup coffee and started to visit the project members. I decided that the first two days I would not look at my tools or access the software: I would talk talk talk to people. I also decided to not push for information about the software, if anyone signaled they wanted to chat about something else I would do so (or leave if they did not want to chat).

What I experienced was that by not pushing, the developers often started to explain the system on their own initiative. By not saying how I wanted to test, the developers started to spawn ideas of were testing might be beneficial. That information might or might not end up being useful for my testing but it is was a type of information that I did not usually receive before. The developers started thinking on what I might need and such a small thing as someone remembering a meeting or demo that might be interesting for me was a great boost for my work. When the two days ended I had a better understanding than ever before about the new project.

Another specific thing happened to me the other day happened when I strolled over to a developer with whom I sometimes chat but he has no direct relation to the project in which I am working. When asked how things are going I mentioned that I had trouble finding a certain piece of information. Hearing about my small issue he turned to his desk neighbor and 30 seconds later I was making a copy of the information I had been looking for. Just by taking a coffee and chat around I saved a lot of time.

When development gets complex you don´t always know who might have what you need to solve a problem. Building rapport and talking with people can greatly improve your chances of stumbling on a key to a problem.

There were many lessons to be learned from the Problem Solving Leadership course but in this post I will focus on one thing that was a big lesson for me when it comes to problem solving: The rapport (the connection or relation) with the person or the people you work with might be much more important when solving a problem than I previously believed.

During the course we were one day given two tasks to solve. I had been talking a lot over dinner with one of my course mates and we teamed up to create a draft for the first task. We were able to create the draft quite quickly in comparison with the other groups within the team who had not had as much contact with each other before. And when our draft was presented and approved by the team the team got stuck and it took a long time before we were able to proceed and finalize the task. There were several strong wills that wanted to take charge and move the work in a certain direction giving the people who liked to wait for their turn to speak no opportunity to do so. Such a simple thing as refining our task draft, clear the room and order in food turned into a huge issue were the team had difficulties deciding on anything. After a while a couple of the strong wills decided themselves to take a step back and suddenly the workflow loosened and we were able to make progress and we finished in time.

The second task was solved with bravour despite us having much less time. It seemed that we had resolved our issues during the first task and gotten to know and understand each other. Despite the two tasks being given the same day the difference in the cooperation for solving the problem was dramatic.

In a comment from Jerry Weinberg during the course he approximated that about 70% of a developers time is spent on interactions with other people and that is close to an estimation I did for my own work a couple of years ago. Putting together that approximation with my experiences from the course it became much more visible for me how important the interaction is with my fellow colleagues when trying to solve a problem. In the future I can see myself putting a bigger effort on building rapport to tackle future problems in a better way.

HTML Ipsum Presents

Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Vestibulum tortor quam, feugiat vitae, ultricies eget, tempor sit amet, ante. Donec eu libero sit amet quam egestas semper. Aenean ultricies mi vitae est. Mauris placerat eleifend leo. Quisque sit amet est et sapien ullamcorper pharetra. Vestibulum erat wisi, condimentum sed, commodo vitae, ornare sit amet, wisi.

You have waited a long time for this and finally you get software to test from development. Energized and excited you install the program and fire it up. It all looks great, the new interface is stunning. You are ready to beat the crap out of this software. So you roll up you sleeves and hit the first soft button. The screen turns blue. Not that exited any more, a bit pissed off is a better word of how you feel right now. So you put on your friendly face and walk back to the developers, explaining what happened and kindly asking them to fix this problem.Now, back at your desk waiting and waiting and suddenly a new version of the software is released. Wohooo you shout, time to get busy. Installation works great and it fires up, now the shivering moment arrives, you hit the same soft button and it takes you just where you expected it, to the registration page. Great, the devs have done a fine job! Now I can type in my contacts to register. But there is alway a but, I can’t type anything into the input fields, WTF!!! Have you ever been into a similar situation? I sure have a couple of them in my pocket. Isn’t it frustrating when you just have to gently poke at a software and it falls apart? You never come to the point where you actually can practice your testing skills because the bugs just pop up in your face without you doing anything in particular. Why does this happen?

Most of the time it is because there is a lack of checking the software before you are expected to test it.
Michael Bolton wrote a series of wonderful posts about testing vs. checking.
I think this is an important distinction to make and it has served me very well rhetorically when discussing and describing its different purposes and how it brings different values.
However, I see tendencies that some look down on checking and it is not as “fancy” as testing, it is second class.
This however does not serve us good. There is no point in comparing which is “best” since it is not a competition. It is not that if you do testing you do not need checking or (as some agile folks seem to believe) that if you do checking you do not need testing. If you believe so then you have misunderstood the whole concept and I strongly recommend you to go back and read Michaels blog posts very carefully. So checking is an activity that is algorithmic and can be automated, the verdict is pass or fail. Its purpose is to confirm that what we already know about that specific part/function/scenario still is true.  With checking we want to make sure that what previously worked in the product still works. Testing however, is a heuristic based sapient activity. It is exploration, learning and evaluation of a product and our motive for testing is to find new information. We ask new questions to the product to uncover the unknown. So what I mean by checking as an enabler for testing is that we need checks to inform us that our existing believes of the product are still intact. When we have that information we can start testing, that is go further then we did last time by starting at the point we previously stopped at. Checking provides us with a solid foundation to stand on that will give us confidence to start testing.

But we should put checking where it belongs, that is where it provides most value. Most often that is as close in time to the developed code as possible to provide fast feedback. It is developers who often can do this cheapest and at the same time benefit the most from having frequent and rapid checking of their code. There are some really good practices out there that includes checks, like TDD (Test Driven Development), BDD (Behavior Driven Development) and others. What bothers me is when I see testers being requested to do these checks instead of doing testing. These practices are not testing practices, they are development practices. Checking is only one of the benefit with them. When you put a tester to do this task then you tend to loose all the other good and valuable benefits of the practice. Benefits such as knowing when to start coding and when to stop coding, clean and simple design, fast feedback to developer about the code, confidence for the developer to continue coding, improved developer ownership of quality of the code, keeping short iterations with new code before running checks and many more values. Why would someone want to take all that away from the team by putting the tester to do the checks?
When developers do this I very seldom encounter the scenario I described at the beginning. This saves the whole project a lot of time and it keeps stability to ensure the development moving forward. It also keeps a lot of frustration away from the team.
To conclude this, don’t look down on checking. Checking is a very important activity but it is not necessarily most beneficial to put testers to perform this. Use checking as an enabler for testing!

I’ve been working with exploratory and session based testing for a while now and I like this form of testing for its flexibility and adaptability. I manage my sessions with session based test management, outlined by Jonathan Bach in his “Session-Based Test Management” paper [1] with a small addition; I also record all of my session with a screen capture tool. With sessions spanning up to 90 minutes, the screen captures becomes rather large files, even with compression. It also takes a fair bit of resources from the computer to capture and encode in real time, so why do I insist on doing this for every session?

First of all, I’m mainly testing web based software, where I do not have to be overly concerned about what consequences to the SUT the extra load on my PC might have. If I tested locally installed software, I would have to take this more into consideration, but I’ll cross that bridge when I get to it.

The advantages that I see in capturing each and every session are:

• Attaching snippets to bug reports
• Reviewing captures to help reproduce bugs found in sessions
• Helping me keep focused during the session
• Allowing me to be the navigator

Attaching snippets to bug reports

From the capture, I can cut out snippets that highlight a certain sequence or error that I have experienced during the session. This might be the most obvious of the advantages, but I rarely use it myself. The reason for this is that I will generally try to reproduce the bug in a simpler and more straight-forward manner than how I first saw it in the session. When I succeed in this, then I keep to reporting the steps I’ve taken, possibly with attached screenshots. It’s only when I can’t reproduce the bug that I cut out the snippet from the video and attach to the bug. In this case, the snippet is very valuable, though.

Reviewing captures to help reproduce bugs found in sessions

This part ties into what I wrote in the previous section. Personally I like to stay with the charter as much as possible in a session and avoid things that break my “flow”. One of the things that I have found breaks my flow is reproducing and reporting bugs. In a session, when I find something that looks like a bug, I like to just make a note of it in order to return to it after the session. This is just a personal preference and others might prefer to investigate bug when they are fresh in memory, but if you work like me and just make a note of it, it’s crucial to remember what you did to provoke the bug. In most cases I can remember it from the notes I make, but in a few cases I only think I remember, but I have actually missed a small, but critical step. In these situations, I can go back and review the video to retrace my own steps. Just finding that missing step makes the whole recording worth the effort.

Helping me keep focused during the session

The two first advantages have been reasonably objective in what they can help with. The two last ones are much more subjective and directly related to the way I approach sessions. This one is purely psychological and based on the fact that I am very good at tricking myself. As everyone else, I want my sessions to be uninterrupted by email, IM, phone calls, etc. Unfortunately, I’m really bad at remembering to turn off IM, email notifications, etc., which means that I occasionally get notifications popping up on my screen during a session. With these notifications comes an almost irresistible urge to check the message behind the notification, but when I have the recording software running, it’s almost like having someone watching over my shoulder and I get a very bad consciousness if I actually check my email. I know that I can just pause the video, or cut out any part from it later, but that doesn’t matter; the video still helps me focus on the session.

Allowing me to be the navigator

This is an analogy to the roles in an XP team and ties into how I like to work with sessions. In an XP pair, there is a “driver” that writes the code and focuses on details and a “navigator” that looks at the bigger picture, design, architecture, etc. In a session, I like to be the “navigator” in the sense that I don’t want to get bogged down in details about a bug; I want to follow the flow of the charter, think of new tests and notice irregularities in the software. Since the video will record every step I take, I don’t have to divert attentions to this, but can focus on what I want to do in the session.

Those are the advantages I see in recording every session I perform. You might ask how long I store these files, considering the size of each file. As with most other things, the answer is that it depends. I’m likely to keep all recordings for one iteration, if that’s how the development is done, but if some bugs are left open after an iteration, I might consider keeping the video until the bug is closed, just in case the developers need some extra information that I did not think of when reporting the bug. If I have a large enough hard drive, I can even consider keeping all of the videos until the end of the project/release.

In my day-to-day work I use Ubuntu Linux and I always try to use this for testing, whenever the requirements do not prevent this. I like Linux for several reasons, one of them being the abundance of free utilities, for example the ones I can use to record and edit videos. To record my session on Linux, I use XVidCap [2] and for editing the videos and extracting snippets I use AVIDemux [3]. I know that there are several good tools to both Windows and Mac (albeit not for free), but I do not have any personal experience from using these.

[1] http://www.satisfice.com/articles/sbtm.pdf
[2] http://xvidcap.sourceforge.net/
[3] http://fixounet.free.fr/avidemux/

I have been working with test now for some years, and for each year I realize how much more there is to learn, but also that inspiration is important to maintain.

One of my best testing experiences which gave me a lot of new knowledge and inspiration was the SWET conference, Swedish Workshop on Exploratory Testing. So far three SWETs have been held, and I have had the pleasure of attending SWET2 (1) and SWET3 (2).

First of all, it was a feeling of honor to attend SWET2 because it is a conference to which you get invited. At SWET3, Henrik Andersson and I were organizing the conference and acting as facilitators.
Each SWET has a theme on which the presentations and discussions should focus, e.g. for SWET2 the theme was “Test Planning and Status Reporting for Exploratory Testing”, at SWET3 the theme was “Teaching Testing”. All conference participants need to send in an abstract on the subject. Besides relating to the conference theme, the abstracts should be based on personal experiences. The organizers then create a prioritized list of abstracts in which order the presentations will be held.

At SWET, the presentations are not very long, about 20 minutes, but the questions and comments may continue as long as it is interesting. This form is based on LAWST, http://lawst.com/. Each participant gets a set of four cards to be used for getting attention after the presentations; “New question”, “Comment on a previous question”, “Urgent attention” and “Rat hole”. “Rat hole means that a discussion is stuck and is not going forward. One participant, normally one of the organizers, is acting as facilitator and keeping track of by whom and in which order cards are raised.
Because each presentation with questions and comments may take quite a while, it is usually only the 3-4 first presentations on the list that actually are held.

The first thing that struck me when I arrived to SWET2 was the relaxed and friendly atmosphere. I was a bit nervous, but that feeling quickly went away. Some of us arrived the night before the conference and we had a nice dinner and great time together. We talked about a lot of things, including test of course. At SWET3, it was great to once again meet some of the testers from SWET2 and exciting to make some new acquaintances.

When the actual conference starts, each person in the room gets to introduce themselves shortly and to say something about their expectations for the conference. Already at this time, I noticed that focus and concentration sharpened among us. We all wanted to get a lot out of the meeting. It was a group of testers who are very serious and enthusiastic about their profession.

All of the presentations were of high quality and I found it interesting and inspiring to listen to these skilled testers telling us about their experiences. Besides that, I also got a lot of good tips and I could relate to own experiences and how I can change and improve my way of working. During the presentations, the listeners take notes to remember questions and comments they come up with.
You may think that the idea of focusing on a theme would give small variations in the different presentations, but on the contrary; there were big variations. The fact is that each presentation gave input to so many questions, and each question resulted in comments and new questions and so on. The queue of questions and comments became very long so it took some time before one got the chance to say something. Sometimes, the comments and questions actually drifted too far away from the presentation, in which case the facilitator could interrupt and remind the participants to stay focus.
One thing I noticed was that discussions started off at quite a high level, in that sense that some things were already somehow understood, because all of us already has studied Exploratory Testing and we also share many valuations about testing. The questions and comments took a lot of time and this is a situation which I rarely have experienced before, the possibility to discuss something without time pressure. I would love to do this more in life.
But it was not just getting answers for you questions that gave a value; it was also a great forum for practicing the ability to listen, to questioning, to argument and to receive feedback on my own ideas.

SWET also includes a session called “Lightning Talks”. During this session, any one of the attendances is allowed to do a very short presentation about any test related subject that they want to share with the others. There is not very much time for questions during this session, but I managed to take some interesting notes which I brought with me from the conference.

Well, after dinner, the first “working day” was over, but the rest of the evening was just as rewarding, maybe a bit less formal. Lots of interesting discussions, nice music and a few good beers. When the conference ended after lunch the second day, I was a bit tired but really glad and inspired. I had learned a lot and made new friends within the Exploratory Testing community.

1. At SWET2, we were 15 persons; Henrik Andersson, Azin Bergman, Sigurdur Birgisson, Rikard Edgren, Henrik Emilsson, Ola Hyltén, Martin Jansson, Johan Jonasson, Saam Koroorian, Simon Morley, Torbjörn Ryber, Fredrik Scheja, Christin Wiedemann, Steve Öberg and myself Robert Bergqvist.
2. At SWET3, we were 11 persons; Anders Claesson, Henrik Andersson, Johan Jonasson, Maria Kedemo, Ola Hyltén, Oscar Cosmo, Petter Mattsson, Rikard Edgren, Sigurdur Birgisson, Simon Morley and myself Robert Bergqvist.

The Association for Software Testing has just launched the third course in the Black Box Software Testing series, developed by Cem Kaner et al. This course is the Test Design and it follows the Foundations and Bug advocacy courses. I was lucky to be able to join the first pilot version of the Test Design course that is just about to finish these days.

These are the objectives for the course:
This is an introductory survey of test design. The course introduces students to:

• many techniques at a superficial level (what the technique is),
• a few techniques at a practical level (how to do it),
• ways to mentally organize this collection,
• using the Heuristic Test Strategy Model for test planning and design, and
• using concept mapping tools for test planning.

We don’t have time to develop your skills in these techniques. Our next courses will focus on one technique each. THESE will build deeper knowledge and skill, technique by technique.

This looks like a reasonable scoping of the course, but already in lecture one, I went “Wow!”. There are truly many techniques out there that I have not even heard of, never mind given a try. To give you an idea of the amount of data gone through in the first lecture; the videos for the first lecture spans just under 52 minutes total and covers 143 slides!

Fortunately the following lectures started looking into the few selected techniques at a more practical level, thus slowing down the ferocious flow of new information. With the more practical level came exercises, though. The test design course has more exercises than the previous two courses and there are two more extensive exercises that span two lectures. The practical aspect certainly has a more prominent role this course compared to the previous ones.

Most of the exercises in the course do not require peer feedback, which also is a change from earlier courses. It seemed convenient to start with that I did not have to spend time on the peer feedback, but I soon realized that I missed it, both reading through the other students’ work with the aim to give feedback and to get the feedback. In the end I probably ended up reading through more of the other students’ work that I would have if I had been assigned one or two to review. It’s hard to take the time doing this if it’s not mandatory, though.

Talking of time; I was fortunate that I could do this during a period where I could spend some extra time on the course. I spent all the extra time I had available and I still felt like I could have done more. I don’t know how the other poor students managed to get all of the work done besides their full-time jobs. Taking these BBST online courses do require good time management skills of the participants. The foundation course does teach this. Maybe I should revisit that course again?

Throughout all the BBST courses I have taken, the quiz questions have been a source of frustration. This course was no exception in the beginning. My main objection with some (far from all) questions were that they felt more like traps than confirmations about the content. I would review the feedback from the quiz questions I got wrong and think this has nothing to do with my understanding of the course material; it’s just down to how one would interpret the test in the question or in the answer. An interpretation I often did not agree to.

I know that Cem Kaner wants to use the quiz also to help the students practice precise reading and I Cem Kaner did elaborate with more feedback in one of the questions where I voiced my complaints. I can understand his motives. I still don’t really agree to them, but I have decided to accept it. Now, if I miss an answer and I don’t agree to the explanation the quiz gives as feedback, I’ll just ignore it. The other questions are still good opportunities to learn in case I’ve misunderstood something from the lectures.

I also gladly endure the quizzes considering the overall benefit from the course. The test design is a high-paced course with a ton of information. A lot of this information was new to me and I feel I have learned many new techniques during the last few weeks. I have had the chance to scratch on the surface for a few of them. Far from enough to really understand and master any of the techniques, but the course has opened up my eyes to many new things and I now know how I can start an approach and where I can find further information when I need it.

After the course I feel a boost of motivation and lot of very exciting ideas that I now just want to find the first opportunity at work to practice and put my newly found knowledge to good use.

Thanks to Cem Kaner, Becky Fiedler, Michael Larsen and all my fellow students that made the Test Design course such a rewarding challenge!

An objection that we have heard to using an external test team, especially in an agile setup, is that since the developers do most or all of the testing now, they would lose a valuable experience and a chance to learn if the test is outsourced to an external team.

The way I see it, there are a couple of fallacies in this concern. First of all, even though there usually is a higher level of developer tests in agile, these tests are not a substitute for manual tests, they are a complement. Second, if the developers learn from their own testing, they are less likely to find new points of views; they’ll learn, but mainly within their existing box.

Regarding the developer tests; these are to a large extent automated white box tests that the developer or continuous integration system run as an integrated part of the check-in and release process. These tests are invaluable for two reasons, they give early feedback to the developers and they can test things that are not possible with black box testing. However, these tests cannot replace the analytical and exploratory skills of a human tester. The developers tests are great at weeding out bugs that potentially would waste the time of a human tester, which allows the tester to focus on much more complicated tests and harder to find bugs. A symbiosis of automated developer tests and human analytical and exploratory tests brings the best of two worlds together.

When talking about learning and education, there are a couple of benefits of bringing in external testers. We are all limited to look for bugs in areas and situation that we can imagine would happen. This also applies to developers and relying on only developers for testing will bias the testing towards the areas and situations they think of. IF working with an external team of testers, another point of view is added and more areas and situations will be covered. When bugs are found, developers and testers can have a dialog about them and the developers can learn to see new areas and situations.

Tools can to some extent also bring in a new perspective and find bugs in areas where developers are not looking. However, most tools need to be configured and if this configuration is done by the developers, there is still a risk that the tool will biased towards the point of view of the developers. Also, a tool cannot have a dialog with the developers about the thought process behind a bug, limiting the understanding and learning that can be made from bugs a tool finds.

Finally, by bringing testers into the picture and take some of the testing job off the developers, time will be freed up for the developers. This time can  be used for training that will help the developers prevent bugs, not just fixing them.

“Punk rock is very rebellious, of course, but it also means thinking for yourself.”

Dexter Holland

“Questioning anything and everything, to me, is punk rock.”

Henry Rollins

“House of Test has contributed greatly to our organization’s efficiency and quality through its extensive expertise, flexibility and proactivity. Our shared projects have kept time- and cost plans, with deliveries of precise data and documentation. We intend to continue our partnership with House of Test for a long time! “

Sammy Almedal, Managing Director, JAK Medlemsbank

Consulting

House of Test Consulting ist nicht ein gewöhnlicher Personalverleiher oder Ressourcenanbieter. Unsere Beratung fokusiert sowohl auf die strategische Ebene als auch auf die Projektdienstleistungen. House of Test kann Ihnen eine komplette Palette von Test Dienstleistungen anbieten, und wir sind immer darauf bedacht, dass unsere Kunden das Gefühl haben, dass die Zusammenarbeit mit House of Test Wert liefert, sowohl für das aktuelle Projekt als auch für zukünftige.

House of Test bietet kompetenzbasierte Beratungsleistungen an, sowohl für kürzere Projekte, als auch für Teilzeitaufgaben wie Mentoring, Coaching, Assessments und Reorganisationen. Gerne bieten wir auch Vollzeitverträge an, entweder eingebettet in Ihre Teams oder mit uns als Projektleitung.

Unsere Consultants

Wir bei House of Test pflegen Spitzenleistung. Wir sind uns sicher, dass wir einige der besten Test Consultants der Welt im Team haben. Unsere Berater sind Experten bei der Analyse und dem Einarbeiten in neue Gebietskontexte. Das heisst, wir liefern Wert vom ersten Tag an. Unsere Consultants haben in jeder erdenklichen Branche gearbeitet, und sie sind in der Lage, in jedem Kontext zu brillieren, sei es Mobile, Internet oder in reguliertem Umfeld.

Was uns aber wirklich einzigartig macht, ist die Art, wie wir bei House of Test Kompetenzentwicklung angehen. Viele Unternehmen behaupten, guten Wissensaustausch zu pflegen, aber nur wenige gehen so weit wie wir. Wir treffen uns als Gruppe 4-5 mal pro Jahr für mehrere Tage lange Workshops über Software Testing. In viel häufigeren Zyklen tauschen wir Erfahrungen und Feedback über unsere aktuellen Testideen und Herausforderungen aus. Das ist nicht etwas, was wir forcieren müssen. Es ist einfach was passiert, wenn Sie passionierte Menschen mit gemeinsamer Leidenschaft zusammenbringen.

Coaching

House of Test verfügt über einige der erfahrensten Test Coaches auf dem Markt. Wir passen unser Coaching auf die Bedürfnisse Ihres Unternehmens, Ihres Teams oder Einzelpersonen an.

Die Aufgabe eines Coaches ist es, die Bedürfnisse, Motivationen, Wünsche und Fähigkeiten zu fördern und Denkprozesse für die Formung von dauerhaften Veränderungen zu unterstützen. Wenn wir trainieren, verwenden wir den sokratischen Ansatz mit Fragen und Antworten, die den Einzelnen in Richtung von neuen Einsichten und Erkenntnissen bringt.

Unser Coaching ist immer zielbasiert, und wir evaluieren es kontinuierlich und passen uns an, um sicherzustellen, dass wir dem vereinbarten Ziel nach jeder Coaching-Sitzung näher kommen. Wenn wir vom Kurs abkommen, korrigieren wir ihn, und wenn das Ziel mehr relevant ist, passen wir das Ziel an.

Mentoring

Der Unterschied zwischen Coaching und Mentoring ist subtil, und der Fokus zwischen Coaching und Mentoring ändert sich während einem Auftrag häufig. Das Ziel unserer Mentoring-Sitzungen ist es, Wissen und Erfahrung weiterzugeben und Türen zu öffnen, die ausser Reichweite erscheinen. Unsere Trainer verfügen über ein jahrzehntelang angesammeltes Testing Know-How und sind erfahrene Trainer und Lehrer, die eine höchst effektive Beziehung zwischen Mentor und Mentee garantieren.

Test Management

Wir glauben, Software Testing ist zu einem grossen Teil nicht vorhersehbar und unsere gemeinsame Erfahrung bestätigt das. Als Gegenmassnahme möchten wir so wenig Zeit wie möglich für die Planung aufwenden und verbringen stattdessen den Grossteil unserer Bemühungen beim Testen selbst. Natürlich planen wir. Natürlich arbeiten wir Strategien aus. Aber wir unterlassen es, den Kopf in den Sand zu stecken und zu glauben, dass sich nichts ändern wird. Wir wissen, dass wir im Verlauf des Testprojekts alle klüger und besser werden. Nicht alle Tests bleiben wertvoll. Nicht alle Bereiche behalten das gleiche Risiko.

Unser Werkzeugkasten ist sehr versatil und adaptiv und ist in der Regel unaufdringlich und passt sich den Werkzeugen und Methoden an, welche Sie in Ihrer Organisation bereits verwenden: Wir verwenden Mindmaps und Whiteboards, um die Übersicht zu behalten. Visualisierungen sind hilfreich, um Status und Ziele aufzuzeigen und aber gleichzeitig neue Ideen nicht zu beschränken. Wann immer möglich, wenden wir gerne entweder Session-Based oder Thread-Based Testing an, weil dies die Freiheit der Tester fördert, um wirklich gut testen zu können, während trotzdem alles unter Kontrolle bleibt.

Ein Grossteil des Testmanagements ist die Logistik von Menschen und ihren Qualifikationen, Software, Daten, Umgebungen und alles andere zum gewünschten Zeitpunkt bereit zu halten. Wir neigen dazu, uns auf die Tester, ihr Können und ihre berufliche Entwicklung zu konzentrieren. Wir bevorzugen es, Menschen und nicht Testfälle zu managen. Wir verwenden Coaching und Mentoring, weil wir uns nicht mit Mittelmass zufrieden geben. Wir denken, dass Tester, die zuversichtlich, geschickt und Stolz auf ihre Arbeit sind, auch die besten Resultate erbringen. Wer will schon eine “Resource” sein und ferngesteuert durchschnittliche Arbeit verrichten?

Wir ermuntern die Anwendung von verschiedenen Arten von Testreporting, weil unterschiedliche Bedürfnisse nicht mit einem standardisierten “One-Size-Fits-All”-Report abgehandelt werden können, welches vom gerade verwendeten Testmanagement Tool erzeugt wird. Unser Ziel ist es, jederzeit rechtzeitig Ergebnisse melden zu können, auf den Punkt gebracht, ohne die Unsicherheiten zu verbergen. Wie sonst können unsere Ergebnisse für Sie von Nutzen sein?

Letztlich geht es darum, vertrauenswürdige und verständliche Informationen an die richtigen Leute zur richtigen Zeit zu bringen.

Test Engineering

Unter unseren Beratern befinden sich einige der weltweit führenden Experten des Context-Driven Software Testings. House of Test ist in der Lage, Berater einzusetzen, die direkt in Ihrem Testteam arbeiten oder sich in Ihre agiles Teams einbetten, um sowohl grossartige Arbeit zu verrichten und Ihr Team darin zu schulen, das gleiche zu tun.

Wir zeigen uns von unserer besten Seite, wenn wir in einem frühen Stadium in das Projekt hinzukommen wenn die Requirements gesammelt werden, welche dann Basis unserer Teststrategie werden. Natürlich ist das Überprüfen von Anforderungen nur ein kleiner Teil dessen, was wert-orientiertes Testen ist. In dem wir proaktiv arbeiten, überarbeiten wir ständig unsere Teststrategie, und indem wir mehr über die expliziten Requirements lernen, verstehen wir auch mehr über die impliziten Requirements, was uns hilft die “unbekannten Unbekannten” aufzuspüren, bevor Sie ihr Produkt auf den Markt bringen.

Sobald ein testbares Produkt verfügbar ist, werden unsere Consultants sich darin vertiefen und die Software (oder Hardware) von allen Seiten testen. Wir haben umfangreiche Erfahrungen mit dem Testen von Produkten in einer Vielzahl von Domänen und Geschäftsbereichen, seien es mobile Anwendungen und web-basierte Dienste oder Medizinalprodukte. Wir verstehen, dass Ihr Unternehmen einzigartig ist, aber wir wagen zu behaupten, dass wir in der Lage sind, einen grossen Teil unserer kollektiven Erfahrung einzubringen. Wir liefern qualitätsbezogene Informationen, auf die Sie sich verlassen können.

Performance Testing

Eine der schwierigsten Testaufgaben für viele Organisationen ist das Performance-Testing. House of Test verfügt über umfangreiche Erfahrung bei der Erstellung und Durchführung von Performance Teststrategien mit massgeschneiderten Reports, die Ihrer Organisation hilft zu verstehen, wo Ihre Performance Verbesserungbemühungen den meisten Nutzen bringen können.

Test Automatisierung

Unsere Continuous Integration Experten können Ihrer Organisation helfen, zu einem kontinuierlich releasebaren Stand zu kommen. Wir helfen Ihnen, die richtige Build Umgebung zu finden und eine für Sie praktikable und effektive Unit Test Strategie zu implementieren.

Wir können auch dazu beitragen, die Automatisierung Ihrer Tests zu übernehmen, die Ihnen helfen, Informationen über den Zustand ihrer Software zu einem bestimmten Zeitpunkt zu gewinnen.

Automatisierung kann Ihren Testern helfen, sich auf die Exploration des Produktes zu konzentrieren, anstatt das zu prüfen, was eine Maschine besser kann. Ihre unternehmerischen Entscheidungen werden dann auf einer soliden Plattform von zeitnahen und qualitätsbezogenen Informationen ruhen.

Finden Sie den idealen Kandidaten

Wenn es etwas gibt, worin wir wirklich gut sind, dann ist es unsere Fähigkeit, einen guten Tester oder eine gute Testerin in der Menge zu erkennen. Wir bieten an, ihnen bei der Analyse der Lebensläufe zu helfen und erfolgsversprechende Kandidaten zu identifizieren.

Wenn Sie eine paar Kandidaten für ein Vorstellungsgespräch ausgewählt haben, können unsere Berater ein Pre-Screening durchführen und alle durch eine Reihe von Übungen führen, um eine engere Auswahl für eine weitere Runde zu qualifizieren.

Wir bereiten das Screening durch direkte Gespräch mit Ihrem Team und ihren Mitarbeitern vor, um herauszufinden, was die dringendsten Bedürfnisse der Organisationen sind. Das erlaubt es uns, ein Verständnis für Ihre Unternehmenskultur und die herrschende Gruppendynamik zu entwickeln, welche die Kandidaten vorfinden würden, wenn sie bei der Bewerbung erfolgreich sind.

Wir helfen Ihnen, jemanden Besonderes zu finden.

Abstract

This 3-day couse is aimed at providing the students with basic knowledge and experience with planning and performing ET (Exploratory Testing), organizing the work and fitting the test results to the surrounding context. It starts from the context and points to different tools such as heuristics, note taking and SBTM/MTBS (Session-Based Test Management / Managing Testing Based on Sessions), to create effective and valuable testing, based on skill, thought and risks.

The course is highly interactive and packed with exercises that are always debriefed and analyzed to help students internalize their newly learned skills and tools.

The course can be extended with coaching activities to help the students incorporate ET in their work.

Course Objectives

The course covers the principles of CDT (Context-Driven Testing), the discipline of ET (Exploratory Testing), SBTM (Session-based Test Management) for managing the testing as well as Test Strategizing, Planning and Reporting based on Heuristics, Opportunities and Risks. The course relates everything we do to how it works within an Agile context, points to different Tools and Methods that may be of use along the way, and includes a perspective on how testing is fused with different needs (like ‘Cover all the requirements’, ‘What to do with the bugs’ and the impossibility of testing everything and getting effective testing done in the short time that is available).

Participants will gain basic knowledge and experience with planning and performing ET (Exploratory Testing), organizing the work and fitting the test results to the surrounding context

Who will benefit?

This course is most appropriate for Testers, Test Engineers, Test Leads, Test Consultants and Test Managers.

Details

Delegates are required to bring their own Laptop PC’s (not MAC or iPad’s), since a lot of the training is based on real hands-on exercises run on PC).

Duration: 3 days
Language: Danish or English
Teacher/Coach: Carsten Feilberg

Abstract

Agile development methodologies have quickly gained ground around the world and are today one of the more prevalent models for software development. For testers, the transition from traditional development models can be difficult or even frightening. It doesn’t have to be though.

In this 2-day course, you get to do a deep-dive into the what the principles of agile development means for testers and how you can transform your work into becoming both more efficient and effective.

Most parts of the course will contain practical exercises that will allow participants to not only gain theoretical knowledge, but also put it into practice right away, and thus gain a deeper understanding of the different topics.

Course Objectives

We’ll be discussing the tester’s role in agile teams and how to best create an agile test strategy. Participants will gain a better understanding of what exploratory testing can offer as well as knowledge of different sorts of heuristic tools and checklists that are absolutely essential for an agile tester.

Furthermore, we will go through the test management frameworks Session-based Test Management and Thread-based Test Management and discuss how they can be applied to gain traceability and help build a strong foundation for test reporting in an agile setting.

The course also contains sections dedicated to the role of test reporting, test automation, and different test tools that can help simplify your work as an agile tester.

Who will benefit?

Anybody working with testing in an agile setting, as well as Scrum Master and managers who need a deeper understanding for the role testing plays in an agile team or organization.

Details

Duration: 2 Days
Language: Swedish or English

Abstract

Software projects today are often large and complex. This requires a test organization which is able to work with requirements and risks in a structured way. Structured testing includes clear goals, strategies and coverage. One important tool in structured testing is to use test design techniques. Test design techniques make it possible to create test cases with better precision, a more specific coverage and better prioritization.

Many testers know about test design theories, but for some reason, an aware usage of test design techniques is not very common in many test organizations.

During two days, the most common black box test design techniques will be explained in theory and with examples. Several exercises will help the students to understand how and when to apply the different techniques, like Equivalence Partitioning, Boundary Value Analysis, State-Transition testing, Decision-tables, Use-case testing and Pairwise testing.

This workshop can be extended with coaching if needed.

After the workshops, the real challenge is to actually apply the new skills in the everyday work. A period of coaching within the “real” work could be very beneficial, especially for less experienced testers. The coaching could be done with one tester at a time, or with smaller groups of testers. The coach can help with requirement analysis, test case reviews and creation, etc. All with a focus on black box test design techniques.

Course Objectives

To get the tester acquainted with test design techniques usable in daily work.

Who will benefit?

Testers, Test developers

Details

Duration: 2 Days. Can be extended with personal coaching.
Language: Swedish, Danish or English
Teacher/Coach: Carsten Feilberg

Abstract

Exploratory Testing (ET) is an approach to software testing that is concisely described as simultaneous learning, test design and test execution. Cem Kaner, who coined the term in 1983, now defines exploratory testing as “a style of software testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize the quality of his/her work by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities that run in parallel throughout the project.” That definition also says something about why Exploratory Testing is such a powerful approach, once mastered. Some prominent experts in the field have even gone so far as to equate the word “testing” with Exploratory Testing, while referring to test case-driven, scripted testing as “checking”.

Course objectives

Introducing Exploratory Testing and showing its powers as a structured way of challenging a product in order to learn about it, and introducing SBTM as a systematic approach to keeping the direction and provide accountability of the testing done.

Who will benefit?

Testers, test managers, project managers, developers, architects

Details

Duration: 2-day workshop
Language: English, Danish or Swedish
Teacher/Coach: Carsten Feilberg or Henrik Andersson

Abstract

Session-Based Test Management is a way to make the intangibles of Exploratory Testing more tangible. It was specifically developed as a framework to organize testing so as to optimize and stay focused on the more important parts, while keeping up with the flexibility and powerful adaptation to change that Exploratory Testing provides. It means we have a set of expectations for what kind of work will be done and how it will be reported. As in a recording studio, this work is done in “sessions.” Sessions range typically from 45 minutes to several hours, depending on the charter – or “mission statement”, which the tester will focus on – for that time. Learning to describe, select and prioritize charters, debriefing the vital information and constantly meet the needs of the stakeholders is fun, giving and also challenging, especially in fast-moving projects with tight schedules.

Course objectives

To introduce and help the students set up SBTM as their primary tool for directing test activities. The course involves real problem solving related to introducing SBTM in an organisation and gives the students various tools and insights that helps them solving these problems.

Who will benefit?

Test managers, testers (Knowledge of exploratory testing (ET) is recommended)

Details

Duration: 2-day workshop
Language: Danish or English
Teacher/Coach: Carsten Feilberg

Abstract

This workshop mixes practical exercises and theory briefings with the participants’ own learning goals. The students will experience what participating in an actual Scrum project can be like. We will have experiental designed exercises covering all roles, artifacts and ceremonies in Scrum but more important we focus on bringing understanding to the social science aspects of Agile. This to emphasize the importance of good communication and teamwork.

The last day focuses entirely on applying the knowledge participants have gained into the context of a project. This day is about making Scrum work for you, in your circumstances. It is an advantage (but not necessary) if there is a certain amount of time (e.g. 1 week) between the first two workshop days and the third. This to allow the participants the possibility of receiving knowledge, translating it into practice and being able to bring fresh problems and examples to the last day of the workshop.

Course objectives

The goal of this workshop is that you as a participant will get a basic understanding of how and why Scrum works, and gain an insight into how to apply this in practice.

Who will benefit?

Managers, developers, testers

Details

Duration: 3 Days
Language: Swedish, Danish or English
Teacher/Coach: Henrik Andersson