Acceptance Testing with Executable Specifications

[@ContinuousDelivery]

Let me know what you consider as valuable properties of acceptance testing in the comments below 👇

[@PeterGfader]

The collaboration opportunity for the SME and Devs to ask and validate in code the question: "What if ...." And quickly they can write a test that answers that question

[@dawid_dahl]

I love these videos! Do you have a video on how this relates to the one quadrant down, i.e. Supports Programming/Tech Facing? We struggle a bit with how to think about both of these when we write our tickets/tests.

Great explanation. I'd appreciate an example of how to produce an acceptance test in practice. You might even consider this a suggestion for a forthcoming video!!

[@ContinuousDelivery]

Thanks for the suggestion! I do intend to do more on Acceptance Testing, including with some examples.

[@gpzim981]

This is by far the best KZbin channel about software development. Thanks Dave! I have bought your book and I will soon buy your course about testing and about CD.

[@ContinuousDelivery]

Awesome, thank you!

[@roaba3581]

I just recently started writing specifications with SpecFlow for C#, which uses the Gherkin language, which is still really exciting, because you can literally write out a userstory and extend it into user readable steps that are immedeately usable for further discussion/feedback. What I really love about your videos is, that everything you say either makes complete sense to me or shifts my point of view to a new angle, which is always interesting. Thanks a lot!

[@cookstaar]

Another great lesson. Making the “one acceptance test per acceptance criteria” rule work sounds very dependent on how you write your acceptance criteria - else you could end up in a mess. Would be great to see some examples of what level of abstraction you think acceptance criteria should be for this to be effective

[@ContinuousDelivery]

I plan to do more stuff on acceptance testing. It is a complex topic though so finding pieces that make a good, short video is a bit tricky, but I will do more on this subject! I am planning some online training courses that will cover this in more depth, which is probably a better format for this kind of thing.

[@PeterGfader]

Valuable Properties of Acceptance Testing 1. The collaboration opportunity for the SME and Devs to ask and validate in code the question: "What if ...." And quickly they can write a test that answers that question 2. The ease to write new tests especially for Bugs that we want to fix. (Bug First Driven Development)

[@ContinuousDelivery]

I take a pretty DDD approach to solving problems, and one of the things that I really like about the approach that I describe here, is how it helps you to establish the "ubiquitous language" of the system.

[@errrzarrr]

Excellent content. I wish more teams do this

[@ContinuousDelivery]

Thanks, yes, me too.

[@avoodi]

great video and suggestions as always Dave ; one question - when you say decouple test as much as possible from system - so that the test should be able to validate what its supposed to even if the system undergoes big /major changes.. Thats sounds great but exact details/example would help. Lets say there is some 'approval' process in some workflow and we want to write a test for this. and say the last step of this process is user fills in couple of fields, checks few check boxes, adds comments and clicks approved button. now in this case are you saying even if the count of fields on screen change, even if the layout change, even if say we have a need to add additional approver etc etc ; the test should work ?

[@ContinuousDelivery]

I will provide examples in future videos, so stay-tuned :)

[@davemasters]

I was going to ask this exact same question :)

[@ContinuousDelivery]

I didn't do a good job of replying, so let me have another go... I try to approach these kind of tests always from the users perspective. From their perspective the fields that they complete don't matter, their intent, presumably, is to say "I approve". The detail of what it takes to approve is what you, as the developer, care about, not what the user cares about. So separate those two things. In the Acceptance Test Case, create a Domain Specific Language (as I describe in the video) to capture the user's intent. In this language add, if it doesn't already exist, a step called something like "ApproveX". This does several things. It captures the user's intent. If that approval is important then this will always be true, however "Approval" is achieved. It is so general, that you will often find that approval may be useful in other contexts, and finally, you have strengthened and extended the ubiquitous language! Of course, you as the dev, still need the detail of the approval. So in a lower-layer of your test code write the group of interactions that make an "Approval". In these lower layers you get the info that you need and encode the interactions. My preferred approach is a 4 layer strategy... Test Case (Language of problem domain, "What") -> DSL Implementation (param parsing etc) -> Protocol Driver (Translate from DSL to System interactions, "How") -> System Under Test I plan to do videos on this stuff in future, meantime here is a conference presentation on the same topic: kzbin.info/www/bejne/qWK8ZWhqebmFqsk

[@ContinuousDelivery]

See my other reply...

[@tomyr95]

Thank you! Great review!

[@ContinuousDelivery]

Thanks

[@FlaviusAspra]

Great content! Let's dig a little deeper into technicalities, so that I can ask a question: The BDD needs to have underneath some support for the more concrete technologies used, some helpers, etc. E.g. "when operation A available", let's say an assertion that a link exists. In the BDD test, there's no mention of the link, just of the feature. Cool so far. The question is: would you tie those helpers to the technologies, e.g. to the link in this case, or would you introduce an additional "view model" and you'd make the system "render" those models instead of the whole css/HTML bloat, and assert against that view model?

[@ContinuousDelivery]

I think that this may help describe how I divide up the implementation of Acceptance Testing kzbin.info/www/bejne/gHWnZnh7f8yjfrc In it I describe my preferred 4-Layer architecture, Test cases at the top, written in a DSL, which capture What the system should do without saying anything about how it should do it. Next layer down is the DSL, which is shared between many test cases. The the protocol driver layer, which I think is what you are asking about, and finally the System Under Tests (SUT). The protocol drivers take biz-level concepts, captured in the DSL and translate them into something that drives the app. So a test case says "placeOrder" and the DSL sorts the paramaters, helps with abstraction and passes 'placeOrder' on to the protocol layer, which translates that idea into "complete this field with this value, and this other field with this other value, and press this button" or "Create a message with these values, and send it here". This video also describes some aspects of this separation: kzbin.info/www/bejne/sIrNaGN7qKlqq6s

[@mikezhang7096]

Great! I will try this.

[@ContinuousDelivery]

Have fun!

[@l_combo]

Loving these videos Dave, how about a collaboration with Chris Matts :)

[@ContinuousDelivery]

Thanks, I'd be happy to collab with Chris, though now we have to think of something. Hmmm.

[@l_combo]

@@ContinuousDelivery I'll let him know, we work together ;)

[@ContinuousDelivery]

Cool, thanks.

[@emaayan]

The irony is that a book that uses the term ubiquitous language doesn't seem to implement it for itself as most or it is rather unreadable

[@ContinuousDelivery]

Ubiquitous != Readable, it can be highly arcane depending on the problem domain.

[@jordanfox3782]

I think these explanations of continuous delivery processes require more practical demonstrations of how to write a system that does them correctly. This is all just theory based and at a certain point, I think this channel needs to step up and back up the theory with some examples, they don't need to be perfect, but at least prove you can do what you are telling others to do. For example how can I create a deployment pipeline step by step to develop a react application? Like what are the actual technologies to use for acceptance testing, building and deploying the software and how to use them?

[@ContinuousDelivery]

Well it is certainly not "just theory" because all of this stuff is based on real world experience, often with VERY complex systems. There are lots of videos on the channel of real examples of different aspects of CD. When it comes down to it, this is free advice, you don't have to take it 😉 Here's a real world example of a deployment pipeline that shows that this isn't just theory: kzbin.info/www/bejne/mHmueZemaa1gbtk ...but you'll still need to learn to use your own tools to build one like this. Here is how to start: kzbin.info/www/bejne/m6Ddd5-dqpt1r7s