What Makes A Good Acceptance Test? | SHORTS

[@mikeguantonioify]

Dave, your content is inspiring. I work with several quality engineering teams that don't understand the "user dress rehearsal" aspect of these tests and literally test everything. The code is thrown over the wall at them and they are expected to figure out the interaction with the code but not from the user's perspective. This leads to tightly coupled tests to the implementation of the system that are so hard to change and slow to run. If only they could align and realize behavior of the system over its implementation we

[@theTeslaFalcon]

This is the whitebox / blackbox that I was talking about. Whitebox, the testers know how the code works. Blackbox, they don't. There were usually 2 sets of testers. Whitebox testers have read all the docs and talked w us coders from the beginning. Blackbox testers are sent an unannounced email: "Go to ___. Test it, and make it break." The whiteboxers take the req doc which says, "Login screen" & write in the test plan: "Ensure successful login." Blackboxers are the ones who see a login in screen and start entering math equations and monkey on a keyboard gibberish. Not because they should, but just because they can.

[@spadhnik]

That is exactly what I am planning to implement at my work. Glad to find your video.

[@mateuszszczecinski8241]

Thank you for inspiring materials. What is missing in almost all tutorials or examples about BDD/TDD is link between acceptance tests and technical tests, for modern, distributed systems. All materials show only how to write separately hight level acceptance test, and integration/unit tests on component/class level, usually for systems created in classical mvc design. I would like to ask, do you have preferable development process for modern, distributed systems, where ui is separate app (or even many apps), one feature could be implemented by many modules (or microservices), all components could be created by different developers/teams? Do think that classical outside in development still have sense? I mean: creating acceptance test, creating unit/integration test for ui app view, implementing ui feature (mocking beckend), creating test for api service, implementing api service, creating test for application core(mocking another modules or externals system, maybe database), creating test for provider modules, etc.., making acceptance test pass, repeating whole process for another test scenario for story or feature. During whole process, which could take many days, acceptance test in pipeline will fail. Do you prefer some kind of consumer driven contract testing, for example that frontend team specify tests for backend api, etc? Are there another approaches, which uses BDD/TDD techniques? I would be grateful for describing whole process.

[@gruttewibe76]

Hi Dave, I like the idea of automated acceptance tests, but still have difficulty finding how to apply it in practice. Assuming we would build a system in BDD style, we would write end-to-end automated acceptance tests for every individual feature. At some point our suite of tests would be huge. With Continuous Delivery I presume you would want to run all such tests, or *at least* the tests relevant for some code changes, as part of the pipeline. If you run all of them, how do you make them run fast enough? If you run only the relevant ones, how do you know which ones are relevant? Aslak Hellesoy suggests wiring up the system under test differently to make the tests faster. In theory I guess if you use ports and adapters right, you could test a whole distributed-application-spanning feature by running the whole system in-process, in-memory, just for the tests. What are your thoughts about such approaches? Aslak Hellesoy's sub second acceptance tests talk can be found here: kzbin.info/www/bejne/qYbGnYuPgql8b5Y.

[@oleksiidoronin1069]

I am very curious, what would be the best practice for real-time graphics applications? Like the ones built on Vulkan or OpenGL. Would it be comparison between some rendered frame vs a previously saved one, or there is a better option?

[@defeqel6537]

At least for Vulkan, you could theoretically add a testing layer which validates that certain commands are sent for certain game input. Not sure if that is truly helpful though.

[@oleksiidoronin1069]

@@defeqel6537 Yes, it is possible. However, it goes into internal logic of the system. It's not clear what would be an acceptance test.

[@defeqel6537]

@@oleksiidoronin1069 I wouldn't say so, Vulkan layers can be loaded via configuration, and it is the external layer for your program. Of course, that is true only to an extent, as you can get similar results with very different approaches (and thus Vulkan commands). Personally, I wouldn't try to do acceptance testing for real time graphics. I'd rather test the business logic separately from the graphics layer, which allows the testing to be more isolated.

[@JinnGuild]

Unfortunately, almost every time anybody (including Dave) talks about testing, they say binaries like "What it should do, not how it does it." But I have seen a lot of discussion about "What it looks like." Either in terms of 3D Graphics, UI/UX Design, or Accessibility. Most ongoing debates about "how to test" those things cannot possibly fall under the Binary of "What it does, not how it does it." Those discussions almost always end with the concept that they require a human to validate manually. OTOH, it's important to keep in mind that, as complex as graphics and design are, there are still many levels of complex code beneath the visual part. Those things can and should be tested with either Unit or Integration Tests of some kind, as automated as possible.

[@gerelltroche]

I need better shirts

[@theTeslaFalcon]

How does any of this apply to db driven websites like php? How do u automate any of this?

[@theTeslaFalcon]

@Barney Laurence When I got out of programming in 2001, there was NO such thing as automated testing. So saying that it's the same as other tests sound like so much nonsense to me. As he talks about this automated testing stuff, it seems like he's writing custom code to test compiled programs. Since your first msg on automated testing failed, w better info maybe u can try your answer again.

[@theTeslaFalcon]

@Barney Laurence 1) ass.u.me 2) What a rabbit hole!

[@theTeslaFalcon]

@Barney Laurence I'm familiar w whitebox and blackbox testing. I'm familar w arguing w testers about schedules & programming languages. (They tried reject my 1st ASP web application because it used "too many different languages in one application".) I'm familiar w waterfall model. I'm familiar w the self-test / customer as beta testers model. I'm NOT familiar w ANYTHING he's talking about here. My "team" was usually just me to do 100% of the programming. An artist did front-end design / graphics work. An admin would maintain / configure the server & implement the DB I design. That's it.

[@yassenpetrov2303]

@@theTeslaFalcon I tend to think of it as an extension of self-test. When you develop certain functionality you tend to test it by performing sequence of discrete steps: Log in with credentials -> click 'Products' in the navigation -> click on 'Product X' -> see a message 'Product X was added to shopping cart' -> wipe out the test data. A testing framework, like Selenium for example, allow you to describe those steps in a script, and when you run it, Selenium will start a browser and do all the clicking and typing instead of you, while sounding the alarm if anything doesn't match the expected outcome. Basically the same boring, repetitive tasks are performed but by the machine instead of you. The benefit being after the acceptance tests are written and pass, you can then refactor, improve or straight redesign the functionality under test, knowing that if you mess something up in the process, the testing framework will tell you within a couple of minutes. Which is good quality of life improvement compared to a customer telling you the same thing when they clicked though the site Friday after hours... (bane of my existence)