FlexMonkey brings unit testing to Flex user interface developers
In Stu Stern’s recent InfoQ.com article on UI testing and FlexMonkey, he explains the code-a-little-test-a-little (CALTAL) approach to test driven development with user interface development and walks you through using FlexMonkey to support the theory.
The practice of maintaining automated unit test suites has gained widespread acceptance over the past decade to the point where most developers today either engage in some amount of test writing or at least feel bad for not doing it. This rise in automated unit testing has led to some confusion about who should be testing what. Should developers strive for 100% code coverage with their unit tests, and if so, does that mean we no longer need dedicated QA testers? Many development teams draw the line at the user interface, reasoning that since user interfaces can be exercised with little or no programming they can be tested more economically by dedicated testers, either manually or with specialized testing tools. This division of labor has led many to divide the world of testing into “unit testing” and “functional testing”, with developers providing the former and QA testers providing the latter. In this article we’ll explore how Gorilla Logic’s new, open source Flex user interface automation testing tool, FlexMonkey, can enhance the productivity of both developers and QA testers. FlexMonkey allows developers to incorporate user interface testing into unit test suites and continuous integration environments, and allows QA testers to expand those unit tests to create and maintain comprehensive quality tests.
Enjoy the full article at: http://www.infoq.com/articles/flexmonkey-ui-unit-testing
FlexMonkey is a great thing, but the title here is wrong. Flex Monkey is a functional testing tool, not a unit testing tool. Those are very different things and the confusion of the two lead a lot of developers down the wrong path.
@Michael Labriola
Mike,
In the article, I deliberately chose to use the term “unit” to describe FlexMonkey testing, since I wanted to emphasize that FlexMonkey is a tool for developers as well as QA testers, and many developers bristle at any suggestion that they do any testing other than “unit”.
Having said that, and as I mention in the article, I do believe that “developer” testing and “QA” testing is a very real and useful distinction, but the distinction between “functional” and “unit” testing has always seemed to me to be more semantic than practical. Although tests can of course be categorized according to whether that which is being tested is “developer-facing” or “user-facing”, what is the substantive consequence of such a distinction?
More specifically, what is the fundamental difference between a test that exercises an API, and one that exercises a UI? I routinely create test suites that combine both UI and API testing, and never find myself having to stop to consider whether a particular test is a “unit” or “functional” one. Again, what would be the substantive consequence of making such a distinction?
Thanks for your comment! I hope you will elaborate!
-Stu
I’m very excited about FlexMonkey–as a Flex consultant the lack of a good answer to the automated functional testing problem has been a recurrent problem–but I have to go with Michael on this one. There certainly is a grey area between unit and functional testing, but it’s not all grey. For example, I often find myself writing code that transforms data structures (usually to and from remote services), and I write tests that tell me whether my transformations are working correctly.
Further, I often find that I can test a great many more scenarios that I or other testers are likely to encounter when using the application (for example, I may be testing against an optional API that may rarely or never be used).
That kind of a test is clearly different than a test performed by interacting with the application via its interface. Another distinction is that a user interface-driven test script will rely on the application state, whereas the other kind (I’m struggling to not use problematic terms) is typically isolated, often requiring fixtures to provide a snapshot of operational conditions, but not requiring that the application has actually been exercised via its interface to get it into the right state.
So perhaps “unit” and “functional” aren’t good terms, but they are widely used, and it seems to me that there are clear cases where what would normally be considered a “unit test” is not something that FlexMonkey is designed to address.
All that aside, a big tip of the hat to you and your team, and I’m looking forward to giving it a shakedown run in the next week or two.
Regards,
Chris Luebcke
Hi Guys,
Thanks for the comments!
I don’t think we are completely throwing away the distinction between unit and functional testing, but we did step into religious ground for a purpose. We certainly understand the differentiation the industry makes between these two activities, but generally believe that it is over emphasized – particularly in user interface development.
Let’s look at two examples. In the first, I am writing hibernate code to do a custom search or filter. In the second, I am building a complex custom Flex component with a creative layout. The first example lends itself perfectly to a classic unit test. The second, however, is difficult to effectively test exclusively through a classic unit test. Certainly parts of the component can be exercised this way, but it IS hard to achieve the same level of testing with a hand coded unit test, as one can with a pure unit test on the Hibernate code.
In our Flex consulting work, this reality has led us to break testing activities out into two major categories: Developer and QA. We aren’t suggesting that all “developer testing” should be done through FlexMonkey. However, we do believe that the majority of cases that a user interface developer encounters lend themselves toward more functional tools, such as FlexMonkey. I can assure you that we believe that FlexUnit and Fluint are still essential tools for a Flex developer, whether the development work lends towards writing more classic unit tests or one is attempting to automate their test (FlexMonkey and non-FlexMonkey developed tests).
Anyhow, Let us know what you think the Monkey once you give it the “shakedown.”
- Jon ROse
Gorilla Logic, Inc.