I don’t think we are completely throwing away the distinction between unit and functional testing, but we did step into religious ground for a purpose. We certainly understand the differentiation the industry makes between these two activities, but generally believe that it is over emphasized – particularly in user interface development.

Let’s look at two examples. In the first, I am writing hibernate code to do a custom search or filter. In the second, I am building a complex custom Flex component with a creative layout. The first example lends itself perfectly to a classic unit test. The second, however, is difficult to effectively test exclusively through a classic unit test. Certainly parts of the component can be exercised this way, but it IS hard to achieve the same level of testing with a hand coded unit test, as one can with a pure unit test on the Hibernate code.

In our Flex consulting work, this reality has led us to break testing activities out into two major categories: Developer and QA. We aren’t suggesting that all “developer testing” should be done through FlexMonkey. However, we do believe that the majority of cases that a user interface developer encounters lend themselves toward more functional tools, such as FlexMonkey. I can assure you that we believe that FlexUnit and Fluint are still essential tools for a Flex developer, whether the development work lends towards writing more classic unit tests or one is attempting to automate their test (FlexMonkey and non-FlexMonkey developed tests).

Anyhow, Let us know what you think the Monkey once you give it the “shakedown.”

- Jon ROse
Gorilla Logic, Inc.

Further, I often find that I can test a great many more scenarios that I or other testers are likely to encounter when using the application (for example, I may be testing against an optional API that may rarely or never be used).

That kind of a test is clearly different than a test performed by interacting with the application via its interface. Another distinction is that a user interface-driven test script will rely on the application state, whereas the other kind (I’m struggling to not use problematic terms) is typically isolated, often requiring fixtures to provide a snapshot of operational conditions, but not requiring that the application has actually been exercised via its interface to get it into the right state.

So perhaps “unit” and “functional” aren’t good terms, but they are widely used, and it seems to me that there are clear cases where what would normally be considered a “unit test” is not something that FlexMonkey is designed to address.

All that aside, a big tip of the hat to you and your team, and I’m looking forward to giving it a shakedown run in the next week or two.

Regards,
Chris Luebcke

Mike,

In the article, I deliberately chose to use the term “unit” to describe FlexMonkey testing, since I wanted to emphasize that FlexMonkey is a tool for developers as well as QA testers, and many developers bristle at any suggestion that they do any testing other than “unit”.

Having said that, and as I mention in the article, I do believe that “developer” testing and “QA” testing is a very real and useful distinction, but the distinction between “functional” and “unit” testing has always seemed to me to be more semantic than practical. Although tests can of course be categorized according to whether that which is being tested is “developer-facing” or “user-facing”, what is the substantive consequence of such a distinction?

More specifically, what is the fundamental difference between a test that exercises an API, and one that exercises a UI? I routinely create test suites that combine both UI and API testing, and never find myself having to stop to consider whether a particular test is a “unit” or “functional” one. Again, what would be the substantive consequence of making such a distinction?

Thanks for your comment! I hope you will elaborate!

-Stu