aidy lewis wrote:

> True, but if we implement story or scenario driven development then
> these tests lead development. A change in the tests means a change in
> development - not the other way round! I know for sure you are not
> saying; not automate through the browser and validate user
> requirements?

Two big points:

- the first tier of tests must cover as many features as possible, and
be super-fast to run

- test cases must be as close as possible to the tested code.

Put them together, and you should only test GUI things thru the GUI. Never test
biz logic only thru the GUI.

These rules imply "acceptance tests" are _not_ "thru the GUI tests". Pro testers
must work the test rig itself to add their coverage.

--
Phlip

•  1
  New Members

Pekka Laukkanen wrote:

> 1) How do you automate testing browser compatibility through the
> business layer? Or do you plan to do that manually for all the
> browsers on all operating systems?

Then you are testing the GUI directly - the test case is close to the tested case.

Then, if such a test fails (conceptually or literally), you fix it by changing
code in your real unit tests. For example, suppose your HTML accidentally
duplicates IDs - which is forbidden - and automated (or manual) testing
discovers your Javascript does not get the right object thru its ID on some
browsers.

You naturally improve the GUI tests in that area. But the bug is not fixed until
you have improved the unit tests. By "unit" I mean tests that...

- mock the server, so no HTTP or browser get in the way
- interpret the HTML that would have gone out the wire
- parse the HTML with queries, such as XPath
- assert the HTML does not contain various tell-tale signatures

That way, you improve the confidence that a developer's instant unit test
successfully defends behavior real browsers depend on - even if test finds that
actual behavior inconceivable.

So an HTML test could validate entire pages, such as with Tidy, to fail if any
IDs get duplicated. Then an XPath test could find the specific IDs that matter -
no matter where they moved on a page - and could verify their internal details.

> 2) How do you automate full end-to-end tests if you only talk to
> business layer? End users don't really care if the business logic
> works flawlessly if the UI to it doesn't work.

Sometimes a GUI test covers business logic. I took care not to say they never
should. But, once again, there had better be a unit-level test that covers the
exact same business cycle.

> For me the ideal situation is where requirements can be expressed in
> natural language, and executed as acceptance tests against different
> interfaces by changing the driver that is used. This way same
> requirements/tests can be executed both against the business layer
> (fast, might be ready before the UI) and the actual user interface
> (real end-to-end test). In web-testing world this can even be extended
> so that same tests can be executed using a browser driver (e.g.
> Selenium RC, Watir) or a driver working on HTTP/HTML level (e.g.
> HTMLUnit).

Awesome idea - I didn't even think of it like that.

I suspect that some people abuse GUI tests to cover internal business logic.
That's an antipattern...

--
Phlip

•  1
  New Members

TFUI:

How many here are using XPath to test into HTML, to detect its generated details?

I think the book /Working Effectively with Legacy Code/ would call that "sensing"...

--
Phlip
http://www.oreillynet.com/ruby/blog/2008/09/polygamous_assert_latest.html

George Dinwiddie wrote:

> Phlip wrote:

>> I thought the industry would support more XPath testing by now. What libraries
>> are out there (besides mine?)?

> HtmlUnit, for one. You won't like it, though, as it's written in Java.

That wouldn't be very politic of me!

Here's the first hit:

http://www.google.com/codesearch?q=package%3Ahtmlunit+xpath

htmlunit-plugin-0.3-src/.../plugin/htmlunit/HtmlUnitFunctionTagTest.java

75: functionTag.navigate("file:./tst/html/tables.html");
String xpath = "//head[title='tables page']";
functionTag.assertXPathMatches(xpath);

85: MockHtmlUnitHelper helper = setMockHelper();
String xpath = "//head[title='tables page']";
functionTag.clickElementWithXPath(xpath);

Firstly, where did the <at> go? That should be <at> title = 'tables page' in the predicate.

Next, I want to nest them. Suppose we have a report table which must contain a
certain HD. I want this:

assert_xpath :table, :report do # hits //table[ <at> id='report']
assert_xpath 'tbody/tr/td[ . = "SubTotal" ]'
end

The do-end lets us avoid writing a run-on XPath expression. And if the inner
assert_xpath fails, its error diagnostic will only print out its current context
- the outer <table> - not the whole page.

Then you can stack up more assertions inside the block. The goal is not to
mirror your HTML in the tests, but to efficiently skip over the fuzzy details
that might change.

--
Phlip