On Mon, Jul 9, 2012 at 9:25 AM, Michail Ostrowski
<michail.ostrowski-8qTHflVfW6aELgA04lAiVw@public.gmane.org> wrote:
> Hey Phlip,
>
> I stumbled upon your paper "Integration Test Servers for Videogames".
>
>
> My master's thesis is going to be based on your work. I have replaced the
> scripting part with a simple interface wrapping api calls or user input.
> this way any non-programmer is able to create a test case.

Awesome. I honestly hope that game programmers will look at tools like
this more.

However, I'm doing 2D in-house websites for big companies now (instead
of fragile fly-by-nighters like High Moon), so I'm really out of the
loop for both the game industry and 3D unit tests.

--
Phlip
http://c2.com/cgi/wiki?ZeekLand

hanuri7421 wrote:

> hi i wanna get some information about test-driven dev for gui in c++

All desktop views are different, and all desktop views are the same.
They always allow you to declare a window and its controls in code, as
objects. So for each method that configures your view or pushes data
into these objects, there's another method that queries that same data
out of the objects. (Warning: Sometimes these methods are not fully
documented!)

My book on TFUI shows how to do this for WTL. Each step should
trivially map onto any other View platform:

http://www.zeroplayer.com/tfui/TestFirstUserInterfaces.doc

Disclaimer: TFUI is not about screen scraping, or capture playback, or
all the other heavy tools that apply "outside" the application. TFUI
is about testing views as an extension of your TDD (and BDD) test
cases.

joobn wrote:

> I live in Seoul, Republic of Korea. I'm working as Software Development Manager in ECO Inc.

Do your programmers use TDD?

sangpire

> Hi I'm Web site Developer and my concern is 'How to test web site?'.
> I can't understand English fluently. But I know this group will help me
> for get good idea for testing web site.

Ruby on Rails has the best TDD for websites (and it's fully
TFUI-compliant!) That platform is (therefor!) so popular that you will
probably find a book on it in your language.

(You could tell us what your language is, too!) However, not all those
books cover the TDD part. So you might need to use such a book as a
"stepping stone" to the English language Rails TDD tutorials.

--
  Phlip
  http://c2.com/cgi/wiki?ZeekLand

TDDers:

A while ago I invented an assertion, assert{ 2.0 }, that reflects its
expression and all intermediate values when it fails:

x = 7; y = 10; assert { x == 7 && y == 11 }
==>
Expected ((x == 7) and (y == 11)), but
(x == 7) is true
x is 7
(y == 11) is false
y is 10

Actually, that's not my assertion, it's Wrong.rb, by Alex Chaffee. His
diagnostic itself is slightly different, but note that it reflects the
variable names and values. That's the point of assert_equal() and its
ilk. You _can't_ say == in your assertion, despite the readability
benefits, because assertions must compete with debugging.

An assertion, when it fails, should work like debugging with a "local
variables watch" feature.

http://github.com/alexch/wrong

Today I successfully auditioned pytest, and it has a better version of
this technique. It leverages Python, which has different reflection
systems. Its faults look like this:

def test_function():
> assert f() == 4

E assert 3 == 4
E + where 3 = f()

Open question: What other assertion systems have user interfaces this awesome?

Oh, also, with pytest...

A> you don't need any icky { } around your statement -
just write assert! and...

B> even asserts in the middle of your code reflect their values.

Get it here: http://pytest.org/assert.html

--
  Phlip
  http://c2.com/cgi/wiki?ZeekLand

http://ftvt.allew.com

http://zeroplayer.com/tfui/TestFirstUserInterfaces.pdf

It addresses these industry abuses:

- skipping the View layer in TDD
- testing the View thru a user-emulator (such as Watir!)
- writing the View code first, then testing it (a little!)

The goal is new test cases drive new View features, without looking at or
driving the View too often.

That is a tall hill to scale, hence the book is very dense. However, each
chapter and case study contains its own Best Practices that can be learned
alone, just by reading them.

Work with the book inspired my subsequent research with RoR, including
assert{ 2.0 }, assert_xhtml{}, and assert_rjs_. An alternate Web chapter is
my O'Reilly paper on TDD for Ajax: http://oreilly.com/catalog/9780596510657

--
Phlip

[sorry about the premature submit!]

TFUI:

Testing views is not always about testing thru-the-view, like capture-playback
tests, or Watir or Selenium tests, that drive user interfaces.

It's about writing unit tests directly into the layer that runs the view. To get
there, you must treat controls as objects, and learn to bypass your
GUI's event loop.

One of the leading examples of TFUI is Ruby on Rails, and its exemplary support
for unit tests HTML views. When a unit test collects the HTML that would have
traveled out the web server, it can parse it to spot the programmatic details.
In RoR, anything you write into an HTML template with <%= %>, you should
generally specify with a test. This "MockTheServer" pattern bypasses the web
server's and browser's event loops.

This pattern allows your tests to lead development. You can write them first, to
force your view to upgrade. That achieves TFUI.

So now we come to Django, another fine web platform that both learns from and
teaches RoR a thing or two about pragmatism. Its developer test rig allows us to
simulate page generation, collect the HTML, and parse it back.

In theory. I have only begun to study Django, but I suspect it doesn't have the
equivalent of assert_select(), to fetch up HTML elements via CSS selector. (And
it certainly has no assert_xpath() or assert_xhtml()!;)

Do any Djangoists in the house have any confessions to make here? any explainin'
to do? or any code samples I have missed? I have yet to find any difference
between today's pyunit and the one I used in 2002!

--
Phlip
http://c2.com/cgi/wiki?ZeekLand

• New Members 1

TFUI:

Testing views is not about testing thru-the-view, like a capture-playback test,
or a Watir or Selenium test that drives a user interface.

It's about writing unit tests directly into the layer that runs the view. To get
there, you must treat controls as objects, and generally learn to bypass your
GUI's event loop.

One of the leading examples of TFUI is Ruby on Rails, and its exemplary support
for unit testing HTML views. When a unit test

Do any Djangoists in the house have any confessions to make here? any exp

--
Phlip

• New Members 1

> Posted by: "Carlos Ble" ble.jurado <at> gmail.com carlosble

> I've just read the wikipedia entry for Presenter First patter:
> http://en.wikipedia.org/wiki/Presenter_First
>
> To read about TDD there is nice but I don't see why TDD is related to
> Presenter First more than MVC or MVP.

WikiPedia might possibly be ... (mumble) ... incorrect. It has been known to
happen before. It's certainly just an aberration.

> They are all basically the same. One can test drive a controller too.
> I don't remeber hearing about TDD in the
> presenter first shows with Ron Jacobs.

If "presenter first" is derived from "Humble Dialog Box", it's not cannon
(meaning "officially correct"). It's just a really good idea, sometimes.

> A paragraph within the article says:
> "A customer will generally explain features in terms of 'when'
> statements - for example, "When I click the 'save' button then the
> file should be saved and the unsaved file warning should disappear."

To TDD that, write a test that changes a view's "modified" flag, possibly by
modifying its document, then assert that the view activates the "modified" icon.
If, for example, it highlights the Save button, then the test can assert that
the code selected the highlit icon behind that button.

Then a related test can modify the file, simulate a click to the Save button
(usually by calling the handler method that Save would have called). Then the
test asserts the file on disk has changed (yes, sometimes tests detect
filesystem side-effects!). Then the test checks that the button changed its
icon, and that the "modified" flag is down.

> This is wrong again to me. Customers should not decide how that UI
> works but what they need. User stories should not
> be attached to user interfaces I think.

Use Cases should not attach to user interfaces. The early User Stories are like
Use Cases, because the Onsite Customer is not a real Usability Designer, and
should not play one on TV.

Later User Stories will easily refer to changes to a UI that already exists, so
they do not work like Use Cases.

--
Phlip

Lisa Crispin wrote:

> We definitely want to do these investigative efforts. But I still feel
> examples are a good way to start. What have other folks found on their
> projects?

I have to confess a low rate of adoption for my assert_xpath project, to
unit-test raw HTML. As a computer scientist and a DSL-zealot, I _like_ writing
inhumanly elaborate queries, like '//fieldset[ following-sibling::input[ <at> name =
"etc" ] ]'.

Someone on the net recently challenged a forum to write an assertion like that,
but in this format, and I did it:

assert_xhtml do
form <at> action do
fieldset do
label 'first name'
input :name => 'etc'
end
end
end

Easier to read, huh? The HTML element names are all raw Ruby; they call
method_missing(), to convert into XPath expressions like that, on the fly.

That forum was the RSpec mailing list, and this leads to an interesting
observation. RSpec, and Behavior Driven Development, were _supposed_ to be about
the customer tests. Yet these days the general Ruby on Rails community now
considers TDD (and the test/unit library) to be OLD SCHOOL!

All the new, cool projects use RSpec. It appears that programmers like to
specify their designs by example, too.

Here's that assertion's RSpec version:

<at> response.body.should be_html_with{
form <at> action do
fieldset do
label 'first name'
input :name => 'etc'
end
end
}

Like all good examples, those assertions skip over irrelevant details, such as
intervening <table>, <div>, <li> tags, or whatever.

(Those of you with Ruby can try those by grabbing the assert2 gem; then calling
require 'assert2/xhtml'.)

--
Phlip
http://www.zeroplayer.com/

Hendrik Belitz wrote:

> Although I agree with that, I think that unit tests are not the answer
> to detect UI faults. Indeed, Integration and acceptance tests provide
> such capabilities in a much more usable manner, without significantly
> loosing test coverage.

Tests also help you go faster, not just detect faults. Consider this example:

<div style='color: red'>
<div style='background-color: green'>

To assert that your resulting color scheme is nauseating, you might just test
you have a red div followed by one with a green background.

You do not render that - physically, or virtually - then assert the contained
text appears red-on-green. You just assert you have two divs.

Now suppose a careless programmer changed their order:

<div style='background-color: green'>
<div style='color: red'>

The browser would render that correctly, and the user would see no bug. But the
test would fail - it is "hyperactive" - and that is a Good Thing. The test
allowed programmers to rapidly develop, knowing that if any test fails
unexpectedly they have the option to revert, without even debugging to figure
out _why_ the tests failed.

You can't go fast unless your car has brakes and your road has traffic lights!

--
Phlip
http://www.zeroplayer.com/

extremists:

HTML is easy to test and hard to test well. A unit test must intercept and parse
it, hopefully _before_ it goes thru a web server.

I just wrote an assertion which turns that problem on its head. Instead of
asserting how to parse that HTML, we will assert how to build an example of HTML
that the production HTML must match.

This post is the high-level, language-agnostic, overview of these announcement
posts:

http://www.ruby-forum.com/topic/181145#792950

Here's the example:

user = users(:Moses)
get :edit_user, :id => user.id # fetches a web page

assert_xhtml do

form :action => '/users' do
fieldset do
legend 'Personal Information'
label 'First name'
input :type => 'text',
:name => 'user[first_name]'
:value => user.first_name
end
end

end

That's all. Inside the assertion's block, you declare the HTML you want, using a
Builder notation. (Nokogiri, in Ruby's case.)

The assertion skips over any intervening elements, such as <ul>, <table>, etc,
that the block did not build. In the above example, <fieldset> only needs to
appear somewhere inside the <form> - not necessarily its first element. But
those <label> and <input> fields must appear somewhere inside that <fieldset>,
and in the given order!

To create this assertion in your language, start with your nearest HTML builder.
Allow the user-programmer to build an example, then traverse its DOM, and
convert each item into an XPath seeking that item in the production HTML.
Declare a match if all the specified attributes match, and if most of the text
matches - disregarding blanks.

Then, for each match, recursively match each built child element, applying its
XPath relative to the current node in the production HTML's DOM.

When you find a fault, build an error message by rendering the current nodes to
HTML, and return this out of your loop statements. This Gist explicates the
algorithm:

http://gist.github.com/76136

The algorithm essentially matches two trees, depth-first and in-order. The
example HTML must be a subset of the production HTML.

If you survive all that recursing and matching then your assertion passes. You
verified a huge number of details, using extremely lean code, and you ignored
many more details, allowing them to change freely as you upgrade your website.

--
Phlip
http://www.zeroplayer.com/