About the twill Web scripting language ()

James Cameron | 8 Apr 2009 01:16

Re: reraise_unmasked_exceptions()

On Thu, Mar 19, 2009 at 07:27:07PM +0000, John J Lee wrote:
> On Thu, 19 Mar 2009, James Cameron wrote:
> [...]
> > ValueError: invalid literal for int() with base 10: '"1"'
> [...]
> 
> This was fixed in mechanize 0.1.10 (I don't know which version, if any, 
> of twill that corresponds to).

Bypassing twill-sh and using mechanize directly allowed me to create an
exception handler that would ignore this condition.  I'm happy.
Mechanize is still 0.1.7b on Debian.  Also, Debian's Twill uses Debian's
mechanize, not the mechanize inside Twill.  I think.

--

-- 
James Cameron
http://ftp.hp.com.au/sigs/jc/

headers

C. Titus Brown | 8 Apr 2009 03:43

Re: reraise_unmasked_exceptions()

On Wed, Apr 08, 2009 at 09:16:33AM +1000, James Cameron wrote:
-> On Thu, Mar 19, 2009 at 07:27:07PM +0000, John J Lee wrote:
-> > On Thu, 19 Mar 2009, James Cameron wrote:
-> > [...]
-> > > ValueError: invalid literal for int() with base 10: '"1"'
-> > [...]
-> > 
-> > This was fixed in mechanize 0.1.10 (I don't know which version, if any, 
-> > of twill that corresponds to).
-> 
-> Bypassing twill-sh and using mechanize directly allowed me to create an
-> exception handler that would ignore this condition.  I'm happy.
-> Mechanize is still 0.1.7b on Debian.  Also, Debian's Twill uses Debian's
-> mechanize, not the mechanize inside Twill.  I think.

mechanize will be updated in twill 0.9.2, coming Any Day Now.

cheers,
--titus
--

-- 
C. Titus Brown, ctb@...

headers

Tennis Smith | 17 Apr 2009 01:46

authentication problem

Hi,

I'm trying to test a website that requires an id/password. When trying to use twill, I get this response:

>> show
<html>
<head><title>401 Authorization Required</title></head>
<body bgcolor="white">
<center><h1>401 Authorization Required</h1></center>
<hr><center>nginx/0.6.35</center>
</body>
</html>

I have a legit id/pswd on the system, but can't figure out how to pass them via twill.

Anyone have any ideas?

-Tennis

_______________________________________________
twill mailing list
twill@...
http://lists.idyll.org/listinfo/twill

headers

James Cameron | 17 Apr 2009 01:59

Re: authentication problem

On Thu, Apr 16, 2009 at 04:46:12PM -0700, Tennis Smith wrote:
>    I have a legit id/pswd on the system, but can't figure out how to pass
>    them via twill.
>    Anyone have any ideas?

How do you normally present the username and password?

If it is via a pop-up authentication dialog box, then the usual methods
such as twill-sh's add_auth should work.

If if is via a username and password prompt on a web page, then you have
to direct twill to fill in the form.  You will either receive a cookie,
or a session ID in a redirection.

Everybody does it differently.

--

-- 
James Cameron
http://ftp.hp.com.au/sigs/jc/

headers

Tennis Smith | 17 Apr 2009 02:16

Re: authentication problem

On Thu, Apr 16, 2009 at 4:59 PM, James Cameron <james.cameron-VXdhtT5mjnY@public.gmane.org> wrote:

On Thu, Apr 16, 2009 at 04:46:12PM -0700, Tennis Smith wrote:
> I have a legit id/pswd on the system, but can't figure out how to pass
> them via twill.
> Anyone have any ideas?

How do you normally present the username and password?

If it is via a pop-up authentication dialog box, then the usual methods
such as twill-sh's add_auth should work.

If if is via a username and password prompt on a web page, then you have
to direct twill to fill in the form.

Yes, I have to fill in a form. Can you point me to an example of how to do that? THanks. :)

You will either receive a cookie,
or a session ID in a redirection.

Everybody does it differently.

--
James Cameron
http://ftp.hp.com.au/sigs/jc/

_______________________________________________
twill mailing list
twill-zFNWeKVCUJJD5a/XDn3G2A@public.gmane.org
http://lists.idyll.org/listinfo/twill

_______________________________________________
twill mailing list
twill@...
http://lists.idyll.org/listinfo/twill

headers

James Cameron | 17 Apr 2009 02:20

Re: authentication problem

On Thu, Apr 16, 2009 at 05:16:00PM -0700, Tennis Smith wrote:
>    Yes, I have to fill in a form.  Can you point me to an example of how
>    to do that? THanks. :)

Sure.

My internet service provider has a web page to tell me the current
usage, and it works by prompting in a form, then issuing a cookie that
is used to validate further requests.

For this I used twill-sh in a shell script as follows:

twill-sh <<EOF >/dev/null 2>/dev/null
go https://example.com/usage/daily/
fv Login username quozl
fv Login password qwertyuiop
submit
save_html /var/tmp/usage.html
EOF

The remains of my script processed the usage.html file to get the byte
counts.

--

-- 
James Cameron
http://ftp.hp.com.au/sigs/jc/

headers

Tennis Smith | 17 Apr 2009 17:22

Re: authentication problem

Grrrr... it turns out the auth process is putting up a separate window with userid/password fields in it.

Can twill deal w/that?

-T

On Thu, Apr 16, 2009 at 5:20 PM, James Cameron <james.cameron-VXdhtT5mjnY@public.gmane.org> wrote:

On Thu, Apr 16, 2009 at 05:16:00PM -0700, Tennis Smith wrote:
> Yes, I have to fill in a form. Can you point me to an example of how
> to do that? THanks. :)

Sure.

My internet service provider has a web page to tell me the current
usage, and it works by prompting in a form, then issuing a cookie that
is used to validate further requests.

For this I used twill-sh in a shell script as follows:

twill-sh <<EOF >/dev/null 2>/dev/null
go https://example.com/usage/daily/
fv Login username quozl
fv Login password qwertyuiop
submit
save_html /var/tmp/usage.html
EOF

The remains of my script processed the usage.html file to get the byte
counts.

--

James Cameron
http://ftp.hp.com.au/sigs/jc/

_______________________________________________
twill mailing list
twill-zFNWeKVCUJJD5a/XDn3G2A@public.gmane.org
http://lists.idyll.org/listinfo/twill

_______________________________________________
twill mailing list
twill@...
http://lists.idyll.org/listinfo/twill

headers

C. Titus Brown | 17 Apr 2009 21:27

Re: authentication problem

On Fri, Apr 17, 2009 at 08:22:38AM -0700, Tennis Smith wrote:
-> Grrrr... it turns out the auth process is putting up a separate window with
-> userid/password fields in it.
-> 
-> Can twill deal w/that?

Hey Tennis,

is it doing that with JavaScript or with browser authentication, do you
know?

cheers,
--titus

-> On Thu, Apr 16, 2009 at 5:20 PM, James Cameron <james.cameron@...> wrote:
-> 
-> > On Thu, Apr 16, 2009 at 05:16:00PM -0700, Tennis Smith wrote:
-> > >    Yes, I have to fill in a form.  Can you point me to an example of how
-> > >    to do that? THanks. :)
-> >
-> > Sure.
-> >
-> > My internet service provider has a web page to tell me the current
-> > usage, and it works by prompting in a form, then issuing a cookie that
-> > is used to validate further requests.
-> >
-> > For this I used twill-sh in a shell script as follows:
-> >
-> > twill-sh <<EOF >/dev/null 2>/dev/null
-> > go https://example.com/usage/daily/
-> > fv Login username quozl
-> > fv Login password qwertyuiop
-> > submit
-> > save_html /var/tmp/usage.html
-> > EOF
-> >
-> > The remains of my script processed the usage.html file to get the byte
-> > counts.
-> >
-> > --
-> > James Cameron
-> > http://ftp.hp.com.au/sigs/jc/
-> >
-> > _______________________________________________
-> > twill mailing list
-> > twill@...
-> > http://lists.idyll.org/listinfo/twill
-> >

-> _______________________________________________
-> twill mailing list
-> twill@...
-> http://lists.idyll.org/listinfo/twill

--

-- 
C. Titus Brown, ctb@...

headers

James Cameron | 20 Apr 2009 00:26

Re: authentication problem

On Fri, Apr 17, 2009 at 08:22:38AM -0700, Tennis Smith wrote:
>    Grrrr... it turns out the auth process is putting up a separate window
>    with userid/password fields in it.
>    Can twill deal w/that?

The way I deal with that using twill is to find the URL of the separate
window, and begin with that URL instead of the other.

--

-- 
James Cameron
http://ftp.hp.com.au/sigs/jc/

headers

Tennis Smith | 20 Apr 2009 20:26

Doing "showlinks" on every page

Hi,

Is there a way to visit each page in a website and do 'showlinks"? I'm testing a website and want to get a page-by-page listing of links so I can test all the combinations.

Thanks,
-T

_______________________________________________
twill mailing list
twill@...
http://lists.idyll.org/listinfo/twill

Mon	Tue	Wed	Thu	Fri	Sat	Sun

		1	2	3	4	5
6	7	8	9	10	11	12
13	14	15	16	17	18	19
20	21	22	23	24	25	26
27	28	29	30

1	November 2012
3	October 2012
1	April 2012
4	March 2012
3	February 2012
9	October 2011
6	September 2011
1	July 2011
1	June 2011
1	April 2011
14	February 2011
6	January 2011
4	November 2010
6	October 2010
1	August 2010
8	July 2010
5	June 2010
13	May 2010
3	April 2010
16	March 2010
19	February 2010
9	January 2010
7	December 2009
6	November 2009
6	October 2009
9	September 2009
3	July 2009
6	June 2009
6	May 2009
18	April 2009
14	March 2009
8	February 2009
4	January 2009
8	December 2008
2	November 2008
11	October 2008
13	September 2008
4	August 2008
19	July 2008
10	April 2008
5	March 2008
4	February 2008
26	January 2008
25	December 2007
12	November 2007
16	October 2007
22	September 2007
23	August 2007
28	July 2007
13	June 2007
4	May 2007
13	April 2007
26	March 2007
9	February 2007
24	January 2007
10	December 2006
29	November 2006
36	October 2006
16	September 2006
41	August 2006
34	July 2006
18	June 2006
46	May 2006
32	April 2006
123	March 2006
49	February 2006
17	January 2006
42	December 2005
90	November 2005
8	October 2005