headers
Oliver Bock | 25 Jan 00:23

Semicolon URL delimiter

My URLs normally look like this:

http://www.mysite.com/v003/?_s=xx0045&_id=BK807317371

but occasionally one arrives like this:

http://www.mysite.com/v003/?_s=xx0045&_id=BK807317371;bcsi-ac-3CA58FAD43257DE1=1E08471000000005HI6/ziFwyMTzliaD2x6OYpIvEvYyAAAABQAAAK5uLgCAcAAAAQAAAPOKAAA=

Some googling revealed that the ;bcsi-ac... stuff is probably some basic authentication information added by a not-too-clever proxy.  It surprised me to also learn that the semicolon is a valid query item separator (http://stackoverflow.com/questions/3481664/semicolon-as-url-query-separator).  If WebUtils.FieldStorage had known this then my code would have happily ignored this junk and my user would have been OK.

In my view nobody should go around adding parameters to other programs' URLs, so I consider this problem the fault of whatever added that junk.  I thought I'd mention it for interest, and in case it affects anybody else.


  Oliver
------------------------------------------------------------------------------
Keep Your Developer Skills Current with LearnDevNow!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-d2d
_______________________________________________
Webware-discuss mailing list
Webware-discuss@...
https://lists.sourceforge.net/lists/listinfo/webware-discuss
Permalink | Reply | 1 comment |
headers
Christoph Zwerschke | 14 Aug 15:21
Picon
Gravatar

ANN: DBUtils 1.1 released

DBUtils 1.1 has been released.

DBUtils is a suite of tools providing solid, persistent and pooled
connections to a database that can be used in all kinds of
multi-threaded environments such as Webware for Python or other web
application servers. The suite supports DB-API 2 compliant database
interfaces and the classic PyGreSQL interface.

The new version is available for download at:

     * http://www.python.org/pypi/DBUtils/ and
     * http://www.w4py.org/downloads/DBUtils/

Changes:

     * http://www.w4py.org/DBUtils/Docs/RelNotes-1.1.html

User's Guide available at:

     * http://www.w4py.org/DBUtils/Docs/UsersGuide.html

-- Christoph Zwerschke

------------------------------------------------------------------------------
FREE DOWNLOAD - uberSVN with Social Coding for Subversion.
Subversion made easy with a complete admin console. Easy 
to use, easy to manage, easy to install, easy to extend. 
Get a Free download of the new open ALM Subversion platform now.
http://p.sf.net/sfu/wandisco-dev2dev
Permalink | Reply | 0 comments |
headers
Christoph Zwerschke | 3 Aug 23:52
Picon
Gravatar

ANN: Webware for Python 1.1 released

A new version 1.1 of the venerable Webware for Python web framework
is now available for download at
http://downloads.sourceforge.net/webware/Webware-1.1.tar.gz.

This release offers some major new features, e.g. a WSGI adapter,
sessions based on shelve or Memcached, MiddleKit supporting SQLite.

Webware requires Python 2.4 now. The code base has been cleaned up and
modernized very much; all the old cruft that was only there to support
older Python versions has been removed. The older Python versions are
still supported via the latest bugfix version 1.0.3 of the 1.0 branch
which has been released along with the current version 1.1.

You can find more info on the Webware for Python homepage at
http://www.webwareforpython.org/

Support is available via the Webware for Python mailing lists at
https://sourceforge.net/mail/?group_id=4866

-- Christoph Zwerschke

------------------------------------------------------------------------------
BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA
The must-attend event for mobile developers. Connect with experts. 
Get tools for creating Super Apps. See the latest technologies.
Sessions, hands-on labs, demos & much more. Register early & save!
http://p.sf.net/sfu/rim-blackberry-1
Permalink | Reply | 0 comments |
headers
Roger Haase | 23 Jul 07:05
Picon
Favicon

Webware times out on WebFaction Hosting service

I am a new WebFaction user trying to port an old Webware 
WebKit application.

 I have added a WebFaction "Custom app (listening on port)" to obtain a unique port number.  Then I modified
my wsgi 
adapter to point at 184.172.207.73:41759 and modified the app server 
configuration to listen at the same address.

The problem is that the wsgi adapter and app server are not making a connection and the browser always get a
timed out error. I started the AppServer manually in a putty ssh terminal the the AppServer starts but
shows no indication of receiving a request.  The AppServer startup messages include the line
"Listening for Adapter on 184.172.207.73:41759" and the address.adapter file is created with the same value.

My WSGIAdapter.config file includes:

"""
NumRetries = 20
SecondsBetweenRetries = 3
Host = '184.172.207.73'
AdapterPort = 41759
"""

Any ideas as to what I am doing wrong?

Roger Haase

------------------------------------------------------------------------------
Storage Efficiency Calculator
This modeling tool is based on patent-pending intellectual property that
has been used successfully in hundreds of IBM storage optimization engage-
ments, worldwide.  Store less, Store more with what you own, Move data to 
the right place. Try It Now! http://www.accelacomm.com/jaw/sfnl/114/51427378/
Permalink | Reply | 8 comments |
headers
Oliver Bock | 24 Mar 01:25

Mismatch between request.field() and QUERY_STRING

I have had a few requests where the values in request.fields() do not agree with what is in QUERY_STRING.  Below, see that the referrer, QUERY_STRING and REQUEST_URI all show _id=BK0000000579561338, and yet in fields we see _id=BK0000000579558176!  How can this be?  Looking at HTTPRequest.py I can see that everything comes from requestDict, which I guess is coming from the adapter (mod_webkit2).  Therefore what the adapter is sending us in requestDict['input'] does not match what we see in requestDict['environ'].  Does it seem that I am thinking in the right direction?  Has anyone else seen anything like this?

Background: I am now using 1.1b, but I have noticed this problem occasionally over the last few years.  It occurred during a period of reasonably high usage.


fields method pathInfo pathTranslated previousURLPaths queryString serverSidePath servletPath sessionId uri
CONTENT_LENGTH CONTENT_TYPE DOCUMENT_ROOT GATEWAY_INTERFACE HTTP_ACCEPT HTTP_ACCEPT_ENCODING HTTP_ACCEPT_LANGUAGE HTTP_CONNECTION HTTP_COOKIE HTTP_PRAGMA HTTP_REFERER HTTP_USER_AGENT HTTP_VIA HTTP_X_IMFORWARDS PATH PATH_INFO PATH_TRANSLATED QUERY_STRING REMOTE_ADDR REMOTE_PORT REQUEST_METHOD REQUEST_URI SCRIPT_FILENAME SCRIPT_NAME SCRIPT_URI SCRIPT_URL SERVER_ADDR SERVER_ADMIN SERVER_NAME SERVER_PORT SERVER_PROTOCOL SERVER_SIGNATURE SERVER_SOFTWARE UNIQUE_ID
83
application/x-www-form-urlencoded
/var/www/html
CGI/1.1
application/x-ms-application, image/jpeg, application/xaml+xml, image/gif, image/pjpeg, application/x-ms-xbap, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*
gzip, deflate
en-AU
keep-alive
_SID_=20110323082132-d9c44042f70ed31c95834939d79eac32
no-cache
http://www.X.com/v003/?_s=qf0044&_id=BK0000000579561338
Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0)
1.1 imelprirnweb01.infoplex.int:8080 (IronPort-WSA/7.1.0-307)
20
/sbin: /usr/sbin: /bin: /usr/bin
/
/var/www/html/
_s=qf0044&_id=BK0000000579561338
1.1.1.1
56840
POST
/v003/?_s=qf0044&_id=BK0000000579561338
/var/www/html/v003
/v003
http://www.X.com/v003/
/v003/
0.0.0.0
root <at> localhost
www.X.com
80
HTTP/1.1
<address>Apache Server at www.X.com Port 80</address>
Apache
16gr3soE47EAAHBSUgsAAAAG
_action_nextQuestion _id _s _wasQuestion
Next >>
BK0000000579558176
qf0044
ID10
'POST'
'/'
'/var/www/html/'
[]
'_s=qf0044&_id=BK0000000579561338'
'/home/timbock/v003/servlets/QuestionPage.py'
'/v003'
'200323082132-d9c44042f70ed31c95834939d79eac32'
'/v003/?_s=qf0044&_id=BK0000000579561338' 

------------------------------------------------------------------------------
Enable your software for Intel(R) Active Management Technology to meet the
growing manageability and security demands of your customers. Businesses
are taking advantage of Intel(R) vPro (TM) technology - will your software 
be a part of the solution? Download the Intel(R) Manageability Checker 
today! http://p.sf.net/sfu/intel-dev2devmar
_______________________________________________
Webware-discuss mailing list
Webware-discuss@...
https://lists.sourceforge.net/lists/listinfo/webware-discuss
Permalink | Reply | 2 comments |
headers
Steve Schwarz | 2 Mar 21:58
Favicon
Gravatar

Automatic Session Save in Transaction.py

Hi,

We ran across an interesting problem with sessions being automatically saved by Transaction.py. We had a situation where two AJAX requests were issued and one of the requests updates the session and it started and completed within the lifetime of another request. In that scenario the completion of the first request overwrites the session and the changes to the session made by the shorter request are lost. 

Here's an easier to follow timeline:
Request 1 - reads session (assume session is empty)
Request 2 - reads session (assume session is empty)
Request 2 - adds data to session
Request 2 - sleep() called - session is stored (session contains data)
Request 1 - sleep() called - session is stored (session is now empty - the value of the session when Request 1 read it)

So when later requests go to read the data stored in the session by request 2 the data isn't there.

Here's the code from WebKit.Transaction:
def sleep(self):
"""Send sleep() to the session and the servlet.

Note that sleep() is sent in reverse order as awake()
(which is typical for shutdown/cleanup methods).

"""
self._nested -= 1
self._servlet.sleep(self)
if not self._nested and self._session:
self._session.sleep(self)
self._application.sessions().storeSession(self._session)

We put a hack in the longer running AJAX page's sleep() to keep the session from being stored since it doesn't modify the session:
def sleep(self, trans):
    trans._session = None

Is there an official way to disable the automatic storage of sessions?

If not there are a number of ways to resolve this problem more generally:
- Explicit session saves by servlet - Transaction never calls storeSession()
- Explicit opt in/opt out of session storage by servlet
- Explicit opt in/opt out of session storage by AppConfig
- Extend Session functionality to keep a "dirty" flag and only automatically save the session if it has been modified during the request/response.

IMO Webware shouldn't be saving the session automatically at the end of each request. I'd rather the servlet logic explicitly decide when and if to save the session, but this would break backward compatibility. I'd be happy to supply a patch if we can decide on a way forward.

Best Regards,
Steve
Blog: http://tech.agilitynerd.com/

------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in 
Real-Time with Splunk. Collect, index and harness all the fast moving IT data 
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business 
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
Webware-discuss mailing list
Webware-discuss@...
https://lists.sourceforge.net/lists/listinfo/webware-discuss
Permalink | Reply | 6 comments |
headers
Justin Akehurst | 1 Dec 20:13
Favicon

Patch to enable log file rotation via SIGUSR1 handler

I had to hack in the ability to rotate the log files while the Appserver was still running, to avoid downtime.

This works nicely with newsyslog on BSD systems.

 

It isn't pretty, or ideal, but at least it works. Hopefully this will seed the idea of working this feature into WebWare in a future release so that I don't have to patch it on my side.

 

The patch for ThreadedAppServer.py also includes a fix for BSD systems regarding opening the socket with appropriate flags set.

 

 

--- WebKit/ThreadedAppServer.py.orig    Sun Jan 25 06:17:38 2009

+++ WebKit/ThreadedAppServer.py    Wed Jul  8 13:10:30 2009

<at> <at> -271,6 +271,19 <at> <at>

              try:

                     sock.bind(serverAddress)

                     sock.listen(1024)

+                    try:

+                          import fcntl

+                          fd = sock.fileno()

+                          flags = fcntl.fcntl(fd, fcntl.F_GETFD)

+                          try:

+                                 flags |= fcntl.FD_CLOEXEC

+                          except AttributeError, e:

+                                 # Python 2.2 does not include FD_CLOEXEC

+                                 flags |= 1

+                          fcntl.fcntl(fd, fcntl.F_SETFD, flags)

+                    except ImportError, e:

+                          # Platform does not have fcntl module

+                          pass

              except Exception:

                     print "Error: Can not listen for %s on %s" % (

                           handlerClass.settingPrefix, str(serverAddress))

<at> <at> -1310,7 +1323,11 <at> <at>

      print "App server has been signaled to shutdown."

      if server and server._running > 2:

             print "Shutting down at", asclocaltime()

+             global logtuple

+             log, stdout, stderr = logtuple

             sys.stdout.flush()

+             sys.stdout, sys.stderr = stdout, stderr

+             log.close()

             server._running = 2

             if signum == SIGINT:

                    raise KeyboardInterrupt

<at> <at> -1321,6 +1338,34 <at> <at>

      else:

             print "No running app server was found."

+def rotateLogs(signum, frame):

+      print

+      print "App server has been signaled to rotate logs."

+      if server and server._running > 2:

+             print "Rotating logs at", asclocaltime()

+             logFile = server.serverSidePath(server.application().setting('ErrorLogFilename'))

+             # attempt to close logs

+             global logtuple

+             log, stdout, stderr = logtuple

+             if log:

+                    sys.stdout, sys.stderr = stdout, stderr

+                    log.close()

+

+             try:

+                    log = open(logFile, 'a', 1) # append, line buffered mode

+                    print 'Output has been redirected to %r...' % logFile

+                    stdout, stderr = sys.stdout, sys.stderr

+                    sys.stdout = sys.stderr = log

+                    logtuple = (log, stdout, stderr)

+             except IOError, error:

+                    print 'Cannot redirect output to %r.' % logFile

+                    print error.strerror

+                    log = None

+                    sys.exit(1)

+      else:

+             print "No running app server was found."

+

+

try:

      # Use the threadframe module for dumping thread stack frames:

      # http://www.majid.info/mylos/stories/2004/06/10/threadframe.html

<at> <at> -1371,6 +1416,11 <at> <at>

      signal.signal(SIGINT, shutDown)

except AttributeError:

      SIGINT = None

+try:

+      SIGUSR1 = signal.SIGUSR1

+      signal.signal(SIGUSR1, rotateLogs)

+except AttributeError:

+      SIGUSR1 = None

 if threadDump:

<at> <at> -1392,6 +1442,7 <at> <at>

import re

settingRE = re.compile(r'^(?:--)?([a-zA-Z][a-zA-Z0-9]*\.[a-zA-Z][a-zA-Z0-9]*)=')

from MiscUtils import Configurable

+import types

 usage = re.search('\n.* arguments:\n\n(.*\n)*?\n', __doc__).group(0)

<at> <at> -1405,8 +1456,12 <at> <at>

      function = run

      daemon = False

      workDir = None

+      global logtuple

+      logtuple = None

      for a in args[:]:

-             if settingRE.match(a):

+             if isinstance(a, types.TupleType):

+                    logtuple = a

+             elif settingRE.match(a):

                    match = settingRE.match(a)

                    name = match.group(1)

                    value = a[match.end():]

--- WebKit/Launch.py.orig    Sun Jan 25 06:17:38 2009

+++ WebKit/Launch.py    Wed Jul  8 13:10:30 2009

<at> <at> -431,6 +431,8 <at> <at>

                    print 'WARNING: Applications run much slower when profiled,'

                    print 'so turn off profiling in Launch.py when you are finished.'

             else:

+                    if log is not None:

+                          args[2].append((log, stdout, stderr))

                    errorlevel = launchWebKit(*args)

      finally:

             print

 

 

Regards,

 

Justin Akehurst
Software Development Engineer II, UI
Main: (206) 315-7500   Fax: (206) 315-7501

Direct: (206) 315-7576   Mobile: (206) 790-4736
Twitter | Blog | Web | YouTube

Simple is Smart™

 

 

------------------------------------------------------------------------------
Lotusphere 2011
Register now for Lotusphere 2011 and learn how
to connect the dots, take your collaborative environment
to the next level, and enter the era of Social Business.
http://p.sf.net/sfu/lotusphere-d2d
_______________________________________________
Webware-discuss mailing list
Webware-discuss@...
https://lists.sourceforge.net/lists/listinfo/webware-discuss
Permalink | Reply | 0 comments |
headers
Oliver Bock | 2 Dec 04:14

Threads always sitting at MinServerThreads

I recently installed Webware 1.1b on a new machine, and left it at its default Min/Start/MaxServerThreads (5/10/20) in webwarework/Configs/AppServer.config.  I have observed that even when MinServerThreads are all busy, Webware is not starting new threads to handle new connections.  Looking at the code, this seems to be because Webware decides how many threads it needs based on past activity; all threads being busy does not in itself trigger the creation of new threads.  (I could be misunderstanding the code here.)

I observed it using a simple Webware script that reports the number of active threads and roughly what they are doing.  This script is normally very fast because it waits on no locks and does no database work.  However when the server is busy I have occasionally noticed a several second delay before it returns, and when it does return I see that the other threads are all busy, and that the number of threads is still just 5.  In fact, the number of threads is always just 5, so far as I can see.

Perhaps it is just that my requests come in a bad pattern: I observe that most of the Webware threads are idle most of the time, but occasionally a bunch of requests will come in together, and slow each other down trying to update the same table.  As they wait, others pile in until the database clears the backlog and everyone is finished.

I have fixed my problem by upping MinServerThreads to 20.  I also noticed that I had MinServerThreads set to 100 on my old machine (Webware 0.8.1 or thereabouts), courtesy of an incident a few years ago when some of my threads were locking up, so I suspect what I am observing is not new.

So my question is: is this behaviour normal?  Am I handling this the right way?


  Oliver
------------------------------------------------------------------------------
Increase Visibility of Your 3D Game App & Earn a Chance To Win $500!
Tap into the largest installed PC base & get more eyes on your game by
optimizing for Intel(R) Graphics Technology. Get started today with the
Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
http://p.sf.net/sfu/intelisp-dev2dev
_______________________________________________
Webware-discuss mailing list
Webware-discuss@...
https://lists.sourceforge.net/lists/listinfo/webware-discuss
Permalink | Reply | 2 comments |
headers
Christoph Zwerschke | 28 Oct 22:31
Picon
Gravatar

New server for w4py.org

Just a quick note to let you know we migrated the Webware project site 
to a new machine. The website and repositories should now be noticeably 
faster. If anything does not work as expected or you are missing 
anything that was there on the old server, let us know.

Thanks go to tummy.com for complimentary providing the new server, as 
they already did in the past. They are really generous and supportive, 
so I highly recommend their services if you are looking for managed 
hosting of your Webware apps or other Linux based services.

-- Christoph

------------------------------------------------------------------------------
Nokia and AT&T present the 2010 Calling All Innovators-North America contest
Create new apps & games for the Nokia N8 for consumers in  U.S. and Canada
$10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing
Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store 
http://p.sf.net/sfu/nokia-dev2dev
Permalink | Reply | 0 comments |
headers
Oliver Bock | 25 Oct 04:14

1.1b: 'yield' not allowed in a 'try' block with a 'finally' clause

When install.py byte compiles 1.1 beta on Python 2.4.3, I get this 
error, which does not stop install.py:

Compiling /home/timbock/Webware-1.1b1/WebKit/Adapters/WSGIAdapter.py ...
   File "/home/timbock/Webware-1.1b1/WebKit/Adapters/WSGIAdapter.py", 
line 68
     yield chunk
SyntaxError: 'yield' not allowed in a 'try' block with a 'finally' clause

I'm not worried about this for myself because I am not planning to use 
the WSGI stuff, but I thought I should raise it, since this release is 
aimed to work with Python 2.4.  (FYI: stackoverflow suggests a 
workaround 
http://stackoverflow.com/questions/2339358/workaround-for-python-2-4s-yield-not-allowed-in-try-block-with-finally-clause)

   Oliver

------------------------------------------------------------------------------
Nokia and AT&T present the 2010 Calling All Innovators-North America contest
Create new apps & games for the Nokia N8 for consumers in  U.S. and Canada
$10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing
Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store 
http://p.sf.net/sfu/nokia-dev2dev
Permalink | Reply | 2 comments |
headers
Sophana KOK | 24 Sep 20:53
Picon

bug in UnknownFileTypeServlet.py (guess_type multithread)

Hi

I tested webware 1.0.2 on python 2.6.2 (fedora 12) and came into a bug
that I could find.
from python 2.6 guess_type implementation doesn't support multi threading.
The symptom is an infinite recursion exception on guess_type when it
is called concurrently

Looking in webware trunk I could see it that the exception is catched:

            try:
                mimeType, mimeEncoding = guess_type(filename, False)
            except Exception:
                mimeType = None

I'm not sure this is a good method because python does make an
infinite recursion before sending the exception.
Maybe it is better to prevent it, just by using a mutex:

from threading import Lock
guess_type_lock=Lock()

...

with guess_type_lock:
                mimeType, mimeEncoding = guess_type(filename, False)

This works fine for me.

Thanks

------------------------------------------------------------------------------
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev
Permalink | Reply | 1 comment |

Gmane