headers
Kent Johnson | 1 Jun 2006 02:09
Gravatar

Re: XML Parsing Woes.

doug shawhan wrote:
> Having been dragged kicking and screaming into the fussy, fussy world of 
> XML, I find myself in a pickle.
> 
> I keep getting an error complaining of a missing end-tag:
> 
>  : XML Parse error.
> <BR>XML Error Text: "; nested exception is:
>         org.xml.sax.SAXParseException: The element type "Description" 
> must be terminated by the matching end-tag "</Description>".".
> 
> Now of course, I have saved my output and found the end-tag to be there. 
> I have stripped out everything I can think of that might cause the 
> parser to puke. The string enclosed by the <Description> tags varies 
> from a few characters to a couple of thousand: all fail with the same error.
> 
> I am very new to XML and find it's jargon fairly impenetrable. Is there 
> a handy python module that can one can use to scan XML output for 
> errors? I know the mistake is probably mine ... or perhaps I have 
> actually run on to a bug in the remote application? :-) (Riiiiight!)

Both Internet Explorer and Firefox do a good job of displaying XML and 
showing any errors.

XML tags are case sensitive, make sure the case matches on the start and 
end tags. And of course spelling counts ;)

XML tags must nest properly, both of these will cause an error, one of 
them possibly the error you are seeing:
<Desc>text<Foo>more text</Desc>
(Continue reading)

Permalink | Reply | 0 comments |
headers
GNULinuxGeek | 1 Jun 2006 04:01
Picon

Another fine mess

All,

I posted some time back for another task.

Have a new task now.  You were all so kind with advice that I thought I 
might check my thoughts on this.

Being an old guy and a newbie to Python is the double-whammy, but here goes:

I get a CSV file with ~26 fields and ~7000 records.  This data needs to 
be parsed based on specific field values.

So, here is my "draft" set of steps that I think need to be performed.

   1. Select the "file" of data
   2. Clean the data with "strip" to make sure there is no extraneous
      whitespace
   3. Write the file back to a temp file.
   4. On the new file, select records based on the value of some of the
      fields (like a "sort on" in a spreadsheet.
   5. Parse to obtain the matching records
   6. Do a little math based on how many records and the data in one
      field (success vs. fail)
   7. Output the results of the math so it can be used in a spreadsheet
      to make cute graphics.

We have a PERL guy in the division, but I am trying to use Python only 
to get this done.  My question is

"What is a good way to read and process the lines of the CSV file?"
(Continue reading)

Permalink | Reply | 1 comment |
headers
Kent Johnson | 1 Jun 2006 04:22
Gravatar

Re: Another fine mess

GNULinuxGeek wrote:
> All,
> 
> I posted some time back for another task.
> 
> Have a new task now.  You were all so kind with advice that I thought I 
> might check my thoughts on this.
> 
> Being an old guy and a newbie to Python is the double-whammy, but here goes:
> 
> I get a CSV file with ~26 fields and ~7000 records.  This data needs to 
> be parsed based on specific field values.

Do you know about the csv module? It will take care of all the parsing 
and turn each line into a list of values. You can filter these as you 
like and make a list of lists of row values. This can be sorted and 
otherwise processed. I don't think you will need a temp file, it sounds 
like you can easily fit your data in memory.

Kent

> 
> So, here is my "draft" set of steps that I think need to be performed.
> 
>    1. Select the "file" of data
>    2. Clean the data with "strip" to make sure there is no extraneous
>       whitespace
>    3. Write the file back to a temp file.
>    4. On the new file, select records based on the value of some of the
>       fields (like a "sort on" in a spreadsheet.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Tracy R Reed | 2 Jun 2006 03:00

implementing config files

Hello all!

I am writing some code to implement a bunch of passive checks for the 
nagios network monitoring system. It runs some checks on the local 
machine and reports the results back to the nagios server using rpc 
calls via Perspective Broker from the Twisted module. This code and 
associated config file is distributed to all of the machines in my 
infrastructure by cfengine. This part all works quite well. Now I need 
to refactor my code into a more general and flexible infrastructure and 
provide a way to configure the services being checked instead of hard 
coding them (as they currently are).

I need to implement a config file which will provide hostnames, the 
names of checks to run on those hosts, and the options to those checks. 
These things are sets which are nested inside each other which we can 
think of like nested objects. I could make a dictionary containing the 
host names each of which is a dictionary containing the services to be 
checked etc.

But rather than just make dictionaries of dictionaries (which could get 
confusing) I was wondering if I could make it more robust by somehow 
defining objects nested inside of other objects in the config file with 
certain attributes. For example I would like to be able to have a 
check_tcp object which would have two attributes: hostname and port. If 
you try to assign it anything else you get an error. If port isn't a 
number between 0 and 2^16 you get an error. Etc. Basically I don't want 
errors in the config file to propagate all the way to the client machine 
on which the code is going to be executed and have wrong arguments 
passed to the program which we os.popen() and read the results from.
(Continue reading)

Permalink | Reply | 2 comments |
headers
Carlos Daniel Ruvalcaba Valenzuela | 2 Jun 2006 08:36
Picon
Gravatar

Re: implementing config files

Your could try to use XML files to store configuration files, I
already coded something like that, using expat parser and loading the
XML contents to objects and attributes, this is a sample code of how
works my module:

Lets supouse we have this file config.xml with the following contents:
<?xml version="1.0" encoding="ISO-8859-1"?>
<config>
        <connection>
                <server>MySQL</server>
                <host>localhost</host>
                <port>21</port>
                <user>username</user>
                <passwd></passwd>
        </connection>
</config>

and in our code:

from objxml import *

fd = file('config.xml', 'r')
p = XMLParser(fd)

root = p.Root
port = str(root.connection.port)
user = str(root.connection.username)

All nodes are objects, converting them to strings gets you the
content, you can also access the atributes as normal object
(Continue reading)

Permalink | Reply | 0 comments |
headers
linda.s | 2 Jun 2006 11:29
Picon

Re: a question about symbol

On 5/28/06, Bob Gailer <bgailer <at> alum.rpi.edu> wrote:
>
>  linda.s wrote:
>  When I test the following code,
> I got something like (use 80 as argument):
> 80?F=27?C
> Why '?' appear?
>
> # code
> import string, sys
>
> # If no arguments were given, print a helpful message
> if len(sys.argv)==1:
>  print 'Usage: celsius temp1 temp2 ...'
>  sys.exit(0)
>
> # Loop over the arguments
> for i in sys.argv[1:]:
>  try:
>  fahrenheit=float(string.atoi(i))
>  except string.atoi_error:
> print repr(i), "not a numeric value"
>  else:
> celsius=(fahrenheit-32)*5.0/9.0
> print '%i\260F = %i\260C' % (int(fahrenheit), int(celsius+.5))
>
>  On my computer I get the desired result. I paste it here 80°F = 27°C and I
> see degree symbols.
>
>  What operating system / terminal hardware are you using?
(Continue reading)

Permalink | Reply | 1 comment |
headers
Kent Johnson | 2 Jun 2006 11:51
Gravatar

Re: a question about symbol

linda.s wrote:
> On 5/28/06, Bob Gailer <bgailer <at> alum.rpi.edu> wrote:
>>  linda.s wrote:
>>  When I test the following code,
>> I got something like (use 80 as argument):
>> 80?F=27?C
>> Why '?' appear?
>>
>> # code
>> print '%i\260F = %i\260C' % (int(fahrenheit), int(celsius+.5))
>>
>>  On my computer I get the desired result. I paste it here 80°F = 27°C and I
>> see degree symbols.
>>
>>  What operating system / terminal hardware are you using?
>>  --
>> Bob Gailer
>> 510-978-4454
> 
> mac and terminal.

\260 represesents the character with octal value 260, hex B0. In Latin-1 
and Unicode this is a degree sign. My guess is that your terminal is set 
to display UTF-8 characters rather than latin-1; in UTF-8 B0 by itself 
is an error. It's also possible it is set to MacRoman but in that case I 
think you would see an infinity sign instead of a question mark.

 From the Python interpreter prompt, type
import sys
sys.stdout.encoding
(Continue reading)

Permalink | Reply | 0 comments |
headers
Kent Johnson | 2 Jun 2006 12:08
Gravatar

Re: implementing config files

Tracy R Reed wrote:
> I need to implement a config file which will provide hostnames, the 
> names of checks to run on those hosts, and the options to those checks. 
> These things are sets which are nested inside each other which we can 
> think of like nested objects. I could make a dictionary containing the 
> host names each of which is a dictionary containing the services to be 
> checked etc.
> 
> But rather than just make dictionaries of dictionaries (which could get 
> confusing) I was wondering if I could make it more robust by somehow 
> defining objects nested inside of other objects in the config file with 
> certain attributes. For example I would like to be able to have a 
> check_tcp object which would have two attributes: hostname and port. If 
> you try to assign it anything else you get an error. If port isn't a 
> number between 0 and 2^16 you get an error.

One possibility is to make the config file be an actual Python module. 
Then it can instantiate objects directly. The class constructors can do 
as much checking as you like. Your config file might look like this:

import testdefs

host1 = testdefs.Host('192.168.0.53', '80')
host2 = testdefs.Host('192.168.0.55', '27')
tests = [
   testdefs.check_tcp(host1),
   testdefs.check_http(host1),
   testdefs.check_tcp(host2),
   testdefs.check_ftp(host2),
]
(Continue reading)

Permalink | Reply | 0 comments |
headers
Carroll, Barry | 2 Jun 2006 19:45

Re: implementing config files

Carlos:

Where does one find the objxml module?  I have looked on python.org and
Google and can't find it.  

Thanks in advance,

Barry
barry.carroll <at> psc.com
541-302-1107
________________________
We who cut mere stones must always be envisioning cathedrals.

-Quarry worker's creed

> -----Original Message-----
> Date: Thu, 1 Jun 2006 23:36:27 -0700
> From: "Carlos Daniel Ruvalcaba Valenzuela" <clsdaniel <at> gmail.com>
> Subject: Re: [Tutor] implementing config files
> To: "Tracy R Reed" <treed <at> ultraviolet.org>
> Cc: tutor <at> python.org
> Message-ID:
> 	<4fae7dfa0606012336k1999a7e7w3c5a290b4d413272 <at> mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
> 
> Your could try to use XML files to store configuration files, I
> already coded something like that, using expat parser and loading the
> XML contents to objects and attributes, this is a sample code of how
> works my module:
>
(Continue reading)

Permalink | Reply | 0 comments |
headers
Carroll, Barry | 3 Jun 2006 01:23

Trouble with os.path.isfile

Greetings:

 

One of the functions of my test system web server program is to allow the user to view the contents of the image file directories and select files to load onto the hardware.  Obviously, a central part of this function is to identify and display the names of directories and files in a specified directory.  I haven’t been able to find any canned code or cookbook entries that do this, so I’m making one from scratch.  Here is the helper function that gets and returns the  directory and file names:

 

>>>>>>>>>>>>>>>>>>>>>>>>>>

 

def lstdirsnfiles(tdir):

    flst = os.listdir(tdir)

    flst.sort()

    fnames = []

    dnames = ['.', '..']

    for name in flst:

        absfn = os.path.join(tdir, name)

        if os.path.isdir(absfn):

            dnames.append(name)

        elif os.path.isfile(absfn):

             fnames.append(name)

    return dnames, fnames

>>>>>>>>>>>>>>>>>>>>>>>>>>

 

 

The root of the image file directory tree is “/home/iip/images”.  When lstdirsnfiles is run with tdir set appropriately,  the lists returned contain no additional information: dnames = ['.', '..'] and fnames = [].  I added some tracing statements to see what was happening inside the  function:

 

>>>>>>>>>>>>>>>>>>>>>>>>>>

 

def lstdirsnfiles(tdir):

    global DEBUG, dbgfname

    DEBUG = True

   

    flst = os.listdir(tdir)

    flst.sort()

    fnames = []

    dnames = ['.', '..']

   

    if DEBUG:

        dbgf = open(dbgfname,mode="a")

        dbgf.write("\n"+str (time.localtime())+"\n")

        dbgf.write("Entering lstdirsnfiles\n")

        dbgf.write("The directory = %s\n" % tdir)

        dbgf.write("The file list = %s\n" % flst)

 

    for name in flst:

        absfn = os.path.join(tdir, name)

   

        if DEBUG:

            dbgf.write("File path = %s, isfile = %s\n" %

                           (absfn, os.path.isfile(absfn)))

 

        if os.path.isdir(absfn):

            dnames.append(name)

        elif os.path.isfile(absfn):

             fnames.append(name)

   

    if DEBUG:

        dbgf.write("dnames = %s\n" % dnames)

        dbgf.write("fnames = %s\n" % fnames)

        dbgf.close()

    DEBUG = False

   

    return dnames, fnames

>>>>>>>>>>>>>>>>>>>>>>>>>>

 

 

The log vile received the following:

>>>>>>>>>>>>>>>>>>>>>>>>>>

 

(2006, 6, 2, 15, 23, 4, 4, 153, 1)

Entering lstdirsnfiles

The directory = /home/iip/images

The file list = ['test1.bmp', 'test2.bmp', 'test3.bmp', 'test4.bmp', 'test5.bmp', 'test6.bmp']

File path = /home/iip/images/test1.bmp, isfile = False

File path = /home/iip/images/test2.bmp, isfile = False

File path = /home/iip/images/test3.bmp, isfile = False

File path = /home/iip/images/test4.bmp, isfile = False

File path = /home/iip/images/test5.bmp, isfile = False

File path = /home/iip/images/test6.bmp, isfile = False

dnames = ['.', '..']

fnames = []

>>>>>>>>>>>>>>>>>>>>>>>>>>

 

 

So there are entries in the directory, but the function doesn’t recognize them as files. 

 

Finally, I opened the interpreter and tried doing this, as nearly as possible, interactively.  I got this:

 

>>>>>>>>>>>>>>>>>>>>>>>>>>

 

>>> import os

>>> tdir = '/home/iip/images'

>>> os.listdir(tdir)

['test4.bmp', 'test3.bmp', 'test2.bmp', 'test1.bmp', 'test5.bmp', 'test6.bmp']

>>> flst = os.listdir(tdir)

>>> flst.sort()

>>> flst

['test1.bmp', 'test2.bmp', 'test3.bmp', 'test4.bmp', 'test5.bmp', 'test6.bmp']

>>> for name in flst:

...     absfn = os.path.join(tdir,name)

...     print absfn, "is",

...     if not os.path.isfile(absfn):

...         print "NOT",

...     print "a file"

...

/home/iip/images/test1.bmp is a file

/home/iip/images/test2.bmp is a file

/home/iip/images/test3.bmp is a file

/home/iip/images/test4.bmp is a file

/home/iip/images/test5.bmp is a file

/home/iip/images/test6.bmp is a file

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

 

 

As you can see, the interpreter correctly identifies the strings in the list as names of files, but my function does not.  I can’t see anything wrong with the logic in the function.  Can any or you see the problem I’m missing?

 

As always, thanks in advance for your help.

 

Barry

barry.carroll <at> psc.com

541-302-1107

________________________

We who cut mere stones must always be envisioning cathedrals.

—Quarry worker's creed

 

_______________________________________________
Tutor maillist  -  Tutor <at> python.org
http://mail.python.org/mailman/listinfo/tutor
Permalink | Reply | 0 comments |

Gmane