headers
Pedro Ferreira | 10 Feb 10:49
Picon
Picon
Gravatar

Server-side caching

Hello all,

A (possibly silly) question: does ZEO have some kind of server-side 
cache? I mean, each time an oid is requested by one of the clients is it 
retrieved from the DB file directly, or are some of the objects kept in 
memory? From what I see in the code, the latter doesn't seem to happen.

I know there are client-side caches, but in a multiple client/server 
context I wonder if it's not faster to ask the DB for an oid that is 
already in memory instead of retrieving it from the client cache?

Thanks in advance,

Pedro

--

-- 
José Pedro Ferreira

Software Developer, Indico Project
http://indico-software.org

+-----------+
+  '``'--- `+  CERN - European Organization for Nuclear Research
+ |CERN|  / +  1211 Geneve 23, Switzerland
+ ..__. \.  +  IT-CIS-AVC
+  \\___.\  +  Office: 513-1-005
+      /    +  Tel. +41227677159
+-----------+
_______________________________________________
For more information about ZODB, see http://zodb.org/
(Continue reading)

Permalink | Reply | 1 comment |
headers
Marius Gedminas | 9 Feb 17:32
Gravatar

Re: zeopack error

On Thu, Feb 09, 2012 at 02:26:10AM +0100, Kaweh Kazemi wrote:
> Thanks a lot for the investigation, Marius. That was awesome.
> 
> Seems to me, I need to upgrade to Python 2.7 and be done with the
> problem then. :/

I wonder, though, if ZODB ought to use the 'noload' method of
cPickle.Unpickler, which is undocumented and broken on 2.6 (and gone on
3.x, AFAIU).

> > Incidentally, since you use cross-database references, please make sure
> > they continue to work after you pack your storage.  I've lost data that
> > way (the ZODB garbage collector doesn't see references that exist in
> > other storages, and can assume objects are garbage when it shouldn't).
> > Packing with GC disabled ought to be safe.
> 
> Cross-database references are only one-way in our implementation.
> "users" storage is referencing objects in the "game" storage - never
> vice versa / no circles. Also we make sure we never delete any objects
> in *any* storage, just "disable" them on an application level. This
> worked fine so far. Are you seeing any problems with this?

I don't use cross-database references.  My only encounter with them was
on Zope 2.x: I uploaded a new version of a website from a .zexp file
into /temp_folder, tested it briefly there, then used the ZMI to move it
into /.  It worked fine for a day, and was broken the next day.  Turns
out /temp_folder in Zope 2 uses a different backend storage, and moving
the folder to root just created a cross-db reference, but didn't move
the objects across.  Then the garbage collector for the /temp_folder
storage noticed that my site was not directly reachable from
(Continue reading)

Permalink | Reply | 3 comments |
headers
Kaweh Kazemi | 8 Feb 13:24
Gravatar

Re: zeopack error

Hi there,

Recap: last week I examined problems I had packing our 4GB users storage. With Martijn's help I was able to
fix zeo's exception output and write out the first broken pickle that throws an exception. During my
checks I realized that running the pack in a Python 2.7 environment (using the same ZODB version - 3.10.3)
works fine, the pack reduces our 4GB storage to 1GB. But our production server uses Python 2.6 (same
ZODB3.10.3) which yields the problem (though the test had been done on OS X 10.7.3 - 64bit, and the
production server is Debian Squeeze 32bit).

Currently I see only three ways to resolve that problem:

	- Upgrade production server from Python 2.6 to Python 2.7 which would probably solve the current problem
(if it's really a Python 2.6/2.7 problem).

	- Pack the storage with Python 2.7/ZODB3.10.3 on the working system and deploy it back to the Python 2.6
production environment - very time consuming (because I have to download/pack/upload a big storage and
during that time the production server wouldn't be available to avoid changes during that time)/
potentially risky(?).

	- Find out what is broken regarding that pickle and possibly the Python 2.6/ZODB3.10.3 combination and
deploy a fix.

You can download the broken pickle from here: http://www.reversepanda.com/download/brokenpickle

If someone has more experience in parsing and understanding pickles in regards to ZODB3, any help would be appreciated.

Kind regards,
Kaweh
_______________________________________________
For more information about ZODB, see http://zodb.org/
(Continue reading)

Permalink | Reply | 5 comments |
headers
Kaweh Kazemi | 30 Jan 11:22
Gravatar

zeopack error

Hi zodb-dev,

during our latest zeopack runs, zeopack for our users storage fails giving this error:

Traceback (most recent call last):
  File
"/usr/local/lib/python2.6/dist-packages/ZODB3-3.10.3-py2.6-linux-i686.egg/ZEO/scripts/zeopack.py",
line 159, in _main
    cs.pack(packt, wait=True)
  File
"/usr/local/lib/python2.6/dist-packages/ZODB3-3.10.3-py2.6-linux-i686.egg/ZEO/ClientStorage.py",
line 916, in pack
    return self._server.pack(t, wait)
  File
"/usr/local/lib/python2.6/dist-packages/ZODB3-3.10.3-py2.6-linux-i686.egg/ZEO/ServerStub.py",
line 155, in pack
    self.rpc.call('pack', t, wait)
  File
"/usr/local/lib/python2.6/dist-packages/ZODB3-3.10.3-py2.6-linux-i686.egg/ZEO/zrpc/connection.py",
line 768, in call
    raise inst # error raised by server
TypeError: unhashable type: 'list'
Error:
Error packing storage users in ('localhost', 9100)

Any ideas how to start to debug this behavior to find the bad data structure that prevents zeopack to run
correctly? There is no verbose mode with zeopack as far as I know that would provide me with more helpful
information (like the transaction id?). Maybe I can use any of the other tools to get to the source of the problem?

Any experience you could share to help me find the bad structure is very much appreciated. Thanks for your help.
(Continue reading)

Permalink | Reply | 11 comments |
headers
steve | 24 Jan 14:50
Gravatar

[X-Post] Figure out bottle neck in a repoze.bfg based web app

Hi All,

I apologize for the cross-post but by this mail I simply hope to get a few
pointers on how to narrow down to the problem I am seeing. I shall post to the
relevant list if I have further questions.

So here is the issue:

Short description:
I've got a repoze.bfg application running on top of zeo/zodb across multiple
servers, served using mod_wsgi and it's showing bad resource usage (both high
memory consumption as well as CPU usage). Are there any steps i can do to
localise whether this is an issue with zeo/zodb/mod_wsgi configuration, and/or
usage ?

Long description:

* I have a repoze.bfg (version 1.3) based app, which uses zodb (over zeo,
version 3.10.2) as the backend and is served up using apache+mod_wsgi. All
running on a minimal debian 6.0 based amazon instances.

* The architecture is 1 zodb server and 4 app instances running on individual
EC2 instances (all in the same availability zone). All of the instances are
behind an amazon Elastic Load Balancer

* At the web-server, we don't customize apache much (ie: we pretty much use the
stock debian apache config). We use mod_wsgi (version 3.3-2) to serve the
application in daemon mode, with the following parameters:

WSGIDaemonProcess webapp user=appname threads=7 processes=4
(Continue reading)

Permalink | Reply | 9 comments |
headers
tsmiller | 28 Dec 03:36

Can iterate thru records - but cannot access same record directly by key


I have a problem that occurs sporadically, and I do not have a clue why it
happens.  When I restart my cherrypy python server using ZODB3 I cannot
access records that were previously written to the database.  The ZODB3
database works fine for a long time (weeks) and then it seems that something
internal happens to the ZODB3 and the saved changes start screwing up. The
data exists because I can iterate through it and print out the records, but
if I try to access the record directly via the key - then I get a KeyError.

Please see the code below and its output.  My keys are all character and in
this case, all of my records with an item id greater than 6198 exist in the
database, but I cannot access them.  Can someone put me on a path to
enlightenment!

thanks,

tom

	base = db.dataConn["1292289873"]["books"]["1292289873"]
	print
	print "Length of Data Conn", len(base)
	print "Account Key", db.dataConn["1292289873"]
	print "Store Type", db.dataConn["1292289873"]["books"]
	print "Store Key", db.dataConn["1292289873"]["books"]["1292289873"]
	print
	for k, v in base.items():
		if int(k) > 6195:
			try:
				print k, type(k), base[k]
				print
(Continue reading)

Permalink | Reply | 2 comments |
headers
Chris Withers | 21 Dec 19:35
Picon

where does 32it vs 64bit matter?

Hi All,

I have a set of ZODBs, they're currently served from a 64-bit mysql 
cluster running relstorage.

At the moment, all the clients are 32-bit. At some point, those may be 
changing to be 64-bit.

What are the implications of this?
Do all relstorage clients need to be the same bit width?
What about FileStorage and ZEO connections?

Are there any migration steps or minimum ZODB/RelStorage versions needed 
to make this step?

cheers,

Chris

--

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
             - http://www.simplistix.co.uk
_______________________________________________
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev <at> zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev
Permalink | Reply | 4 comments |
headers
Eugene Morozov | 19 Dec 10:50
Picon
Gravatar

Removing "undeleteable" objects from ZODB

Hello,
I have a very old Zope instance (it was upgraded from Zope 2.4 to Zope 2.7 and later to Zope 2.13) that has a number of problems. There're instances of products that no longer exist, for example. They can't be removed by using any means. They can't be removed from Zope user interface (some of them aren't even visible). They can't be removed from python prompt as described in this document:
http://zodb.org/documentation/articles/multi-zodb-gc.html

The simplest script to replace broken object with some pristine object doesn't work:

replace_obj = OrderedFolder('removeme')
replace_obj._p_oid = p64(BROKEN_OBJECT_ID)
replace_obj._p_jar = app._p_jar
app._p_jar._register(replace_obj)
app._p_jar._added[p64(BROKEN_OBJECT_ID)] = replace_obj

This code always fails because with the following exception:
ConflictError: database conflict error (oid 0x0db8, class OFS.OrderedFolder.OrderedFolder, serial this txn started with 0x00 1900-01-01 00:00:00.000000, serial currently committed 0x03384864da015019 2000-11-05 00:36:51.094952)

As far as I understand ConflictError is raised because transaction code always attempts to read broken object and compare it with the new object. I couldn't find any working way to remove such broken objects from the database and they often get in the way. :(

Is there some other way to remove them?
Eugene
_______________________________________________
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev <at> zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev
Permalink | Reply | 1 comment |
headers
Jim Fulton | 15 Dec 17:51
Favicon

RFC: Remove KeyError as base class of POSKeyError

POSKeyError subclasses KeyError.  While this makes sense conceptually,
it's really unfortunate in practice.  Application code that catches
key errors when getting an item from a container, will unintentionally
hide POSKeyErrors.

I propose to remove KeyError as a base class of POSKeyError.  While
this is a backward incompatible change, I can't imagine unbroken code
that intentionally catches POSKeyErrors via KeErrors.

If there are no objections, I'll make this change in 3.11.

JIm

--
Jim Fulton
http://www.linkedin.com/in/jimfulton
_______________________________________________
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev <at> zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev
Permalink | Reply | 2 comments |
headers
Raoni Castro | 9 Dec 22:49
Picon

Question about zeo client cache

Hi all,


Sorry if I'm asking this in the wrong place.

I've a production site running in ha with plone 4.1.0 with zeo running in other machine. The site was  installed from unified installer and my blobstorage cache started to grow too much.

I saw that version 4.1.8 of plone.recipe.zope2instance provided zeo-client-blob-cache-size, so I updated this recipe to this version in the development instance for testing and ran buildout with the following configuration changes (in the zeo client):

base.cfg:
[instance]
zodb-cache-size = 5000
..
zeo-client-cache-size= 8 MB
zeo-client-blob-cache-size = 15 MB
.. 
blob-storage = ${buildout:directory}/var/filestorage/blobstorage
shared-blob-dir = off

But, the folder ${buildout:directory}/var/filestorage/blobstorage already holds more than 28 MB.

Have I done something wrong? Can someone please point me to the right path?

Thanks in advance,
Raoni Castro.
_______________________________________________
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev <at> zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev
Permalink | Reply | 6 comments |
headers
Jim Fulton | 9 Dec 16:51
Favicon

RFC: reduce ZEO default min-disconnect-poll to 1 and default max-disconnect-poll count to min-disconnect-poll.

ZEO client storages have parameters, min-disconnect-poll and
max-disconnect-poll that control how often they try to reconenct when
disconnected.  They start trying every min-disconnect-poll seconds and
gradually increate the wait times up to max-disconnect-poll seconds.

A backoff strategy like this is typically used when there is a
competition for resources. There's no such competition here.  Also,
attempts to reconnect aren't very expensive.

I propose to change the default min-disconnect-poll and
max-disconnect-poll to 1 (adjusting the latter if someone provides a
min-disconnect-poll > >).

If there are no objections, I'll make this change in 3.11.

Jim

--
Jim Fulton
http://www.linkedin.com/in/jimfulton
_______________________________________________
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev <at> zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev
Permalink | Reply | 0 comments |

Gmane