Fedora Commons Users List for FEDORA Repository Architecture ()

Richard Green | 1 Nov 2007 18:12

Re: User group meeting London

A draft agenda, registration and joining details will be sent out in the next few days - sorry for the slight delay. To be kept in touch I suggest you subscribe to the Fedora UK and Ireland User Group mailing list at:

http://www.jiscmail.ac.uk/lists/fedora-uki.html

Richard Green
__________________________
Richard Green
Manager, RIDIR, REMAP and RepoMMan Projects
e-SIG, Academic Services
University of Hull

r.green-Cu7FvJMYcQG1Qrn1Bg8BZw@public.gmane.org
www.hull.ac.uk/ridir
www.hull.ac.uk/remap
www.hull.ac.uk/esig/repomman

From: Lodewijk Bogaards [mailto:Lodewijk.Bogaards-vFS3hOaSXjagvsoY7HRnDg@public.gmane.org]
Sent: 01 November 2007 15:50
To: Richard Green
Subject: Re: [Fedora-commons-users] User group meeting London

Hi,

As this sounds interesting to me: I would like to know more about this meeting. Can someone supply me with more information?

Thanks,

Lodewijk Bogaards

On 10/26/07 12:20 PM, "Richard Green" <R.Green-Cu7FvJMYcQG1Qrn1Bg8BZw@public.gmane.org> wrote:

Yes. Full details will be sent out on the Fedora UK&I mailing list (a JISC mail service to which you can subscribe) next week.

Richard Green

From: fedora-commons-users-bounces-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org [mailto:fedora-commons-users-bounces-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org] On Behalf Of Antony Corfield [awc]
Sent: 26 October 2007 12:14
To: fedora-commons-users-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
Subject: [Fedora-commons-users] User group meeting London

Hi, could someone please confirm if this meeting is on 22/11/07 at Birkbeck, London.

Regards,

Antony

--

Antony Corfield

ROAD Project, AU

tel. 01970 628724

*****************************************************************************************
To view the terms under which this email is distributed, please go to http://www.hull.ac.uk/legal/email_disclaimer.html
*****************************************************************************************
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Fedora-commons-users mailing list
Fedora-commons-users-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
https://lists.sourceforge.net/lists/listinfo/fedora-commons-users

*****************************************************************************************
To view the terms under which this email is distributed, please go to http://www.hull.ac.uk/legal/email_disclaimer.html
*****************************************************************************************

headers

Carl Lagoze | 1 Nov 2007 20:00

OAI-ORE Open Meeting, March 3 2008

Apologies for cross-posting

A meeting will be held on March 3, 2008 at Johns Hopkins University to
roll-out the first beta release of the OAI-ORE specifications. These
specifications describe a data model to identify and describe
aggregations of web resources, and the encoding of the data model in the
XML-based Atom syndication format.

Additional details are available at:

- The full press release for this event:

http://www.openarchives.org/ore/documents/ore-hopkins-press-release.pdf

- The registration site for the meeting:

http://www.regonline.com/oai-ore

Note that registration is required and space is limited.

Carl Lagoze and Herbert Van de Sompel
-----------------------------------------------------

Carl Lagoze

Cornell Information Science

301 College Ave.

Ithaca, NY 14850

WWW: http://www.cs.cornell.edu/lagoze

Phone: +1-607-255-6046

FAX: +1-607:255-5196

headers

Ganesh N R YANAMANDRA | 6 Nov 2007 08:57

Fedora repository threshold


Hi,

We are loading a huge amount of objects into fedora, each object has a lot
of datastreams. thus resulting in millions of records.

I would like to ask the community few questions-

a) what is the maximum threshold reached so far.

b) what impacts are forseen as regards to performance, indexing once the
threshold is reached and what is the workaround to improve performance.

c) Can fedora datastore support clustering.

regards,
Ganesh
Celebrate Singapore Library Week with us from 1 to 9 Nov 2007. Join in the
festivities as libraries around Singapore come together to bring you
exciting activities! Visit http://www.singaporelibraryweek.sg for more
details.

Win great prizes by donating or digitising your photographs, postcards,
pictures, or illustrations of historic buildings, landmarks or street
scenes of Singapore that are no longer in existence, especially those
before the 1970s. Bring them down to National Library on 10 Nov 2007
between 11am to 5pm!

Drop Everything And Read (DEAR) - Double up on your reading with double
your loan quota from 1 Nov 2007 to 31 Jan 2008! Be the person who registers
NLB's 1 millionth loan during Singapore Library Week and win attractive
prizes!

WARNING :
This e-mail and any attached files may contain privileged or confidential
information. If you are not the intended addressee, you must not disclose,
reproduce, copy, distribute, disseminate, use this communication or take
any action in reliance thereon. Communication of any information in this
e-mail to any unauthorised person may constitute an offence under the
Official Secrets Act (Cap 213). Please notify the sender immediately if you
receive this by error.

headers

John A Parker | 9 Nov 2007 17:22

Oracle Connections Not Being ReUsed???

Good morning everyone.

We are currently running fedora-2.1.1 but need to make use of inline (object level) XACML policies. As I could only get this feature of the XACML support to work using v2.2.1 (and because being current is good...), we have been attacking upgrading to v2.2.1 on multiple fronts:

- We have successfully customized a fedora.war such that it's deployable in our hosted tomcat 5 environment. (Observations and issues in a follow up note.)

- We have installed a 2.2.1 instance using a "converted ("do" table renamed to "dobj")" copy of our 2.1.1 Oracle database.

The latter installation is the subject of this note.

We have developed a set of Coldfusion -> Fedora APIs which are used extensively by our hosted Coldfusion customers. The custom API developer also put together a set of unit tests which exercise the code paths included.

This set of unit tests run successfully - both individually and as a complete set - against our 2.1.1 instance. They also run successfully individually against the 2.2.1 instance. However, at first (see "After Adjusting Oracle DB" below) when run as a set, the tests fail. The details of this failure are:

With the "out-of-the-box" configuration, in the fedora.log we could see a line similar to the following (please note the "similarity" between numActive and maxActive):

DEBUG 2007-11-08 10:22:45.077 [TP-Processor2] (ConnectionPool) Got connection from pool (fedora <at> jdbc:oracle:thin: <at> talc3.cit.cornell.edu:1521:hostingt, numIdle=0, numActive=100, maxActive=100)
DEBUG 2007-11-08 10:22:45.078 [TP-Processor2] (FieldSearchResultSQLImpl) SELECT doFields.pid FROM doFields WHERE doFields.pid = 'unittest:xargs.1'

With this configuration, when the number of active connections max'ed out the server logging stopped

We then adjusted the configuration parameter "whenExhaustedAction" to the value "2" (grow i.e., increase the size of the pool). We then cycled the server and reran the set of tests and ended up failing with:

DEBUG 2007-11-08 11:10:32.127 [TP-Processor6] (ConnectionPool) Got connection from pool (fedora <at> jdbc:oracle:thin: <at> talc3.cit.cornell.edu:1521:hostingt, numIdle=0, numActive=129, maxActive=-1)
ERROR 2007-11-08 11:10:32.127 [TP-Processor6] (FedoraAPIABindingSOAPHTTPImpl) Error finding objects
fedora.server.errors.StorageDeviceException: Error querying sql db: Io exception: Got minus one from a read call
at fedora.server.search.FieldSearchSQLImpl.findObjects(FieldSearchSQLImpl.java:298)
at fedora.server.search.FieldSearchSQLModule.findObjects(FieldSearchSQLModule.java:152)

"After Adjusting Oracle DB"

When we asked our DBA if anything was appearing in their logs which might help us debug this we were told we had swamped the database. We were also told the database was configured for a maximum of 150 processes. I had to kill fedora to free up the database so the DBA could shut it down, re-configure the max number of processes to 300 and then restart it. In the mean time I adjusted the minEvictableIdleTimeMillis value to 5000.

An aside: Was the default setting of 1800000 (30 minutes) deliberate?

In any event, with the database opened up and the max allowed idle time knocked down to 5 seconds we ran the set of unit tests again. This time the tests ran to completion...

...but reached a maximum active connections of 180 with a maximum number of idle connections of 2.

Please see attached pdf for the connection information detail.

Our sequence of unit tests are serial, and (as mentioned above) ran fine in out 2.1.1 install against the same database. Our concern is that connections are not being released back to its pool (it seems to run almost as if the code counts on idle evictions to free up connections).

Has anyone else encountered this situation? Am I configuring something wrong? (I have attached the fedora.fcfg for reference.)

Or, is this a bug in fedora.2.2.1?

All help will be gratefully appreciated.

Thanks,

John

Attachment (Fedora-2.2.1.m2.2_OracleConnections_Unit_tests.pdf): application/pdf, 73 KiB

Attachment (fedora.fcfg): application/octet-stream, 45 KiB

headers

Carol Minton Morris | 13 Nov 2007 15:52

NEWS RELEASE: Fedora Commons/Sun StorageTek (TM) 5800 System

FOR IMMEDIATE RELEASE

CONTACT:

Fedora Commons: Sandy Payette

(607) 255-9222, payette-NfByoz7J8eJ7M33PncoYlQ@public.gmane.org

Carol Minton Morris,

(607) 255-2702, clt6-HmMyXyqgL2CVc3sceRu5cw@public.gmane.org

http://www.fedora-commons.org

Fedora Commons Integrates Its Software Platform with the Sun StorageTek(TM) 5800 System

Ithaca, New York, November 13, 2007 - Fedora Commons , a non-profit organization which provides sustainable technologies to create, manage, publish, share and preserve digital information assets, announced today plans to integrate the Fedora Commons software platform with the Sun StorageTek(TM) 5800 System. This collaboration provides a substantial opportunity to advance open systems for durable access to the digital information assets, which increasingly forms the basis for our intellectual, organizational, scientific and cultural heritage.

The Fedora Commons free, open-source software platform uses a service-oriented architecture that enables the creation of collaborative, integrated information spaces where any information entity can be linked to any other entity. The StorageTek 5800 system has advanced data integrity, resilience and failure tolerance over other storage system designs, and includes customer-defined arbitrary metadata indexing and search. Customers can seamlessly scale, saving millions of dollars in administrative costs over traditional storage, and be assured of an advanced level of data availability.

During the creation or authoring of intellectual works, changes occur rapidly, but over their lifecycle these works become fixed information assets. New scholarship is built upon earlier works and new science is built upon other research. Without durable access to previous works, research progress cannot be sustained. Unless key digital assets such as datasets and analyses are reliably kept and their authenticity is guaranteed, the scientific method may be compromised and the results may be questioned. The StorageTek 5800 provides a powerful capability to handle fixed unstructured information assets.

The Fedora Commons software platform and StorageTek 5800 architecture form a natural synergy for the creation, management, use and care of fixed information. Both systems recognize that, over time, all technologies and formats change. Dan Davis, Fedora Commons Chief Software Architect, notes, "We must develop an open-ended, evolutionary, interoperable architecture, built upon open standards, to ensure that our digital heritage remains accessible and useful, now and in the future."

It is hard to imagine all the ways in which our digital information assets will be used and how knowledge will be synthesized in the future. Users and software applications must be free to use, fuse (mash-up), and repurpose information in new and innovative ways. Dan Davis observed, "The StorageTek 5800 extensive and flexible support for metadata closely integrated with the content is a natural enabler for the Fedora Commons semantic linking and dissemination capabilities. Using our offerings together will provide a straightforward, easy way to support the entire information lifecycle from creation to preservation, and to facilitate rapidly building innovative applications or mash-ups while ensuring your information is secure."

To learn more about Fedora and Sun solutions for digital content, visit http://www.fedora-commons.org and http://sun.com/storagetek/. For more information contact Carol Minton Morris at (607)255-2702.

Sun, Sun Microsystems, the Sun logo and StorageTek are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries.

--

Carol Minton Morris
Communications Director
National Science Digital Library (NSDL)

http://NSDL.org

Communications and Media Director

Fedora Commons

http://www.fedora-commons.org

Cornell Information Science
301 College Ave.
Ithaca, NY 14850
607 255-2702

clt6-HmMyXyqgL2CVc3sceRu5cw@public.gmane.org

headers

Carol Minton Morris | 13 Nov 2007 21:55