Stephen Bayliss | 27 Jan 11:20

[fcrepo-dev] Fedora validation enhancements - FCREPO-1026

I've made some validation enhancements as per https://jira.duraspace.org/browse/FCREPO-1026, these are currently in the fcrepo-1026 branch on GitHub.  Some documentation is in the Fedora 3.6 documentation space at https://wiki.duraspace.org/display/FEDORA36/Validation
 
I've some questions on how far to take this, so feedback is welcomed.
 
The current implementation:
 
* allows configuration of the XML ingest validation via a new DOManager fedora.fcfg parameter (with a suitable warning in the documentation about decreasing the level of validation)
* allows all objects to be validated when they are modified, with the API operation being failed if the resulting object would be invalid
 
Object validation is configured via spring (see doobjectvalidator.xml in the server/config/spring directory) - by default it is turned off, so out-of-the-box there's no performance hit.  This feature enables for instance ECM validation to be turned on for every object modification to enforce repository content conformance with the CModel specification via ECM.  Certainly this isn't for everyone, but there are use cases.  Custom validators can be written and added that validate the Java Fedora object (rather than the XML).  Any number of validators can be added, these will execute in turn until (if) one fails.
 
Questions and thoughts:
 
* HTTP response code for REST API operations:  Currently if an ingest fails XML validation this is reported via HTTP status code 500 (Server Error).  To maintain consistency with the existing behaviour, object validation failures will also result in this code, with the text of the exception containing details of the validation failure.  I'd suggest that maybe 400 - Bad Request [1] might be more appropriate for both of these; but this would essentially represent a REST API change - would that be acceptable for a Fedora 3.6 release?  If this change was made I'd suggest implementing this by catching ObjectValidityException at the API level, and extending this exception to contain details of the validation failure for the response body (rather than the 500 exception reporting that occurs currently).
 
* Validate API method.  Currently this performs the ECM validation as it did in previous releases.  This could be modified to perform object validation as specified in the spring config for this - would this make sense?  It should be configurable so that custom validation can be plugged into the validate API method *without* enforcing validation on object commital of course.
 
* Comments on the implementation and code in that branch are most welcome
 
Thanks
Steve
 
 
------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
_______________________________________________
Fedora-commons-developers mailing list
Fedora-commons-developers@...
https://lists.sourceforge.net/lists/listinfo/fedora-commons-developers
Favicon

[fcrepo-dev] JMeter notes

On the committers' call this morning, we investigated the use of Apache JMeter to begin performance
testing for Fedora (the lack of which has been keenly felt in the community and amongst developers). Some
of the issues that arose included:

* Best practices for connecting JMeter to Fedora's various integration points
* Integrating performance testing with the extant build machinery
* Modularizing and parameterizing tests for best usability
* The important distinction between single-threaded load and multi-threaded load
* Interpreting test results to achieve actionable insight into Fedora's performance

I've begun a skeletal wiki page at:

https://wiki.duraspace.org/display/FCREPO/JMeter+notes

to coagulate info as it comes about evolving plans to use JMeter. Anyone with knowledge or opinions is
welcome to contribute! 

---
A. Soroka
Online Library Environment
the University of Virginia Library

------------------------------------------------------------------------------
Keep Your Developer Skills Current with LearnDevNow!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-d2d
Stephen Bayliss | 25 Jan 16:29

[fcrepo-dev] Ingest performance testing

Following the call timing test that Chris did a little while ago [1, 2] I have now had chance to have a quick play-around with YourKit myself.
 
Initial results from some not-very-extensive tests confirm the findings that Chris made - I was seeing that the "prepare" of the stylesheet for the Schematron validation taking around 25% of DefaultManagement.ingest total time (so this is a little lower than Chris' results, but still significant).
 
So should we just go ahead and raise an issue for this?
 
I was planning on taking a look at FCREPO-1026 and as part of this can explore caching of the prepared stylesheets as it is related (and including a configuration option to disable Schematron validation entirely).  I was also planning taking a look at FCREPO-1029, FCREPO-1030 and FCREPO-1031 (and possibly FCREPO-1027) as part of this piece of work as a lot of them are in the same code area.
 
Any thoughts welcomed!
 
Steve
 
 
------------------------------------------------------------------------------
Keep Your Developer Skills Current with LearnDevNow!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-d2d
_______________________________________________
Fedora-commons-developers mailing list
Fedora-commons-developers@...
https://lists.sourceforge.net/lists/listinfo/fedora-commons-developers
Andreas Nef | 25 Jan 13:56
Picon
Gravatar

[fcrepo-dev] XAM/CAS-Storage

Dear all

I was wondering whether someone has continued the question raised a couple of years ago regarding
content-addressable storage (CAS) or the XAM interface. I'd be happy getting to know the road taken or –
if it has not been done yet – whether there are general requirements from the community for it to be done.

Best,

Andreas

------------------------------------------------------------------------------
Keep Your Developer Skills Current with LearnDevNow!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-d2d
frank asseg | 24 Jan 16:30
Picon
Gravatar

[fcrepo-dev] Testability issues

Hola Guys,

as foreseen by you guys i ran into trouble when creating a unit test for 
FCREPO-1047 (https://jira.duraspace.org/browse/FCREPO-1047).There seemes 
to be a lot of static stuff going on in Object creation. E.g. 
DOTranslationUtility which has a whole static block that goes on to call 
the static method Server.getConfig(), which tries to load fedora.fcfg 
from the filesystem. This of course results in a IOException in a unit 
testing environment and prevents mocking of dependent objects.

Do you guys think it's feasible to refactor these classes in order to 
improve unit testing ability?

--

-- 
*frank asseg*
softwareentwicklung
sebastian-kneipp-strasse 14a
79104 freiburg i. brsg.
tel.: ++49-761-88790001
fax.: ++49-761-88790002
mail: frank.asseg@...
web: http://www.congrace.de/

------------------------------------------------------------------------------
Keep Your Developer Skills Current with LearnDevNow!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-d2d
Chris Wilper | 10 Jan 06:14
Favicon
Gravatar

[fcrepo-dev] [Ann] Committer meetings moved to Thursday

Hi all,


The latest round of votes are in, and it's official: Starting this week, Fedora Committer meetings will now be held each Thursday at 9am US Eastern time. Of all the days and times proposed, this worked for the largest group.

As always, these are open meetings and all are welcome to dial in. More about these meetings can be found here:


Thanks,
Chris
------------------------------------------------------------------------------
Write once. Port to many.
Get the SDK and tools to simplify cross-platform app development. Create 
new or port existing apps to sell to consumers worldwide. Explore the 
Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join
http://p.sf.net/sfu/intel-appdev
_______________________________________________
Fedora-commons-developers mailing list
Fedora-commons-developers@...
https://lists.sourceforge.net/lists/listinfo/fedora-commons-developers
Stephen Bayliss | 4 Jan 09:17

Re: [fcrepo-dev] Status of trippi src?

These branches have been merged back into master, and release 1.5.5 tagged.
Also a 1.5.4 tag has been added for the version released with Fedora 3.5
(which was the same as 1.5.3).  Fedora master has also been updated to use
this Trippi version.

This is based on Mulgara 2.1.12.

Mulgara and Trippi artefacts (and dependencies not hosted elsewhere) are now
uploaded to m2.duraspace.org.

As has been pointed out to me, the documentation at Sourceforge is
significantly out-of-date (it refers to Trippi 1.1.2) as are the release
notes.  Should we be moving this documentation to the duraspace wiki (as
well as updating it!)?

http://trippi.sourceforge.net/ 

Steve

> -----Original Message-----
> From: Stephen Bayliss
[mailto:stephen.bayliss@...] 
> Sent: 12 December 2011 14:57
> To: fedora-commons-developers@...
> Subject: Re: [fcrepo-dev] Status of trippi src?
> 
> 
> Thanks Ben.  Yes that test code is the only place where the 
> RIOTripleIterator is being constructed.
> 
> Let's discuss on the call tomorrow to clarify any remaining 
> issues, and I can then merge and release.  I'll need help 
> from someone in terms of the duraspace-hosted maven artefacts 
> and how to do this (I have revised the existing Mulgara 2.1.4 
> POM to 2.1.11, and installed that in my local repo for 
> testing alongside the other artefacts not available on central).
> 
> Steve
> 
> > -----Original Message-----
> > From: Benjamin Armintor [mailto:armintor@...]
> > Sent: 12 December 2011 13:00
> > To: fedora-commons-developers@...
> > Subject: Re: [fcrepo-dev] Status of trippi src?
> > 
> > 
> > Re: The rebuilder shutdown bug: I don't think so; Spring
> > handles the shutdown of the ExecutorService.
> > 
> > Re: Constructors: Your approach is totally reasonable, though
> > the Factory is more my intent.  I'm honestly not sure that 
> > integration test is doing the right thing instantiating its 
> > own RIOTripleIterator,
> > though: I don't see any other place in the codebase that 
> > happens.  I guess that's testing client code?
> > 
> > - Ben
> > 
> > On Mon, Dec 12, 2011 at 3:27 AM, Stephen Bayliss
> > <stephen.bayliss@...> wrote:
> > > Hi Ben
> > >
> > > Thanks for the clarification on this. It sounds like we 
> are not too
> > > far off tagging 1.5.4 of trippi from fix_executor; and 
> moving on to 
> > > 1.5.5 for Mulgara 2.1.11.
> > >
> > > What I'm not sure about is the rebuilder shutdown bug -
> > > https://jira.duraspace.org/browse/FCREPO-946 - related to 
> > > https://jira.duraspace.org/browse/FCREPO-873.  Is there still 
> > > something that needs implementing to deal with this for 
> > Trippi 1.5.4
> > > (ie in the fix_executor branch)?
> > >
> > > I don't believe there is a default constructor.  I wasn't perhaps
> > > entirely clear - the constructor change - addition of the 
> > > ExecutorService -- was affecting compilation of the Fedora 
> > integration
> > > tests; I modified these at
> > > 
> > 
> https://github.com/fcrepo/fcrepo/commit/41b26a3c6ff26da6738e78481c57de
> > > 1cf30292b7#diff-1 - maybe this code should be using
> > > TripleIteratorFactory.defaultInstance().fromStream(...) instead?
> > >
> > > Steve
> > >
> > >
> > > -----Original Message-----
> > > From: Benjamin Armintor [mailto:armintor@...]
> > > Sent: 11 December 2011 20:37
> > > To: fedora-commons-developers@...
> > > Subject: Re: [fcrepo-dev] Status of trippi src?
> > >
> > > Steve,
> > > This is a blunder on my part... that branch should be ready
> > to merge
> > > (it was the change that was meant to be tagged 1.4). If it doesn't
> > > have that default constructor it should, provided that an 
> > appropriate
> > > shutdown behavior can be implemented. Otherwise we'll 
> reinstate the
> > > bug causing a one minute hang before shutdown (eg on rebuild ).
> > >
> > > I think there's an argument to be made that there shouldn't be a
> > > default constructor for that reason... maybe we should discuss on 
> > > Tuesday.
> > >
> > > - Ben
> > >
> > > On Dec 11, 2011 12:08 PM, "Stephen Bayliss"
> > > <stephen.bayliss@...> wrote:
> > >>
> > >> Trippi master doesn't seem to build and pass all of its tests
> > >> currently; but I can see there's a branch fix_executor with the 
> > >> latest commits which does build and pass tests.
> > >>
> > >> I'm working on FCREPO-702 (update to latest Mulgara
> > version), so it
> > >> would be good to have a "known good" codebase to work from.
> > >>
> > >> Currently I've created branch fcrepo-702 from branch 
> fix_executor,
> > >> updated to Mulgara 2.1.11 plus dependencies, and this is 
> > passing all
> > >> tests.
> > >>
> > >> I've also got a Fedora branch fcrepo-702 with this integrated.
> > >> So:
> > >>
> > >> 1) Is fix_executor fit (or close) to merge to master?  And
> > are there
> > >> any other outstanding planned changes?  If not it would 
> be good to
> > >> release a new version of trippi soon with changes from 
> fcrepo-702 
> > >> merged in also (once the fcrepo tests are all passing).
> > >>
> > >> 2) As soon as we can get the new mulgara and trippi jars
> > maven-hosted
> > >> (I may need some help with that! - though I have a 
> revised pom for
> > >> 2.1.11) I can also merge fcrepo-702 from fcrepo into 
> > master once I've
> > >> finished testing.
> > >>
> > >> Particularly on (1) there is a constructor change on
> > >> RIOTripleIterator, the constructor now takes an executor 
> > service - I
> > >> wonder if the old constructor with some kind of
> > default/singe thread
> > >> executor should be reinstated, or whether consumers should be
> > >> providing their own (this has only affected Fedora 
> > integration tests;
> > >> but anyone else who is also using trippi artefacts would
> > also have to
> > >> make that change - unless we bring back the existing
> > constructor).
> > >> Similarly there are a few other modified constructors
> > elsewhere that
> > >> now take a TripleIteratorFactory; again I wonder if we
> > should bring
> > >> back in the old constructors also - to avoid any client software
> > >> changes in a 3.6 release (although the changes are 
> > relatively minor).
> > >> All thoughts welcomed on this!
> > >>
> > >> Thanks
> > >> Steve
> > >>
> > >>
> > >> 
> > 
> ---------------------------------------------------------------------
> > >> ---------
> > >> Learn Windows Azure Live!  Tuesday, Dec 13, 2011
> > >> Microsoft is holding a special Learn Windows Azure
> > training event for
> > >> developers. It will provide a great way to learn Windows
> > Azure and what it
> > >> provides. You can attend the event by watching it streamed
> > LIVE online.
> > >> Learn more at http://p.sf.net/sfu/ms-windowsazure
> > >> _______________________________________________
> > >> Fedora-commons-developers mailing list 
> > >> Fedora-commons-developers@...
> > >> 
> > 
> https://lists.sourceforge.net/lists/listinfo/fedora-commons-developers
> > >>
> > >
> > > 
> > 
> ----------------------------------------------------------------------
> > > --------
> > > Learn Windows Azure Live!  Tuesday, Dec 13, 2011
> > > Microsoft is holding a special Learn Windows Azure training
> > event for
> > > developers. It will provide a great way to learn Windows
> > Azure and what it
> > > provides. You can attend the event by watching it streamed
> > LIVE online.
> > > Learn more at http://p.sf.net/sfu/ms-windowsazure
> > > _______________________________________________
> > > Fedora-commons-developers mailing list 
> > > Fedora-commons-developers@...
> > > 
> > 
> https://lists.sourceforge.net/lists/listinfo/fedora-commons-developers
> > >
> > 
> > --------------------------------------------------------------
> > ----------------
> > Learn Windows Azure Live!  Tuesday, Dec 13, 2011
> > Microsoft is holding a special Learn Windows Azure training 
> event for
> > developers. It will provide a great way to learn Windows 
> > Azure and what it 
> > provides. You can attend the event by watching it streamed 
> > LIVE online.  
> > Learn more at http://p.sf.net/sfu/ms-windowsazure
> > _______________________________________________
> > Fedora-commons-developers mailing list 
> > Fedora-commons-developers@...
> > 
> https://lists.sourceforge.net/lists/listinfo/fedora-commons-developers
> > 
> 
> 
> --------------------------------------------------------------
> ----------------
> Learn Windows Azure Live!  Tuesday, Dec 13, 2011
> Microsoft is holding a special Learn Windows Azure training event for 
> developers. It will provide a great way to learn Windows 
> Azure and what it 
> provides. You can attend the event by watching it streamed 
> LIVE online.  
> Learn more at http://p.sf.net/sfu/ms-windowsazure
> _______________________________________________
> Fedora-commons-developers mailing list
> Fedora-commons-developers@...
> https://lists.sourceforge.net/lists/listinfo/fedora-commons-developers
> 

------------------------------------------------------------------------------
Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
infrastructure or vast IT resources to deliver seamless, secure access to
virtual desktops. With this all-in-one solution, easily deploy virtual 
desktops for less than the cost of PCs and save 60% on VDI infrastructure 
costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
David Wilcox | 20 Dec 20:52
Picon
Gravatar

[fcrepo-dev] Islandora Announces 11.3.0 Release

* Apologies for cross-posting*

We are pleased to announce the release of Islandora 11.3.0!

You can download all available modules from
http://islandora.ca/download, or test drive the release at
http://sandbox.islandora.ca.

A guide to this release is available here:
http://islandora.ca/11-3-guide. The guide introduces changes and new
features in this version, including improvements to the batch ingest
and book solution pack modules, as well as a host of bug fixes.

The documentation is undergoing updates to match this version of
Islandora, and is available here:
https://wiki.duraspace.org/display/ISLANDORA.

Please report any issues to the google developer or users lists, or at
our JIRA: https://jira.duraspace.org/browse/ISLANDORA.

Please read the documentation carefully before updating as this
version may cause issues with previously installed versions of
Islandora - particularly versions released prior to Islandora 11.2.
All modules should be updated simultaneously to keep everything in
sync. If you encounter an undocumented issue, please let us know so
that we can help build and document a complete upgrade path.

------------------------------------------------------------------------------
Write once. Port to many.
Get the SDK and tools to simplify cross-platform app development. Create 
new or port existing apps to sell to consumers worldwide. Explore the 
Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join
http://p.sf.net/sfu/intel-appdev
Favicon

[fcrepo-dev] Kaltura and Institutional Repository connector

Hi all,

**Please excuse the cross-posting of this email**

Here at Northwestern University we are currently evaluating Kaltura's video streaming service.  There has been some discussion between some institutions and Kaltura regarding Kaltura building a connector to use with different institutional repositories (e.g. Fedora, DSpace, ContentDM).  I'm trying to compile a list of institutions that would be interested in using such a connector.

If you're interested, I'd greatly appreciate you replying to me (m-stroming-angj7XTnwNpXfO9P/gJGhg@public.gmane.org) with your institution and which repository software you're using.

We need your responses very soon (if possible, before the holiday break) in order to bring this information to Kaltura in a timely manner.  Feel free to pass this email along to other lists.

Thanks very much,

Mike Stroming

Senior Software Developer, Enterprise Systems

Library Technology Division

Northwestern University Library

(847) 491-8345


------------------------------------------------------------------------------
Cloud Computing - Latest Buzzword or a Glimpse of the Future?
This paper surveys cloud computing today: What are the benefits? 
Why are businesses embracing it? What are its payoffs and pitfalls?
http://www.accelacomm.com/jaw/sdnl/114/51425149/
_______________________________________________
Fedora-commons-developers mailing list
Fedora-commons-developers@...
https://lists.sourceforge.net/lists/listinfo/fedora-commons-developers
Chris Wilper | 12 Dec 15:10
Favicon
Gravatar

[fcrepo-dev] Trying Google+ Hangouts for dev meetings?

Hi all,


We've used HiDef Conferencing for a few years now for the Fedora committer calls with very little trouble. Unfortunately, the service is changing soon and will no longer offer Skype dialin -- the capability that sold us on it to begin with. DuraSpace is experimenting with other teleconference providers, some of which provide a VoIP option in addition to normal phone numbers.

In the meantime, a few of us have been playing with Google Hangouts for meetings and have had pretty good experiences so far with both audio and video (we don't typically do video on the committer calls, but having the option might be nice...). It also has a simple shared whiteboard and text editing thing.

So my question is, for people who attend these calls regularly (or occasionally), have you tried Hangouts yet, and what do you think about using them? Obviously it would require attendees to have a G+ account, which is the main disadvantage I can think of.

- Chris
------------------------------------------------------------------------------
Learn Windows Azure Live!  Tuesday, Dec 13, 2011
Microsoft is holding a special Learn Windows Azure training event for 
developers. It will provide a great way to learn Windows Azure and what it 
provides. You can attend the event by watching it streamed LIVE online.  
Learn more at http://p.sf.net/sfu/ms-windowsazure
_______________________________________________
Fedora-commons-developers mailing list
Fedora-commons-developers@...
https://lists.sourceforge.net/lists/listinfo/fedora-commons-developers
Stephen Bayliss | 11 Dec 21:10

[fcrepo-dev] Status of trippi src?

Trippi master doesn't seem to build and pass all of its tests currently; but I can see there's a branch fix_executor with the latest commits which does build and pass tests.
 
I'm working on FCREPO-702 (update to latest Mulgara version), so it would be good to have a "known good" codebase to work from.
 
Currently I've created branch fcrepo-702 from branch fix_executor, updated to Mulgara 2.1.11 plus dependencies, and this is passing all tests.
 
I've also got a Fedora branch fcrepo-702 with this integrated.
So:
 
1) Is fix_executor fit (or close) to merge to master?  And are there any other outstanding planned changes?  If not it would be good to release a new version of trippi soon with changes from fcrepo-702 merged in also (once the fcrepo tests are all passing).
 
2) As soon as we can get the new mulgara and trippi jars maven-hosted (I may need some help with that! - though I have a revised pom for 2.1.11) I can also merge fcrepo-702 from fcrepo into master once I've finished testing.
 
Particularly on (1) there is a constructor change on RIOTripleIterator, the constructor now takes an executor service - I wonder if the old constructor with some kind of default/singe thread executor should be reinstated, or whether consumers should be providing their own (this has only affected Fedora integration tests; but anyone else who is also using trippi artefacts would also have to make that change - unless we bring back the existing constructor).  Similarly there are a few other modified constructors elsewhere that now take a TripleIteratorFactory; again I wonder if we should bring back in the old constructors also - to avoid any client software changes in a 3.6 release (although the changes are relatively minor).  All thoughts welcomed on this!
 
Thanks
Steve
------------------------------------------------------------------------------
Learn Windows Azure Live!  Tuesday, Dec 13, 2011
Microsoft is holding a special Learn Windows Azure training event for 
developers. It will provide a great way to learn Windows Azure and what it 
provides. You can attend the event by watching it streamed LIVE online.  
Learn more at http://p.sf.net/sfu/ms-windowsazure
_______________________________________________
Fedora-commons-developers mailing list
Fedora-commons-developers@...
https://lists.sourceforge.net/lists/listinfo/fedora-commons-developers

Gmane