Sangamnerkar Tushar | 1 Nov 05:55 2007
Picon

Imported records ignored by sub-daily

Hello
 
We are running dspace 1.3.2 on FC3.0.  sub-daily is not sending the additions to dspace collections if appended thru import (Item import) utility.  How to handle this ?
 
Tushar
 
 

__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
Mika Stenberg | 1 Nov 08:44 2007
Picon
Picon

Running two prefixes with one handle server

Im running two Dspace instances on one machine. I registered another 
handle prefix for the second one, but am little confused on how to 
configure the handle server for two prefixes. Has anyone done this 
before? How should I go on about it?

Thanks,
Mika

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
Stuart Lewis [sdl] | 1 Nov 09:51 2007
Picon

Re: Blocking a malicious user

Hi Sue,

pg_hba.conf only controls who can communicate with Postgres, not who can
communicate with DSpace.

Normally it is only 'applications' (e.g. DSpace) that talk to Postgres,
not users.

A user talks to DSpace, who in turn talks to Postgres. Postgres has no
idea or interest in the IP address of the user who is using DSpace, only
that of the DSpace application.

Therefore adding malicious IP address into that config file will sadly
have no effect. You have to block users higher in the stack, either at
the application level (apache or tomcat directives), or at the network
level (firewall changes).

Thanks,

Stuart
_________________________________________________________________

Gwasanaethau Gwybodaeth                      Information Services
Prifysgol Aberystwyth                      Aberystwyth University

            E-bost / E-mail: Stuart.Lewis@...             
                 Ffon / Tel: (01970) 622860
_________________________________________________________________

-----Original Message-----
From: dspace-tech-bounces@...
[mailto:dspace-tech-bounces@...] On Behalf Of
Thornton, Susan M. (LARC-B702)[NCI INFORMATION SYSTEMS]
Sent: 31 October 2007 17:51
To: Mika Stenberg; dspace-tech@...
Subject: Re: [Dspace-tech] Blocking a malicious user

You can block ip addresses at the postgreSQL level in the pg_hba.conf
file.  Here is a person I blocked by ip address who was sending all
kinds of GET requests to our DSpace server:

host    all         all         malicious.ip    255.255.255.255
reject

Sue Walker-Thornton
NASA Langley Research Center
ConITS Contract
757-224-4074
Susan.m.thornton@...

-----Original Message-----
From: dspace-tech-bounces@...
[mailto:dspace-tech-bounces@...] On Behalf Of Mika
Stenberg
Sent: Wednesday, October 31, 2007 6:00 AM
To: dspace-tech@...
Subject: Re: [Dspace-tech] Blocking a malicious user

We've had problems like that as well. Blocking specific IP's works only
for
a while since many bots and spammers seem to change their IP frequently.
We
didnt come up with a decent solution for this, but  blocking an entire
country of origin for a period of time has been on my mind. Managing the
allowed requests / timeslot for a specific IP might also do the trick.

-Mika

> If they're nasty enough, though, they'll drown your Apache or Tomcat
> server in replying with 403s. I've had times that I needed to be
> absolutely merciless and block at the firewall level, using iptables;
> then they don't even get as far as userspace.
> 
> On Tue, 2007-10-30 at 14:01 -0500, Tim Donohue wrote:
> > George,
> > 
> > We had a similar problem to this one in the past (a year or so ago).
I
> 
> > just flat out blocked the IP altogether (not even specific to 
> > /bitstream/) via this Apache configuration:
> > 
> > <Location />
> >      Order Allow,Deny
> > 
> >      Deny from {malicious ip}
> > 
> >      Allow from all
> > </Location>
> > 
> > This looks similar to your config though (except it blocks all
access 
> > from that IP).
> > 
> > - Tim
> > 
> > George Kozak wrote:
> > > Hi...
> > > 
> > > I am having a problem with an IP that keeps sending thousands of
"GET
> 
> > > /bitstream/..." requests for the same item.
> > > 
> > > I have placed the following in my Apache.conf file:
> > > 
> > > <Directory /bitstream/≥
> > > Options Indexes FollowSymLinks MultiViews
> > > AllowOverride All
> > > Order allow,deny
> > > allow from all
> > > deny from {malicious ip}
> > > </Directory>
> > > 
> > > I also placed the following in my server.xml in Tomcat:
> > > <Valve className="org.apache.catalina.valves.RemoteAddrValve" 
> > > deny="xxx\.xxx\.xxx\.xx" />
> > > 
> > > However, this person still seems to be getting through.  My java 
> > > process is running from 50%-80% CPU usage.  Does anyone have a
good 
> > > idea on how to shutout a malicious IP in DSpace?
> > > 
> > > ***************************
> > > George Kozak
> > > Coordinator
> > > Web Development and Management
> > > Digital Media Group
> > > 501 Olin Library
> > > Cornell University
> > > 607-255-8924
> > > ***************************
> > > gsk5@... 
> > > 
> > > 
> > >
>
------------------------------------------------------------------------
-
> > > This SF.net email is sponsored by: Splunk Inc.
> > > Still grepping through log files to find problems?  Stop.
> > > Now Search log events and configuration files using AJAX and a
> browser.
> > > Download your FREE copy of Splunk now >> http://get.splunk.com/
> > > _______________________________________________
> > > DSpace-tech mailing list
> > > DSpace-tech@...
> > > https://lists.sourceforge.net/lists/listinfo/dspace-tech
> > > 
> > 
> 
>
------------------------------------------------------------------------
-
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a
browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> DSpace-tech mailing list
> DSpace-tech@...
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
> 
> 

------------------------------------------------------------------------
-
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
DSpace-tech mailing list
DSpace-tech@...
https://lists.sourceforge.net/lists/listinfo/dspace-tech

------------------------------------------------------------------------
-
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
DSpace-tech mailing list
DSpace-tech@...
https://lists.sourceforge.net/lists/listinfo/dspace-tech

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
Robert Roggenbuck | 1 Nov 11:32 2007
Picon
Picon

Re: Starting DSpace: Tomcat can't start DSpace

Hi Christian,

hurray! Finally I got it. You were right with the assumption of the 
dspace tables. After removing them by 'dropdb -U dspace dspace' I got an 
successful build with ant. Many thanks to Your detailed help!
Now I will contribute to the Wiki and then going further with the 
configuration of DSpace.

Smiling greetings

Robert :-)
------------

Christian Voelker schrieb:
> Hello,
> 
> Am 31.10.2007 um 17:07 schrieb Robert Roggenbuck:
> 
>>> Then, the unix
>>> user under which tomcat runs will write into your dspace
>>> directory. It is obvious with the assetstore subdirectory,
>>> but is also true e.g. with the history subdirectory and
>>> others. Once running, you will see by file-dates which
>>> are written during operation. The short solution is that
>>> you run your ant tasks under the unix user which is
>>> tomcat-55 in your case. For consistency, I would make
>>> it a habit, but it is actually required only during the
>>> fresh-install.
>> You really mean I should say
>> $sudo -u tomcat55 ant fresh_install
>> instead of
>> $sudo -u dspace ant fresh_install
>> ?
> 
> Yes, I would probably do so.
> 
>> If this is true, the Wiki page should be corrected at this point. There
>> it is said to execute
>> $sudo chown dspace /var/lib/tomcat5/webapps/*.war
>> This makes no sense if the change of ownership is gone after the
>> unpacking of the *.war files. But if I understand it right the setting
>> of 'TOMCAT5_USER=dspace' in /etc/default/tomcat5.5 prevent this change...
> 
> No, the wiki page just chooses the opposite solution. They
> change the setting mentioned (I did not know this option)
> and then go on using the user dspace. From my point of view,
> they pay twice, but it is consistent within the scheme and
> an absolutely valid solution; you have to choose one of them,
> then follow the track consistently. As the wiki page is more
> elaborate, I would suggest, you follow the wiki.
> 
>> setup_database:
>>       [java] 2007-10-31 16:46:06,811 INFO
>> org.dspace.storage.rdbms.InitializeDatabase  <at>  Initializing Database
>>
>> load_registries:
>>
>> BUILD FAILED
>> /opt/dspace-1.4.2-source/build.xml:333: Java returned: 1
>>
>> Total time: 14 seconds
>>
>>
>> As far as I see, according to the line 333 in build.xml, it fails on
>> looking up the class org.dspace.administer.RegistryLoader . Is there a
>> connection to the warnings in the beginning, the notes from javac? Could
>> it be that my CLASSPATH setting is wrong?
> 
> No, I dont think (but might be mistaken as well). I would stick
> with the very last error encountered and hope that everything
> runs smoothly the next time I try if I am capable to solve this
> particular issue. What I wonder about is, that the build only
> complains during the load_registries target but not yet at the
> setup_database target. Did you remove the database before this
> attempt? I mean not uninstall postgres but drop the dspace db
> and preparing the very same situation that was there before
> the first install?
> 
> fresh_install expects the user dspace to exist in the db and
> the db to be accessible by this user and the password confi-
> gured in the config file. It then sets up all tables required
> and finally fills them with some basic data called the regis-
> tries. If the database already contained all tables required,
> then I would expect the target setup_database to fail already.
> No matter what, it is clear that the registry-data was in your
> database already which made the load_registries target fail.
> 
> It may be annoying, to start over once again but I feel you
> are pretty close to success. Just leave it for today and give
> it another try tomorrow.
> 
> Bye, Christian
> 

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
Cory Snavely | 1 Nov 13:32 2007
Picon

Re: Blocking a malicious user

It has an effect if your Postgres instance isn't blocked at the
firewall, and people are actually trying to access it. Which they will,
unless you block them. As I said, probably much safer to block at the
firewall level--better protection from DOS as well.

On Thu, 2007-11-01 at 08:51 +0000, Stuart Lewis [sdl] wrote:
> Hi Sue,
> 
> pg_hba.conf only controls who can communicate with Postgres, not who can
> communicate with DSpace.
> 
> Normally it is only 'applications' (e.g. DSpace) that talk to Postgres,
> not users.
> 
> A user talks to DSpace, who in turn talks to Postgres. Postgres has no
> idea or interest in the IP address of the user who is using DSpace, only
> that of the DSpace application.
> 
> Therefore adding malicious IP address into that config file will sadly
> have no effect. You have to block users higher in the stack, either at
> the application level (apache or tomcat directives), or at the network
> level (firewall changes).
> 
> Thanks,
> 
> 
> Stuart
> _________________________________________________________________
> 
> Gwasanaethau Gwybodaeth                      Information Services
> Prifysgol Aberystwyth                      Aberystwyth University
> 
>             E-bost / E-mail: Stuart.Lewis@...             
>                  Ffon / Tel: (01970) 622860
> _________________________________________________________________
> 
> 
> 
> 
> -----Original Message-----
> From: dspace-tech-bounces@...
> [mailto:dspace-tech-bounces@...] On Behalf Of
> Thornton, Susan M. (LARC-B702)[NCI INFORMATION SYSTEMS]
> Sent: 31 October 2007 17:51
> To: Mika Stenberg; dspace-tech@...
> Subject: Re: [Dspace-tech] Blocking a malicious user
> 
> You can block ip addresses at the postgreSQL level in the pg_hba.conf
> file.  Here is a person I blocked by ip address who was sending all
> kinds of GET requests to our DSpace server:
> 
> host    all         all         malicious.ip    255.255.255.255
> reject
> 
> Sue Walker-Thornton
> NASA Langley Research Center
> ConITS Contract
> 757-224-4074
> Susan.m.thornton@...
> 
> -----Original Message-----
> From: dspace-tech-bounces@...
> [mailto:dspace-tech-bounces@...] On Behalf Of Mika
> Stenberg
> Sent: Wednesday, October 31, 2007 6:00 AM
> To: dspace-tech@...
> Subject: Re: [Dspace-tech] Blocking a malicious user
> 
> We've had problems like that as well. Blocking specific IP's works only
> for
> a while since many bots and spammers seem to change their IP frequently.
> We
> didnt come up with a decent solution for this, but  blocking an entire
> country of origin for a period of time has been on my mind. Managing the
> allowed requests / timeslot for a specific IP might also do the trick.
> 
> -Mika
> 
> 
> > If they're nasty enough, though, they'll drown your Apache or Tomcat
> > server in replying with 403s. I've had times that I needed to be
> > absolutely merciless and block at the firewall level, using iptables;
> > then they don't even get as far as userspace.
> > 
> > On Tue, 2007-10-30 at 14:01 -0500, Tim Donohue wrote:
> > > George,
> > > 
> > > We had a similar problem to this one in the past (a year or so ago).
> I
> > 
> > > just flat out blocked the IP altogether (not even specific to 
> > > /bitstream/) via this Apache configuration:
> > > 
> > > <Location />
> > >      Order Allow,Deny
> > > 
> > >      Deny from {malicious ip}
> > > 
> > >      Allow from all
> > > </Location>
> > > 
> > > This looks similar to your config though (except it blocks all
> access 
> > > from that IP).
> > > 
> > > - Tim
> > > 
> > > George Kozak wrote:
> > > > Hi...
> > > > 
> > > > I am having a problem with an IP that keeps sending thousands of
> "GET
> > 
> > > > /bitstream/..." requests for the same item.
> > > > 
> > > > I have placed the following in my Apache.conf file:
> > > > 
> > > > <Directory /bitstream/≥
> > > > Options Indexes FollowSymLinks MultiViews
> > > > AllowOverride All
> > > > Order allow,deny
> > > > allow from all
> > > > deny from {malicious ip}
> > > > </Directory>
> > > > 
> > > > I also placed the following in my server.xml in Tomcat:
> > > > <Valve className="org.apache.catalina.valves.RemoteAddrValve" 
> > > > deny="xxx\.xxx\.xxx\.xx" />
> > > > 
> > > > However, this person still seems to be getting through.  My java 
> > > > process is running from 50%-80% CPU usage.  Does anyone have a
> good 
> > > > idea on how to shutout a malicious IP in DSpace?
> > > > 
> > > > ***************************
> > > > George Kozak
> > > > Coordinator
> > > > Web Development and Management
> > > > Digital Media Group
> > > > 501 Olin Library
> > > > Cornell University
> > > > 607-255-8924
> > > > ***************************
> > > > gsk5@... 
> > > > 
> > > > 
> > > >
> >
> ------------------------------------------------------------------------
> -
> > > > This SF.net email is sponsored by: Splunk Inc.
> > > > Still grepping through log files to find problems?  Stop.
> > > > Now Search log events and configuration files using AJAX and a
> > browser.
> > > > Download your FREE copy of Splunk now >> http://get.splunk.com/
> > > > _______________________________________________
> > > > DSpace-tech mailing list
> > > > DSpace-tech@...
> > > > https://lists.sourceforge.net/lists/listinfo/dspace-tech
> > > > 
> > > 
> > 
> >
> ------------------------------------------------------------------------
> -
> > This SF.net email is sponsored by: Splunk Inc.
> > Still grepping through log files to find problems?  Stop.
> > Now Search log events and configuration files using AJAX and a
> browser.
> > Download your FREE copy of Splunk now >> http://get.splunk.com/
> > _______________________________________________
> > DSpace-tech mailing list
> > DSpace-tech@...
> > https://lists.sourceforge.net/lists/listinfo/dspace-tech
> > 
> > 
> 
> 
> 
> ------------------------------------------------------------------------
> -
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> DSpace-tech mailing list
> DSpace-tech@...
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
> 
> ------------------------------------------------------------------------
> -
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> DSpace-tech mailing list
> DSpace-tech@...
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> DSpace-tech mailing list
> DSpace-tech@...
> https://lists.sourceforge.net/lists/listinfo/dspace-tech

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
asim naseem | 1 Nov 17:43 2007
Picon

trouble running video files in Dspace

Hi
 
I have uploaded a video file(.WMV) format to our dspace and it uploaded successfully, but when i try to open it ,the windows media player give the error  say the source filter not found what should i do to run the video file correctly in dspace
 
Any idea how to resolve this
 
Thanks
 
ASIM
Wichita state University,USA

__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
Tim Brody | 1 Nov 18:22 2007
Picon

ANNOUNCE: IRStats statistics package beta

Dear all,

We are happy to announce the beta release of the IRStats package - a 
tool for analysing usage of institutional repositories.

http://trac.eprints.org/projects/irstats/wiki

Please see the examples page for demonstrations of IRStats:
http://trac.eprints.org/projects/irstats/wiki/Examples

Headline features:
- Repository agnostic design - currently supports DSpace and GNU EPrints
- Aggressive filtering of robots and other automated agents (using 
AWStats and bespoke techniques)
- Analyse groups of eprints based on a simple CSV-format specification - 
by-author, by-school etc.

IRStats is released as Open Source under a BSD license.

If you experience any problems using IRStats please get in contact. I'm 
keen to get contributions from the DSpace community - I believe there's 
a number of existing statistics activities going on for DSpace and I'm 
happy to discuss how best to make use of your experience.

Tim Brody
- http://www.eprints.org/

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
Blanco, Jose | 1 Nov 19:22 2007
Picon

Index-all taking a long time.

Recently we had a citation where our postgres db was consuming our space
allocation of 30 GB.  This did not seem right, and we realized that we
had never run "vacuum --full".  When this was run, tons of memory was
freed-up.  Today, I'm running index-all, and the portion that "Indexes
all items in Dspace..." has been running now for close to 4 hours.  Last
week when I ran index-all, this portion of the script took about 1 hour
to run.  I'm getting a bit concerned.  Does this make sense to any one?
Unfortunately, when this portion of index-all runs there are no writes
to log the file ( I have INFO set, not DEBUG ), so I have no idea if
things are progressing smoothly or not.

Thanks for any thoughts on this.

Jose

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
John S. Erickson | 1 Nov 20:13 2007
Picon

[announcement] OAI-ORE Open Meeting (03 March 2008, Johns Hopkins Univ)

Forwarded for Carl Lagoze and Herbert Van de Sompel:

=================
Apologies for cross-posting

A meeting will be held on March 3, 2008 at Johns Hopkins University to
roll-out the first beta release of the OAI-ORE specifications.  These
specifications describe a data model to identify and describe 
aggregations of web resources, and the encoding of the data model in the
XML-based Atom syndication format.

Additional details are available at:

- The full press release for this event:

http://www.openarchives.org/ore/documents/ore-hopkins-press-release.pdf

- The registration site for the meeting:

http://www.regonline.com/oai-ore

Note that registration is required and space is limited.

Carl Lagoze and Herbert Van de Sompel

-----------------------------------------------------
Carl Lagoze
Cornell Information Science
301 College Ave.
Ithaca, NY 14850
WWW: http://www.cs.cornell.edu/lagoze
Phone: +1-607-255-6046
FAX:  +1-607:255-5196

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
Christian Voelker | 1 Nov 20:17 2007
Picon
Picon

Re: trouble running video files in Dspace

Hello,

Am 01.11.2007 um 17:43 schrieb asim naseem:

> I have uploaded a video file(.WMV) format to our dspace and it  
> uploaded successfully, but when i try to open it ,the windows media  
> player give the error  say the source filter not found what should  
> i do to run the video file correctly in dspace
>
> Any idea how to resolve this

Do a request on the bitstream by telnet and check the
header mimetype designation. I guess the file extension
does not get changed, so it should not be the offending
aspect. If you run DSpace through apache, it should be
possible to register the mimetype the mediaplayer ex-
pects for the file type in /etc/httpd/mime.types.
This file is used for mimetype lookup if your apache
uses mod_mime. Or enable mod_mime_magic to support the
file type. I have not tried this, but I guess your
problem has to do with this.

If you have trouble to do these things and your
server is public, then post us a link to the bitstream
in question. Then, give us the section with the lines
starting LoadModule and AddModule of your httpd.conf
to find which modules you use. Third, google for the
wmv and mime to find the required setting. We will
probably be able to solve this on the list.

Bye, Christian

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/

Gmane