headers
VINAYAKUMAR B (Created) (JIRA | 11 Feb 00:40
Picon
Favicon

[jira] [Created] (HADOOP-8057) hadoop-setup-conf.sh not working because of some extra spaces.

hadoop-setup-conf.sh not working because of some extra spaces.
--------------------------------------------------------------

                 Key: HADOOP-8057
                 URL: https://issues.apache.org/jira/browse/HADOOP-8057
             Project: Hadoop Common
          Issue Type: Bug
          Components: scripts
    Affects Versions: 0.24.0
         Environment: Suse11
            Reporter: VINAYAKUMAR B
             Fix For: 0.24.0

hadoop-setup-conf.sh is not working because of one extra space, while configuring the getopt.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Permalink | Reply | 0 comments |
headers
Mikhail Bautin | 10 Feb 22:20
Picon
Gravatar

Hadoop QA machine configuration / getting a Jenkins account

Hello Hadoop Developers,

A couple of questions about Hadoop QA:
(1) Does anyone know what the configuration of Hadoop QA boxes is?
Specifically, I am interested in the amount of memory available on these
boxes.
(2) How to get an account for https://builds.apache.org/?

Thanks,
--Mikhail
Permalink | Reply | 4 comments |
headers
Ralph Castain | 10 Feb 18:42
Favicon
Gravatar

Trunk build failure

Hi folks

I'm a newbie to the Hadoop code and am trying to build the svn trunk per instructions on the wiki and the
mailing list.  I'm hitting a failure and would appreciate any suggestions:

main:
     [exec] tar: Failed to open 'hadoop-project-dist-0.24.0-SNAPSHOT.tar.gz'

It looks like this file isn't getting created, but I don't know why. I used the -e option for mvn, and saw this:

[INFO] --- maven-antrun-plugin:1.6:run (pre-dist) @ hadoop-project-dist ---
[INFO] Executing tasks

main:
     [exec] ./dist-copynativelibs.sh: line 10: [: too many arguments
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (dist) @ hadoop-project-dist ---
[WARNING] The following patterns were never triggered in this artifact exclusion filter:
o  'org.apache.ant:*:jar'
o  'jdiff:jdiff:jar'

[INFO] Copying files to /Volumes/Macintosh HD 2/rhc/hadoop/svn-trunk/hadoop-project-dist/target/hadoop-project-dist-0.24.0-SNAPSHOT
[INFO] 
[INFO] --- maven-source-plugin:2.1.2:jar-no-fork (hadoop-java-sources) @ hadoop-project-dist ---
[INFO] 
[INFO] --- maven-enforcer-plugin:1.0:enforce (dist-enforce) @ hadoop-project-dist ---
[INFO] 
[INFO] --- maven-site-plugin:3.0:attach-descriptor (attach-descriptor) @ hadoop-project-dist ---
[INFO]
(Continue reading)

Permalink | Reply | 0 comments |
headers
Luca Pireddu (Created) (JIRA | 10 Feb 18:27
Picon
Favicon

[jira] [Created] (HADOOP-8056) Configuration doesn't pass empty string values to tasks

Configuration doesn't pass empty string values to tasks
-------------------------------------------------------

                 Key: HADOOP-8056
                 URL: https://issues.apache.org/jira/browse/HADOOP-8056
             Project: Hadoop Common
          Issue Type: Bug
          Components: conf
    Affects Versions: 1.0.0, 0.20.2
            Reporter: Luca Pireddu

If I assign an *empty string* as a value to a property in a JobConf 'job' while I'm preparing it to run, the
Configuration does store that value.  I can retrieve it later while in the same process and the value is maintained.

However, if then call JobClient.runJob(job), the Configuration that is received by the Map and Reduce
tasks doesn't contain the property, and calling JobConf.get with that property name returns null
(instead of an empty string).  Futher, if I inspect the job's configuration via Hadoop's web interface,
the property isn't present.

It seems as if whatever serialization mechanism that is used to transmit the Configuration from the job
client to the tasks discards properties with "" value.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Permalink | Reply | 0 comments |
headers
Eric Charles (Created) (JIRA | 10 Feb 12:57
Picon
Favicon

[jira] [Created] (HADOOP-8055) Distribution tar.gz does not contain etc/hadoop/core-site.xml

Distribution tar.gz does not contain etc/hadoop/core-site.xml
-------------------------------------------------------------

                 Key: HADOOP-8055
                 URL: https://issues.apache.org/jira/browse/HADOOP-8055
             Project: Hadoop Common
          Issue Type: Bug
          Components: build
    Affects Versions: 0.24.0
            Reporter: Eric Charles

A dist built from trunk (0.24.0-SNAPSHOT) does not contain a core-site.xml in $HADOOP_HOME/etc/hadoop/ folder.

$HADOOP_HOME/sbin/start-dfs.sh without that folder gives an exception
Exception in thread "main" java.lang.IllegalArgumentException: URI has an authority component
 at java.io.File.<init>(File.java:368)
 at org.apache.hadoop.hdfs.server.namenode.NNStorage.getStorageDirectory(NNStorage.java:310)
 at org.apache.hadoop.hdfs.server.namenode.FSEditLog.init(FSEditLog.java:178)
...

Manually creating $HADOOP_HOME/etc/hadoop/core-site.xml solves this problem and hadoop starts fine.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Permalink | Reply | 0 comments |
headers
Amareshwari Sriramadasu (Created) (JIRA | 10 Feb 09:19
Picon
Favicon

[jira] [Created] (HADOOP-8054) NPE with RawLocalFileSystem

NPE with RawLocalFileSystem
---------------------------

                 Key: HADOOP-8054
                 URL: https://issues.apache.org/jira/browse/HADOOP-8054
             Project: Hadoop Common
          Issue Type: Bug
    Affects Versions: 0.23.1
            Reporter: Amareshwari Sriramadasu
            Priority: Critical

While running Hive tests, I'm seeing the following exception with 0.23.1,
{noformat}
ava.lang.NullPointerException
        at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
        at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
        at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
        at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
        at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
        at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
        at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
        at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
        at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
        at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
{noformat}

Did not see this with 0.23.0, though.

--
This message is automatically generated by JIRA.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Carl Steinbach (Created) (JIRA | 10 Feb 03:29
Picon
Favicon

[jira] [Created] (HADOOP-8053) "har://hdfs-/foo" is not a valid URI

"har://hdfs-/foo" is not a valid URI
------------------------------------

                 Key: HADOOP-8053
                 URL: https://issues.apache.org/jira/browse/HADOOP-8053
             Project: Hadoop Common
          Issue Type: Bug
          Components: fs
    Affects Versions: 0.23.0, 0.21.0
            Reporter: Carl Steinbach

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Permalink | Reply | 0 comments |
headers
Grace | 10 Feb 02:49
Picon
Gravatar

Another 2 failures found in hadoop 1.0.0

Hi all,

I Met 2 errors in the unit test(test-core-excluding-commit-and-smoke) of
hadoop 1.0.0. One is org.apache.hadoop.mapred.TestTTResourceReporting and
the other is org.apache.hadoop.tools.rumen.TestRumenJobTraces.

Does anyone have similar problem or have some idea? Thanks.

-Grace.
Permalink | Reply | 0 comments |
headers
Varun Kapoor (Created) (JIRA | 9 Feb 21:58
Picon
Favicon

[jira] [Created] (HADOOP-8052) Hadoop Metrics2 should emit Float.MAX_VALUE (instead of Double.MAX_VALUE) to avoid making Ganglia's gmetad core

Hadoop Metrics2 should emit Float.MAX_VALUE (instead of Double.MAX_VALUE) to avoid making Ganglia's
gmetad core
---------------------------------------------------------------------------------------------------------------

                 Key: HADOOP-8052
                 URL: https://issues.apache.org/jira/browse/HADOOP-8052
             Project: Hadoop Common
          Issue Type: Bug
          Components: metrics
    Affects Versions: 1.0.0, 0.23.0
            Reporter: Varun Kapoor
            Assignee: Varun Kapoor

Ganglia's gmetad converts the doubles emitted by Hadoop's Metrics2 system to strings, and the buffer it
uses is 256 bytes wide.

When the SampleStat.MinMax class (in org.apache.hadoop.metrics2.util) emits its default min value
(currently initialized to Double.MAX_VALUE), it ends up causing a buffer overflow in gmetad, which
causes it to core, effectively rendering Ganglia useless (for some, the core is continuous; for others
who are more fortunate, it's only a one-time Hadoop-startup-time thing).

The fix needed to Ganglia is simple - the buffer needs to be bumped up to be 512 bytes wide, and all will be well -
but instead of requiring a minimum version of Ganglia to work with Hadoop's Metrics2 system, it might be
more prudent to just use Float.MAX_VALUE.

An additional problem caused in librrd (which Ganglia uses beneath-the-covers) by the use of
Double.MIN_VALUE (which functions as the default max value) is an underflow when librrd runs the
received strings through libc's strtod(), but the librrd code is good enough to check for this, and only
emits a warning - moving to Float.MIN_VALUE fixes that as well.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Kihwal Lee (Created) (JIRA | 9 Feb 21:30
Picon
Favicon

[jira] [Created] (HADOOP-8050) Deadlock in metrics

Deadlock in metrics
-------------------

                 Key: HADOOP-8050
                 URL: https://issues.apache.org/jira/browse/HADOOP-8050
             Project: Hadoop Common
          Issue Type: Bug
          Components: metrics
    Affects Versions: 1.0.0, 0.20.205.0, 0.20.204.0
            Reporter: Kihwal Lee
             Fix For: 1.1.0, 1.0.1

The metrics serving thread and the periodic snapshot thread can deadlock.
It happened a few times on one of namenode we have. When it happens RPC works but the web ui and hftp stop
working. I haven't look at the trunk too closely, but it might happen there too.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Permalink | Reply | 0 comments |
headers
Steve Loughran (Created) (JIRA | 9 Feb 18:33
Picon
Favicon

[jira] [Created] (HADOOP-8049) Add entry point/script to do preflight checking of script resolution

Add entry point/script to do preflight checking of script resolution
--------------------------------------------------------------------

                 Key: HADOOP-8049
                 URL: https://issues.apache.org/jira/browse/HADOOP-8049
             Project: Hadoop Common
          Issue Type: New Feature
          Components: util
    Affects Versions: 1.0.0, 0.24.0
            Reporter: Steve Loughran
            Priority: Minor

The script supplied by the ops team could have many problems
# not run
# have the wrong permissions
# fail to parse/resolve some or all hosts
# not be the string set in the configuration file.

Diagnostics could be improved here with an entry point that
# reads in the configuration file
# instantiates whatever mapper is declared
# attempts to resolve all hosts passed in as arguments, logging success and failures
# prints out a topology map at the end
A key goal would be to recognise and report a complete failure to report a mapping as per MAPREDUCE-50, so let
people identify problems in their script before updating the cluster

Test plan
-Java entry point: invoke it with arguments, get the output as a string and scan.
-Shell script. Include whatever post-install script tests are being written.
(Continue reading)

Permalink | Reply | 0 comments |

Gmane