headers
Jeremy Custenborder (JIRA | 1 Aug 2011 17:28
Picon
Favicon

[Created] (PIG-2198) pig.additional.jars does not support s3 paths

pig.additional.jars does not support s3 paths
---------------------------------------------

                 Key: PIG-2198
                 URL: https://issues.apache.org/jira/browse/PIG-2198
             Project: Pig
          Issue Type: Bug
    Affects Versions: 0.8.1
            Reporter: Jeremy Custenborder
            Priority: Minor

The delimiter for additional jars is ":" which is part of the uri for a S3 path. 

*Repro*
Add the following to the command line.
{quote}
-Dpig.additional.jars=s3://bucket/udf.jar
{quote}

{quote}
Error before Pig is launched
----------------------------
ERROR 101: Could not copy s3 to local destination /mnt/var/lib/hadoop/steps/1/tmp/s3

org.apache.pig.backend.executionengine.ExecException: ERROR 4010: Failed to register jar :s3.
Caught exception.
	at org.apache.pig.PigServer.addJarsFromProperties(PigServer.java:253)
	at org.apache.pig.PigServer.<init>(PigServer.java:239)
	at org.apache.pig.PigServer.<init>(PigServer.java:215)
	at org.apache.pig.tools.grunt.Grunt.<init>(Grunt.java:55)
(Continue reading)

Permalink | Reply | 9 comments |
headers
Jeremy Custenborder (JIRA | 1 Aug 2011 17:40
Picon
Favicon

[Updated] (PIG-2198) pig.additional.jars does not support s3 paths


     [
https://issues.apache.org/jira/browse/PIG-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jeremy Custenborder updated PIG-2198:
-------------------------------------

    Release Note: Changed the delimiter from : to ;
          Status: Patch Available  (was: Open)

> pig.additional.jars does not support s3 paths
> ---------------------------------------------
>
>                 Key: PIG-2198
>                 URL: https://issues.apache.org/jira/browse/PIG-2198
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.8.1
>            Reporter: Jeremy Custenborder
>            Priority: Minor
>
> The delimiter for additional jars is ":" which is part of the uri for a S3 path. 
> *Repro*
> Add the following to the command line.
> {quote}
> -Dpig.additional.jars=s3://bucket/udf.jar
> {quote}
> {quote}
> Error before Pig is launched
> ----------------------------
(Continue reading)

Permalink | Reply | 0 comments |
headers
Jeremy Custenborder (JIRA | 1 Aug 2011 17:42
Picon
Favicon

[Updated] (PIG-2198) pig.additional.jars does not support s3 paths


     [
https://issues.apache.org/jira/browse/PIG-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jeremy Custenborder updated PIG-2198:
-------------------------------------

    Attachment: PIG-2198.patch

Changed the delimiter from : to ;

> pig.additional.jars does not support s3 paths
> ---------------------------------------------
>
>                 Key: PIG-2198
>                 URL: https://issues.apache.org/jira/browse/PIG-2198
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.8.1
>            Reporter: Jeremy Custenborder
>            Priority: Minor
>         Attachments: PIG-2198.patch
>
>
> The delimiter for additional jars is ":" which is part of the uri for a S3 path. 
> *Repro*
> Add the following to the command line.
> {quote}
> -Dpig.additional.jars=s3://bucket/udf.jar
> {quote}
(Continue reading)

Permalink | Reply | 0 comments |
headers
Jeremy Custenborder (JIRA | 1 Aug 2011 17:44
Picon
Favicon

[Commented] (PIG-2198) pig.additional.jars does not support s3 paths


    [
https://issues.apache.org/jira/browse/PIG-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13073578#comment-13073578
] 

Jeremy Custenborder commented on PIG-2198:
------------------------------------------

I'm not sure if we want to change the delimiter because this will break environments that are already using
this parameter.I added a patch that uses a different delimiter but I'm not sure if that is the best action.  

> pig.additional.jars does not support s3 paths
> ---------------------------------------------
>
>                 Key: PIG-2198
>                 URL: https://issues.apache.org/jira/browse/PIG-2198
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.8.1
>            Reporter: Jeremy Custenborder
>            Priority: Minor
>         Attachments: PIG-2198.patch
>
>
> The delimiter for additional jars is ":" which is part of the uri for a S3 path. 
> *Repro*
> Add the following to the command line.
> {quote}
> -Dpig.additional.jars=s3://bucket/udf.jar
> {quote}
(Continue reading)

Permalink | Reply | 0 comments |
headers
Thejas M Nair (JIRA | 1 Aug 2011 20:32
Picon
Favicon

[Commented] (PIG-2198) pig.additional.jars does not support s3 paths


    [
https://issues.apache.org/jira/browse/PIG-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13073649#comment-13073649
] 

Thejas M Nair commented on PIG-2198:
------------------------------------

':' is what is used for separating path names in linux (including java class path), so that is what is
intuitive to users on linux.  Also, ';' is interpreted by (bash) shell as end of command, users will have to
use quotes to avoid that interpretation.  

I think it is better to make the delimiter configurable. Another option is to not use ':' within quoted
strings as delimiters, but getting those quotes through the shell to pig would require escaping the
quotes, and that would be ugly.

> pig.additional.jars does not support s3 paths
> ---------------------------------------------
>
>                 Key: PIG-2198
>                 URL: https://issues.apache.org/jira/browse/PIG-2198
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.8.1
>            Reporter: Jeremy Custenborder
>            Priority: Minor
>         Attachments: PIG-2198.patch
>
>
> The delimiter for additional jars is ":" which is part of the uri for a S3 path.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Raghu Angadi (JIRA | 1 Aug 2011 22:56
Picon
Favicon

[Updated] (PIG-2193) Problem with HBase loader 0.90.3 and PIG 0.8.1


     [
https://issues.apache.org/jira/browse/PIG-2193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Raghu Angadi updated PIG-2193:
------------------------------

    Attachment: PIG-2193.patch

patch for trunk is attached.
  - This is the complete fix.
  - includes a unit test
  - earlier simple patch works, but does not set the project before the scan is initialized.
  - removed 'initialized' variable since we know when schema and projections are set:
    -- store schema is set inside checkSchema()
    -- projection is set in pushProjection()
    -- on the backend, projection is handled in side setLocation().
    -- on the backend store schema is set inside setStoreLocation().

> Problem with HBase loader 0.90.3 and PIG 0.8.1
> ----------------------------------------------
>
>                 Key: PIG-2193
>                 URL: https://issues.apache.org/jira/browse/PIG-2193
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.8.1
>         Environment: HBase 0.90.3, Hadoop 0.20-append
>            Reporter: Vincent BARAT
>         Attachments: PIG-2193.patch, PIG-2193.patch
(Continue reading)

Permalink | Reply | 0 comments |
headers
Raghu Angadi (JIRA | 1 Aug 2011 23:00
Picon
Favicon

[Updated] (PIG-2193) Problem with HBase loader 0.90.3 and PIG 0.8.1


     [
https://issues.apache.org/jira/browse/PIG-2193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Raghu Angadi updated PIG-2193:
------------------------------

    Fix Version/s: 0.10
                   0.9.1
         Assignee: Raghu Angadi
           Status: Patch Available  (was: Open)

> Problem with HBase loader 0.90.3 and PIG 0.8.1
> ----------------------------------------------
>
>                 Key: PIG-2193
>                 URL: https://issues.apache.org/jira/browse/PIG-2193
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.8.1
>         Environment: HBase 0.90.3, Hadoop 0.20-append
>            Reporter: Vincent BARAT
>            Assignee: Raghu Angadi
>             Fix For: 0.9.1, 0.10
>
>         Attachments: PIG-2193.patch, PIG-2193.patch
>
>
> I've some data in HBase 0.90.3 and I run a simple script on them.
> This script badly returns 0 records. From time to time, under yet undefined conditions, the same script on
(Continue reading)

Permalink | Reply | 0 comments |
headers
Dmitriy V. Ryaboy (JIRA | 1 Aug 2011 23:54
Picon
Favicon

[Commented] (PIG-2193) Problem with HBase loader 0.90.3 and PIG 0.8.1


    [
https://issues.apache.org/jira/browse/PIG-2193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13073808#comment-13073808
] 

Dmitriy V. Ryaboy commented on PIG-2193:
----------------------------------------

I will review.

> Problem with HBase loader 0.90.3 and PIG 0.8.1
> ----------------------------------------------
>
>                 Key: PIG-2193
>                 URL: https://issues.apache.org/jira/browse/PIG-2193
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.8.1
>         Environment: HBase 0.90.3, Hadoop 0.20-append
>            Reporter: Vincent BARAT
>            Assignee: Raghu Angadi
>             Fix For: 0.9.1, 0.10
>
>         Attachments: PIG-2193.patch, PIG-2193.patch
>
>
> I've some data in HBase 0.90.3 and I run a simple script on them.
> This script badly returns 0 records. From time to time, under yet undefined conditions, the same script on
the same data works (it return correct data).
> When data are loaded from HDFS instead of HBase, the script runs perfectly.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Jeremy Custenborder (JIRA | 2 Aug 2011 01:02
Picon
Favicon

[Commented] (PIG-2198) pig.additional.jars does not support s3 paths


    [
https://issues.apache.org/jira/browse/PIG-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13073983#comment-13073983
] 

Jeremy Custenborder commented on PIG-2198:
------------------------------------------

Good catch. I have no clue why I thought of ;. I was thinking about using the same delimiter that is used for
FileInputFormat.addInputPaths which is actually a comma. I think this parameter will be much more
useful with the support for dfs locations. 

> pig.additional.jars does not support s3 paths
> ---------------------------------------------
>
>                 Key: PIG-2198
>                 URL: https://issues.apache.org/jira/browse/PIG-2198
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.8.1
>            Reporter: Jeremy Custenborder
>            Priority: Minor
>         Attachments: PIG-2198.patch
>
>
> The delimiter for additional jars is ":" which is part of the uri for a S3 path. 
> *Repro*
> Add the following to the command line.
> {quote}
> -Dpig.additional.jars=s3://bucket/udf.jar
(Continue reading)

Permalink | Reply | 0 comments |
headers
Doug Daniels (JIRA | 2 Aug 2011 02:38
Picon
Favicon

[Updated] (PIG-2199) Penny throws Exception when netty classes are missing


     [
https://issues.apache.org/jira/browse/PIG-2199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Doug Daniels updated PIG-2199:
------------------------------

    Status: Patch Available  (was: Open)

> Penny throws Exception when netty classes are missing
> -----------------------------------------------------
>
>                 Key: PIG-2199
>                 URL: https://issues.apache.org/jira/browse/PIG-2199
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.9.0, 0.10
>            Reporter: Doug Daniels
>
> Running the data sampler tool from the penny library causes a ClassNotFoundException for a netty class. 
Per the mailing list, this is because the netty classes are not accessible to Penny.
> I've attached a patch that adds netty to the penny jar.
> For reference, I'm running a simple script that uses pig test data from
> test/org/apache/pig/test/data/InputFiles/jsTst1.txt :
>     x = LOAD 'jsTst1.txt' USING PigStorage('\t');
>     x_filtered = FILTER x BY (int)$1 > 100;
>     STORE x_filtered INTO 'jsTst1Filtered';

--
This message is automatically generated by JIRA.
(Continue reading)

Permalink | Reply | 0 comments |

Gmane