Praveen Rachabattuni (JIRA | 30 Jan 10:48 2015
Picon

[Updated] (PIG-4209) Make stream work with Spark


     [
https://issues.apache.org/jira/browse/PIG-4209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Praveen Rachabattuni updated PIG-4209:
--------------------------------------
    Attachment: test_harnesss_1422610455

> Make stream work with Spark
> ---------------------------
>
>                 Key: PIG-4209
>                 URL: https://issues.apache.org/jira/browse/PIG-4209
>             Project: Pig
>          Issue Type: Sub-task
>          Components: spark
>            Reporter: Praveen Rachabattuni
>            Assignee: liyunzhang_intel
>         Attachments: PIG-4209.patch, PIG-4209_1.patch, test_harnesss_1417075517, test_harnesss_1422610455
>
>
> Related e2e-tests: ComputeSpec_[1-13]

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Praveen Rachabattuni (JIRA | 30 Jan 10:48 2015
Picon

[Commented] (PIG-4209) Make stream work with Spark


    [
https://issues.apache.org/jira/browse/PIG-4209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14298419#comment-14298419
] 

Praveen Rachabattuni commented on PIG-4209:
-------------------------------------------

Got all ComputeSpec tests passing in Spark local mode, also attached the log file.

{code}
ant -Dhadoopversion=23 -Dharness.cluster.conf=$HADOOP_HOME/conf
-Dharness.cluster.bin=$HADOOP_HOME/bin -Dharness.hadoop.home=$HADOOP_HOME
-Dtests.to.run="-t ComputeSpec" test-e2e-spark
      ......
     [exec] Final results ,    PASSED: 13   FAILED: 0    SKIPPED: 0    ABORTED: 0    FAILED DEPENDENCY: 0
{code}

Resolving this issue. Thanks [~kellyzly]

> Make stream work with Spark
> ---------------------------
>
>                 Key: PIG-4209
>                 URL: https://issues.apache.org/jira/browse/PIG-4209
>             Project: Pig
>          Issue Type: Sub-task
>          Components: spark
>            Reporter: Praveen Rachabattuni
>            Assignee: liyunzhang_intel
(Continue reading)

Praveen Rachabattuni (JIRA | 30 Jan 10:48 2015
Picon

[Resolved] (PIG-4209) Make stream work with Spark


     [
https://issues.apache.org/jira/browse/PIG-4209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Praveen Rachabattuni resolved PIG-4209.
---------------------------------------
    Resolution: Fixed

> Make stream work with Spark
> ---------------------------
>
>                 Key: PIG-4209
>                 URL: https://issues.apache.org/jira/browse/PIG-4209
>             Project: Pig
>          Issue Type: Sub-task
>          Components: spark
>            Reporter: Praveen Rachabattuni
>            Assignee: liyunzhang_intel
>         Attachments: PIG-4209.patch, PIG-4209_1.patch, test_harnesss_1417075517, test_harnesss_1422610455
>
>
> Related e2e-tests: ComputeSpec_[1-13]

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Praveen Rachabattuni (JIRA | 30 Jan 10:45 2015
Picon

[Updated] (PIG-4209) Make stream work with Spark


     [
https://issues.apache.org/jira/browse/PIG-4209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Praveen Rachabattuni updated PIG-4209:
--------------------------------------
    Description: Related e2e-tests: ComputeSpec_[1-13]  (was: Related e2e-tests: StreamingLocal_[1-18])

> Make stream work with Spark
> ---------------------------
>
>                 Key: PIG-4209
>                 URL: https://issues.apache.org/jira/browse/PIG-4209
>             Project: Pig
>          Issue Type: Sub-task
>          Components: spark
>            Reporter: Praveen Rachabattuni
>            Assignee: liyunzhang_intel
>         Attachments: PIG-4209.patch, PIG-4209_1.patch, test_harnesss_1417075517
>
>
> Related e2e-tests: ComputeSpec_[1-13]

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Picon

[Commented] (PIG-4395) Exception while executing Pig 0.13


    [
https://issues.apache.org/jira/browse/PIG-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14298352#comment-14298352
] 

Venkatasubramaniam Ramakrishnan commented on PIG-4395:
------------------------------------------------------

Can you please tell me how is this related to the problem at hand?  Is the exception raised an Hadoop-related exception?

I would like follow your line of diagnosis so that I can help you better.

Thx,
Venkat.

> Exception while executing Pig 0.13
> ----------------------------------
>
>                 Key: PIG-4395
>                 URL: https://issues.apache.org/jira/browse/PIG-4395
>             Project: Pig
>          Issue Type: Bug
>          Components: data
>    Affects Versions: 0.13.0
>         Environment: Windows 7 Ultimate, HADOOP_HOME=D:\HADOOP-2-6-0\HADOOP-2.6.0
> JAVA=c:\progra~1\java\jdk1.8.0_25\bin\java
> JAVA_HOME=c:\progra~1\java\jdk1.8.0_25
>            Reporter: Venkatasubramaniam Ramakrishnan
>            Priority: Critical
>              Labels: test
(Continue reading)

Praveen Rachabattuni (JIRA | 30 Jan 09:00 2015
Picon

[Resolved] (PIG-4309) Enable unit test "TestStreaming" for spark


     [
https://issues.apache.org/jira/browse/PIG-4309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Praveen Rachabattuni resolved PIG-4309.
---------------------------------------
    Resolution: Fixed

> Enable unit test "TestStreaming" for spark
> ------------------------------------------
>
>                 Key: PIG-4309
>                 URL: https://issues.apache.org/jira/browse/PIG-4309
>             Project: Pig
>          Issue Type: Bug
>          Components: spark
>            Reporter: liyunzhang_intel
>            Assignee: liyunzhang_intel
>         Attachments: PIG-4309.patch, PIG-4362_4.patch, TEST-org.apache.pig.test.TestStreaming.txt
>
>
> error log is attached

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Praveen Rachabattuni (JIRA | 30 Jan 08:59 2015
Picon

[Commented] (PIG-4309) Enable unit test "TestStreaming" for spark


    [
https://issues.apache.org/jira/browse/PIG-4309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14298344#comment-14298344
] 

Praveen Rachabattuni commented on PIG-4309:
-------------------------------------------

All TestStreaming unit tests are now passing with PIG-4362 patch. Resolving this issue. Thanks [~kellyzly]

> Enable unit test "TestStreaming" for spark
> ------------------------------------------
>
>                 Key: PIG-4309
>                 URL: https://issues.apache.org/jira/browse/PIG-4309
>             Project: Pig
>          Issue Type: Bug
>          Components: spark
>            Reporter: liyunzhang_intel
>            Assignee: liyunzhang_intel
>         Attachments: PIG-4309.patch, PIG-4362_4.patch, TEST-org.apache.pig.test.TestStreaming.txt
>
>
> error log is attached

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

(Continue reading)

jira | 30 Jan 08:00 2015
Picon

Subscription: PIG patch available

Issue Subscription
Filter: PIG patch available (21 issues)

Subscriber: pigdaily

Key         Summary
PIG-4389    Flag to run selected test suites in e2e tests
            https://issues.apache.org/jira/browse/PIG-4389
PIG-4385    testDefaultBootup fails because it cannot find "pig.properties"
            https://issues.apache.org/jira/browse/PIG-4385
PIG-4377    Skewed outer join produce wrong result in some cases
            https://issues.apache.org/jira/browse/PIG-4377
PIG-4341    Add CMX support to pig.tmpfilecompression.codec
            https://issues.apache.org/jira/browse/PIG-4341
PIG-4323    PackageConverter hanging in Spark
            https://issues.apache.org/jira/browse/PIG-4323
PIG-4313    StackOverflowError in LIMIT operation on Spark
            https://issues.apache.org/jira/browse/PIG-4313
PIG-4264    Port TestAvroStorage to tez local mode
            https://issues.apache.org/jira/browse/PIG-4264
PIG-4251    Pig on Storm
            https://issues.apache.org/jira/browse/PIG-4251
PIG-4193    Make collected group work with Spark
            https://issues.apache.org/jira/browse/PIG-4193
PIG-4111    Make Pig compiles with avro-1.7.7
            https://issues.apache.org/jira/browse/PIG-4111
PIG-4004    Upgrade the Pigmix queries from the (old) mapred API to mapreduce
            https://issues.apache.org/jira/browse/PIG-4004
PIG-4002    Disable combiner when map-side aggregation is used
            https://issues.apache.org/jira/browse/PIG-4002
(Continue reading)

Daniel Dai (JIRA | 30 Jan 07:42 2015
Picon

[Commented] (PIG-4395) Exception while executing Pig 0.13


    [
https://issues.apache.org/jira/browse/PIG-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14298279#comment-14298279
] 

Daniel Dai commented on PIG-4395:
---------------------------------

Did you compiled Hadoop? Do you have winutils.exe in your $HADOOP_HOME\bin?

> Exception while executing Pig 0.13
> ----------------------------------
>
>                 Key: PIG-4395
>                 URL: https://issues.apache.org/jira/browse/PIG-4395
>             Project: Pig
>          Issue Type: Bug
>          Components: data
>    Affects Versions: 0.13.0
>         Environment: Windows 7 Ultimate, HADOOP_HOME=D:\HADOOP-2-6-0\HADOOP-2.6.0
> JAVA=c:\progra~1\java\jdk1.8.0_25\bin\java
> JAVA_HOME=c:\progra~1\java\jdk1.8.0_25
>            Reporter: Venkatasubramaniam Ramakrishnan
>            Priority: Critical
>              Labels: test
>
> Pig file with contents below:
> a = LOAD 'o.csv' AS (NAME:chararray,COUNTRY:chararray,YEAR:int,SPORT:chararray,GOLD:int,SILVER:int,BRONZE:int,TOTAL:int);
> DUMP a;
> o.csv contents:
(Continue reading)

Picon

[Commented] (PIG-4395) Exception while executing Pig 0.13


    [
https://issues.apache.org/jira/browse/PIG-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14298261#comment-14298261
] 

Venkatasubramaniam Ramakrishnan commented on PIG-4395:
------------------------------------------------------

Hi Daniel,

Hadoop version installed in my computer is 2.6.0. 

It's windows compatible as per the documentation here:

http://wiki.apache.org/hadoop/Hadoop2OnWindows

I am not using HDP.

Thx,
Venkat.

> Exception while executing Pig 0.13
> ----------------------------------
>
>                 Key: PIG-4395
>                 URL: https://issues.apache.org/jira/browse/PIG-4395
>             Project: Pig
>          Issue Type: Bug
>          Components: data
>    Affects Versions: 0.13.0
(Continue reading)

Mohit Sabharwal (JIRA | 30 Jan 07:07 2015
Picon

[Assigned] (PIG-4307) Enable unit test "TestSplitStore" for spark


     [
https://issues.apache.org/jira/browse/PIG-4307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mohit Sabharwal reassigned PIG-4307:
------------------------------------

    Assignee: Mohit Sabharwal

> Enable unit test  "TestSplitStore" for spark
> --------------------------------------------
>
>                 Key: PIG-4307
>                 URL: https://issues.apache.org/jira/browse/PIG-4307
>             Project: Pig
>          Issue Type: Bug
>          Components: spark
>            Reporter: liyunzhang_intel
>            Assignee: Mohit Sabharwal
>         Attachments: TEST-org.apache.pig.test.TestSplitStore.txt
>
>
> error is attached

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Gmane