This list is for users of Mahout to ask questions, share knowledge, and discuss issues. Do send ()

Lance Norskog | 1 Aug 2011 05:29

Picon

Re: Parallel FPGrowth driver - what is a good demo?

I've rewritten the FPGrowth wiki page. Is still a bit ragged. Please
critique for content.

https://cwiki.apache.org/confluence/display/MAHOUT/Parallel+Frequent+Pattern+Mining

On Thu, Jul 28, 2011 at 12:59 AM, Lance Norskog <goksron <at> gmail.com> wrote:
> Ok, now I've succeeded in running fpgrowth, both sequential and
> mapreduce, from the 'fpg' job and the flag that chooses 'sequential'
> from 'mapreduce'. I've done this from two different datasets,
> accidents.dat and retail.dat. I only ran the first thousand lines of
> both datasets for time reasons.
>
> Both sequential and mapreduce locate the same ids as being in
> patterns. Examining the patterns in detail, they do not match but
> patterns involving id X generally the same size. Successive runs of
> each variant give exactly the same results, so having sequential and
> mapreduce give different result sets is puzzling. Pulling the
> distances is a little difficult with text processing.
>
> What can account for the different outputs of map/reduce and
> sequential (pseudo-distributed) modes?
>
>
>
>
> On 7/27/11, Lance Norskog <goksron <at> gmail.com> wrote:
>> I'll prep a current version.
>>
>> On 7/27/11, Robin Anil <robin.anil <at> gmail.com> wrote:
>>> On Tue, Jul 26, 2011 at 11:06 PM, Lance Norskog <goksron <at> gmail.com>

(Continue reading)

Sean Owen | 1 Aug 2011 09:19

Picon

Re: OSX/Hadoop problem: filename 'LICENSE' and dir 'license/' clash in mahout-examples-0.6-SNAPSHOT-job.jar

Great report here. I imagine the answer is to make 'license' into
'licenses'. Let me have a look and file a JIRA with patch.

Sean

On Sun, Jul 31, 2011 at 6:41 PM, Dan Brickley <danbri <at> danbri.org> wrote:
> With SVN 'At revision 1152597.', and freshly rebuilt:
>
> jar -tvf /Users/danbri/Documents/workspace/trunk/examples/target/mahout-examples-0.6-SNAPSHOT-job.jar
> | grep -i license
>
>  19355 Sat Feb 26 19:16:30 CET 2011 META-INF/LICENSE.txt
>  11358 Sun Apr 11 21:45:12 CEST 2010 META-INF/LICENSE
>  1596 Mon Dec 20 15:47:30 CET 2010 LICENSE
>     0 Sun Dec 01 11:57:24 CET 2002 license/
>  4083 Sun Dec 01 11:57:24 CET 2002 license/LICENSE.dom-documentation.txt
>  3595 Sun Dec 01 11:57:24 CET 2002 license/LICENSE.dom-software.txt
>   804 Sun Dec 01 11:57:24 CET 2002 license/LICENSE.sax.txt
>  2827 Sun Dec 01 11:57:24 CET 2002 license/LICENSE.txt
>  1274 Sun Dec 01 11:57:24 CET 2002 license/README.dom.txt
>   715 Sun Dec 01 11:57:24 CET 2002 license/README.sax.txt
>   672 Sun Dec 01 11:57:24 CET 2002 license/README.txt
>
>
> This situation seems to quite confuse Hadoop. The underlying OSX
> filesystem doesn't support file and directory names differing only by
> case; see http://developer.apple.com/library/mac/#documentation/Java/Conceptual/Java14Development/01-JavaOverview/JavaOverview.html
>
> mahout  lucene.vector --dir solr/data/index/ --output bar/vecs --field
> label --idField id --dictOut bar/dict.out --norm 2

(Continue reading)

Dan Brickley | 1 Aug 2011 10:50

Re: OSX/Hadoop problem: filename 'LICENSE' and dir 'license/' clash in mahout-examples-0.6-SNAPSHOT-job.jar

On 1 August 2011 09:19, Sean Owen <srowen <at> gmail.com> wrote:
> Great report here. I imagine the answer is to make 'license' into
> 'licenses'. Let me have a look and file a JIRA with patch.

Thanks. I tried adding into ./examples/src/main/assembly/job.xml

        <exclude>LICENSE</exclude>
        <exclude>license*</exclude>

...but that was a bad guess. Renaming to 'licenses' sounds better and
simpler; I just didn't track down yet where it's actually coming from.

cheers,

Dan

Jeff Eastman | 1 Aug 2011 18:15

RE: Analyzing the clusterdump output - kmeans clustering


-----Original Message-----
From: Abhik Banerjee [mailto:banerjee.abhik.hcl <at> gmail.com] 
Sent: Friday, July 29, 2011 3:48 PM
To: user <at> mahout.apache.org
Subject: Analyzing the clusterdump output - kmeans clustering

Hi,

I managed to run the kmeans algorithm on a cloudera vm , using the
help provided at the wiki and help at the forum . I got my output and
am trying to use the clusterdump to analyze my result.

 (I seemed to give 5 iterations , but it seems to have formed only 4
clusters , I am also curious about that , I ran this below command )

mahout kmeans -i hdfs://localhost/mahout_input/ip -o
hdfs://localhost/mahout_output/output_kmeans_07_29_1/ -dm
org.apache.mahout.common.distance.EuclideanDistanceMeasure -cd 1.0 -c
hdfs://localhost/mahout_input/centroids_07_29_1 -k 5 -x 5 -cl

after k means completion on hadoop cloudera vm I ran this command :-

mahout clusterdump --seqFileDir
hdfs://localhost/mahout_output/output_kmeans_07_29_1/clusters-5/part-r-00000
--pointsDir hdfs://localhost/mahout_output/output_kmeans_07_29_1/clusteredPoints
--output kmeans_07_29_1_cl5.tx

and when I look into the text file I see a structure like this

(Continue reading)

Jeff Eastman | 1 Aug 2011 18:19

RE: Analyzing the clusterdump output - kmeans clustering

It is possible for kmeans to fail to assign any points to one of its clusters, and I believe this is the
explanation for your seeing only 4 clusters when you requested 5. Also, looking at your clustered data,
the points are printed out in sparse vector notation (index:value) and your input vectors only appear to
have 2 or 3 nonzero elements. This could be the reason for dropping one of the clusters. 

-----Original Message-----
From: Abhik Banerjee [mailto:banerjee.abhik.hcl <at> gmail.com] 
Sent: Friday, July 29, 2011 3:48 PM
To: user <at> mahout.apache.org
Subject: Analyzing the clusterdump output - kmeans clustering

Hi,

I managed to run the kmeans algorithm on a cloudera vm , using the
help provided at the wiki and help at the forum . I got my output and
am trying to use the clusterdump to analyze my result.

 (I seemed to give 5 iterations , but it seems to have formed only 4
clusters , I am also curious about that , I ran this below command )

mahout kmeans -i hdfs://localhost/mahout_input/ip -o
hdfs://localhost/mahout_output/output_kmeans_07_29_1/ -dm
org.apache.mahout.common.distance.EuclideanDistanceMeasure -cd 1.0 -c
hdfs://localhost/mahout_input/centroids_07_29_1 -k 5 -x 5 -cl

after k means completion on hadoop cloudera vm I ran this command :-

mahout clusterdump --seqFileDir
hdfs://localhost/mahout_output/output_kmeans_07_29_1/clusters-5/part-r-00000
--pointsDir hdfs://localhost/mahout_output/output_kmeans_07_29_1/clusteredPoints

(Continue reading)

Jeff Eastman | 1 Aug 2011 18:20

RE: Doubt regarding the kmeans clustering results on mahout

It's really not possible for the clustering to produce NamedVectors but you are free to send it points which
are named. Those points will pass through the clustering process and be available in the output.

-----Original Message-----
From: Frank Scholten [mailto:frank <at> frankscholten.nl] 
Sent: Saturday, July 30, 2011 4:21 AM
To: user <at> mahout.apache.org
Subject: Re: Doubt regarding the kmeans clustering results on mahout

Maybe it should produce NamedVectors by default as well. This is
another of those optional settings
that is often needed in practice.

On Fri, Jul 29, 2011 at 11:42 PM, Jeff Eastman <jeastman <at> narus.com> wrote:
> No problem. I really think the default needs to be changed anyway. Perhaps this will get me to do it.
>
> -----Original Message-----
> From: Abhik Banerjee [mailto:banerjee.abhik.hcl <at> gmail.com]
> Sent: Friday, July 29, 2011 1:48 PM
> To: user <at> mahout.apache.org
> Subject: Re: Doubt regarding the kmeans clustering results on mahout
>
> Thanks a lot , I missed that part in the wiki , My mistake.
>
>

Abhik Banerjee | 1 Aug 2011 18:40

Picon

Re: Analyzing the clusterdump output - kmeans clustering

Hi Jeff, Thanks a lot for your help.

Frank Scholten | 1 Aug 2011 21:05

Picon

Re: Doubt regarding the kmeans clustering results on mahout

Ah, I meant maybe seq2sparse could produce namedvectors by default.
There was a discussion on that some time ago on
https://issues.apache.org/jira/browse/MAHOUT-401

On Mon, Aug 1, 2011 at 6:20 PM, Jeff Eastman <jeastman <at> narus.com> wrote:
> It's really not possible for the clustering to produce NamedVectors but you are free to send it points
which are named. Those points will pass through the clustering process and be available in the output.
>
> -----Original Message-----
> From: Frank Scholten [mailto:frank <at> frankscholten.nl]
> Sent: Saturday, July 30, 2011 4:21 AM
> To: user <at> mahout.apache.org
> Subject: Re: Doubt regarding the kmeans clustering results on mahout
>
> Maybe it should produce NamedVectors by default as well. This is
> another of those optional settings
> that is often needed in practice.
>
> On Fri, Jul 29, 2011 at 11:42 PM, Jeff Eastman <jeastman <at> narus.com> wrote:
>> No problem. I really think the default needs to be changed anyway. Perhaps this will get me to do it.
>>
>> -----Original Message-----
>> From: Abhik Banerjee [mailto:banerjee.abhik.hcl <at> gmail.com]
>> Sent: Friday, July 29, 2011 1:48 PM
>> To: user <at> mahout.apache.org
>> Subject: Re: Doubt regarding the kmeans clustering results on mahout
>>
>> Thanks a lot , I missed that part in the wiki , My mistake.
>>
>>

(Continue reading)

Jeff Eastman | 1 Aug 2011 21:57

RE: Doubt regarding the kmeans clustering results on mahout

That issue has been closed and seq2sparse, in trunk at least, reports it has a -nv (--namedVector) option.
Have you tried it?

-----Original Message-----
From: Frank Scholten [mailto:frank <at> frankscholten.nl] 
Sent: Monday, August 01, 2011 12:06 PM
To: user <at> mahout.apache.org
Subject: Re: Doubt regarding the kmeans clustering results on mahout

Ah, I meant maybe seq2sparse could produce namedvectors by default.
There was a discussion on that some time ago on
https://issues.apache.org/jira/browse/MAHOUT-401

On Mon, Aug 1, 2011 at 6:20 PM, Jeff Eastman <jeastman <at> narus.com> wrote:
> It's really not possible for the clustering to produce NamedVectors but you are free to send it points
which are named. Those points will pass through the clustering process and be available in the output.
>
> -----Original Message-----
> From: Frank Scholten [mailto:frank <at> frankscholten.nl]
> Sent: Saturday, July 30, 2011 4:21 AM
> To: user <at> mahout.apache.org
> Subject: Re: Doubt regarding the kmeans clustering results on mahout
>
> Maybe it should produce NamedVectors by default as well. This is
> another of those optional settings
> that is often needed in practice.
>
> On Fri, Jul 29, 2011 at 11:42 PM, Jeff Eastman <jeastman <at> narus.com> wrote:
>> No problem. I really think the default needs to be changed anyway. Perhaps this will get me to do it.
>>

(Continue reading)

Jeff Eastman | 1 Aug 2011 22:39

RE: Doubt regarding the kmeans clustering results on mahout

Ah, ok, I guess you meant both should produce those behaviors by default rather than upon request. That
would require either changing the semantics of -cl & -nv to disable these behaviors vs. enabling them,
removing both options and introducing new options to disable them, or making both options require a
true/false value (default=true). None are particularly attractive, IMHO.

-----Original Message-----
From: Jeff Eastman [mailto:jeastman <at> Narus.com] 
Sent: Monday, August 01, 2011 12:57 PM
To: user <at> mahout.apache.org
Subject: RE: Doubt regarding the kmeans clustering results on mahout

That issue has been closed and seq2sparse, in trunk at least, reports it has a -nv (--namedVector) option.
Have you tried it?

-----Original Message-----
From: Frank Scholten [mailto:frank <at> frankscholten.nl] 
Sent: Monday, August 01, 2011 12:06 PM
To: user <at> mahout.apache.org
Subject: Re: Doubt regarding the kmeans clustering results on mahout

Ah, I meant maybe seq2sparse could produce namedvectors by default.
There was a discussion on that some time ago on
https://issues.apache.org/jira/browse/MAHOUT-401

On Mon, Aug 1, 2011 at 6:20 PM, Jeff Eastman <jeastman <at> narus.com> wrote:
> It's really not possible for the clustering to produce NamedVectors but you are free to send it points
which are named. Those points will pass through the clustering process and be available in the output.
>
> -----Original Message-----
> From: Frank Scholten [mailto:frank <at> frankscholten.nl]

(Continue reading)

Group

gmane.comp.apache.mahout.user.

Project Web Page

This list is for users of Mahout to ask questions, share knowledge, and discuss issues. Do send ()

Search Archive

Page

1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 41

Articles in period: 401

August 2011

Language

Change language

Options

Current view: Threads only / Showing only 20 lines / Not hiding cited text.
Change to All messages, whole messages, or hide cited text.

Post a message
NNTP Newsgroup
Classic Gmane web interface
XML

XML

RSS Feed
List Information

About Gmane

Recent entries

Multiple document collections using SparseVectorsFromSequenceFiles (24 May 2013)
FileNotFoundException while trying to run ItemSimilarityJob (24 May 2013)
Mahout Cluster attributes (24 May 2013)
running lda in mahout (24 May 2013)
FileNotFoundException while trying to run ItemSimilarityJob (24 May 2013)
SSVD outputs different things vs R version of SVD (24 May 2013)
running mahout -- plzz help (23 May 2013)
How to set the generic options (23 May 2013)
Implementing the General Bayesian Inference ML Algorithm (23 May 2013)
how to run mahout examples from eclipse in linux? (23 May 2013)
Hidden Markov Models and time series - 2 questions (22 May 2013)
mahout ssvd tuning problem (22 May 2013)
mahout ssvd tuning problem (22 May 2013)
WELCOME to user@mahout.apache.org (22 May 2013)
Interpreting Cluster Dump Metrics (22 May 2013)
Feature vector generation from Bag-of-Words (21 May 2013)
convert input for SVD (20 May 2013)
mahout colt collections (19 May 2013)
Which database should I use with Mahout (19 May 2013)
Set Random Forest Max Depth? (18 May 2013)
More Cross-recommender thoughts (17 May 2013)

Archives

274	May 2013
281	April 2013
338	March 2013
288	February 2013
368	January 2013
224	December 2012
303	November 2012
247	October 2012
332	September 2012
360	August 2012
453	July 2012
343	June 2012
385	May 2012
314	April 2012
416	March 2012
359	February 2012
459	January 2012
540	December 2011
619	November 2011
410	October 2011
378	September 2011
401	August 2011
493	July 2011
547	June 2011
466	May 2011
523	April 2011
555	March 2011
498	February 2011
285	January 2011
309	December 2010
406	November 2010
367	October 2010
359	September 2010
247	August 2010
308	July 2010
93	June 2010
168	May 2010
226	April 2010
176	March 2010
253	February 2010
441	January 2010
263	December 2009
224	November 2009
116	October 2009
171	September 2009
138	August 2009
315	July 2009
223	June 2009
142	May 2009
168	April 2009
42	March 2009
26	February 2009
9	January 2009
19	December 2008
19	November 2008
66	October 2008
25	September 2008
44	August 2008
20	July 2008
9	June 2008
8	May 2008

Design

Zawodny | Right Menu | Left Menu

Your Own Design

Paste the URL of your CSS below. Download CSS template