Josh Walton | 18 Dec 20:41 2013
Picon

Trident State Query Problem

I have an in memory, Java database that I want to use for my Trident State. My testing consists of running a
limited set of data through the topology, and then querying the results through a DRPC stream that
performs a Trident state query.

When the trident state is in a single executor, the same state query returns the same data every time. When
the trident state is in a bolt with multiple executors, I get X different results for the same state query
where X is the number of executors containing the trident state.  

Is there a way to specify which trident state is used for a DRPC query? Is there a way to get the results (and
aggregate) from the all of the different trident state instances?

I'm going to put together a simple example to run and post it here, soon.

--

-- 
You received this message because you are subscribed to the Google Groups "storm-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to storm-user+unsubscribe@...
For more options, visit https://groups.google.com/groups/opt_out.

churly lin | 18 Dec 09:57 2013
Picon

Question about spout's Executors' parallelism hint

Hi all:

I have set the numWorkers = 20 and the number of Spout is 20.

But Why there is only 1 of the 20 spout executor emit tuples while the topology is running?

--

-- 
You received this message because you are subscribed to the Google Groups "storm-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to storm-user+unsubscribe@...
For more options, visit https://groups.google.com/groups/opt_out.

churly lin | 18 Dec 09:41 2013
Picon

Question about spout's Executors' parallelism hint

Hi all:

I have set the numWorkers = 20 and the number of Spout is 20, setMaxSpoutPending = 20.
But Why there is only 1 of the 20 spout executor emit tuples while the topology is running?
Id      Uptime  host    Port    Emitted Transf  Complete latency acked failed
[1-1]	6m 16s	longzh	29008	801160	801160	18.819	400580	0
[10-10]	6m 16s	longzh	29010	0	0	0.000	0	0
[11-11]	6m 16s	longzh	29010	0	0	0.000	0	0
[12-12]	6m 16s	longz	29010	0	0	0.000	0	0
[13-13]	6m 16s	longzh	29011	20	0	0.000	0	0
[14-14]	6m 16s	longzhc	29011	0	0	0.000	0	0
[15-15]	6m 16s	loc	29011	0	0	0.000	0	0
[16-16]	6m 16s	lon	29011	0	0	0.000	0	0
[17-17]	6m 16s	lon	29012	0	0	0.000	0	0
[18-18]	6m 16s	longzh	29012	0	0	0.000	0	0
[19-19]	6m 16s	lon	29012	0	0	0.000	0	0
[2-2]	6m 16s	lon	29008	0	0	0.000	0	0
[20-20]	6m 16s	long	29012	0	0	0.000	0	0
[3-3]	6m 15s	lo	29008	20	0	0.000	0	0
[4-4]	6m 16s	lon	29008	0	0	0.000	0	0
[5-5]	6m 16s	lo	29009	0	0	0.000	0	0
[6-6]	6m 16s	lo	29009	0	0	0.000	0	0
[7-7]	6m 16s	lon	29009	0	0	0.000	0	0
[8-8]	6m 16s	lonc	29009	0	0	0.000	0	0
[9-9]	6m 16s	lo	29010	0	0	0.000	0	0

--

-- 
You received this message because you are subscribed to the Google Groups "storm-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to storm-user+unsubscribe@...
For more options, visit https://groups.google.com/groups/opt_out.
(Continue reading)

Nathan Marz | 13 Dec 00:57 2013

user@...

Now that 0.9.0 is out we are transitioning everything over to Apache infrastructure. I've just set the mailing list to announce only, so please send all future mailing list discussions to the Apache list: user-Scpny14plQLlwc3o7xk8Zw@public.gmane.orgor.apache.org

Thanks!

--
You received this message because you are subscribed to the Google Groups "storm-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to storm-user+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit https://groups.google.com/groups/opt_out.
Amit Singh | 12 Dec 19:22 2013
Picon

Nimbus going Out of Memory

I am facing a issue where internal security scan always results into nimbus going out of memory and since the process does not die , it renders any process monitors we have in place useless.
Is there any way we can self destruct the process or any other fix to get around this problem. Using storm version : 0.8.3


Thanks,
Amit.

--
You received this message because you are subscribed to the Google Groups "storm-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to storm-user+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit https://groups.google.com/groups/opt_out.
Christopher Schmidt | 9 Dec 07:15 2013
Picon

DRPC / Trident

Hi all,

we are using Storm 0.8.2 and the "standard" DRPC / Trident implementation (with DRPCSpout.java etc.).
Two questions:

  1. One if our each-function can have a processing time of more than 60 seconds. Are there any timeouts that I have to be aware of? What do I have to configure?
  2. We do not need any guaranties neither of processing all batches nor the ordering of batches. Is this guaranty switched on by default? What do I have to do to switch it off?
We have some issues in our topology. We think that a function does not return (maybe at all or long running) and that the trident guaranties are blocking other batches to come through. 
What happens is, that we get Read timeouts at the DRPCClient (execute method) then.

thanks in advance
Christopher

--
You received this message because you are subscribed to the Google Groups "storm-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to storm-user+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit https://groups.google.com/groups/opt_out.
Researcher | 13 Dec 00:51 2013
Picon

storm UI & storm configuration

i want to know what's the abbreviation of Storm UI ?
and
what's the important of Storm configuration ?

i installed Storm Cluster without made anything in storm configuration "Storm.yaml"
1- running zookeeper
127.0.01:2181
2-./storm nimbus
3-./storm supervisor
4-./storm ui


--
You received this message because you are subscribed to the Google Groups "storm-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to storm-user+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit https://groups.google.com/groups/opt_out.
Kiran | 12 Dec 21:16 2013
Picon

Nimbus goes out of memory

Hi,

    I see that storm nimbus goes out memory during internal security scan. This renders any process monitors on our end ineffective. I had seen a earlier discussion which mentioned it was due to malformed request being sent on the thrift port.

Is there fix available for this, or is there a way to quit the JVM . I am using storm version : 0.8.3

For now i am planning to use -XX:OnOutOfMemoryError="kill -9 %p”. Has anyone already tried this out ?

Please advice.

Thanks.

--
You received this message because you are subscribed to the Google Groups "storm-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to storm-user+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit https://groups.google.com/groups/opt_out.
Michael Segel | 12 Dec 21:13 2013
Picon

Tracking Supervisors?

Ok,
Weird thing happened to me on my test cluster(S).

I have one cluster on Centos 5, and a small cluster on Centos 6.
Both use the same Zookeeper for management. (These are really small clusters.)

I notices that even though I took down my Centos 5 Cluster, when I was restarting my nimbus and ui, the cluster found the two supervisors running on the Centos 6 machines.

What I found strange is that the storm.yamls  point to different nimbus nodes. (Centos5 points to nimbus on a Centos5 machine and Centos 6 points to nimbus on Centos 6.)

My guess is that the ZK doesn't differentiate the supervisor nodes based on who they have as Nimbus in their storm.yaml files.

Is that the case?  Or should I stop taking LSD and Meth? ;-)

Thx

-Mike

--
You received this message because you are subscribed to the Google Groups "storm-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to storm-user+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit https://groups.google.com/groups/opt_out.

Sasi Panja | 12 Dec 19:57 2013
Picon

Plugin bolts dynamically into existing topology

I have a storm topology with many bolts, each bolt running some business logic, such as a ML algorithm on the stream, to display real-time  results.

I would like to plug-in new bolts ( which have some built in algorithm logic) at runtime into the topolgy. Is this possible? The idea is that I have a topology with an existing configuration of spouts and bolts, but I want to be able to add on new bolts dynamically to execute on the tuples emitted by the exiting bolts.

Thanks
Sasi

--
You received this message because you are subscribed to the Google Groups "storm-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to storm-user+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit https://groups.google.com/groups/opt_out.
Klausen Schaefersinho | 12 Dec 15:58 2013
Picon

State-full Bolts

Hi,

i have the need to process quite a lot of data in form of an event stream. Each event is connected to an user and for most of my operations have to operate on the entire set of events for a given users. So every time a new event arrives I have to load all events from some data storage, do my processing, store the event in the storage  and send the results somewhere. The problem is that reading the old events from the storage is quite expensive, in particular if done over a network. Therefore it would make sense to use the machine the bolt is running on for storage as well. This way I can exploit locality and reduce network bandwidth. Does storm support this kind of scenario some how? 


Cheers,

Klaus

--
You received this message because you are subscribed to the Google Groups "storm-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to storm-user+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit https://groups.google.com/groups/opt_out.

Gmane