Re: Columns limit

So have the TimeUUID as the key?

SearchLogs : {
     TimeUUID_1 : { metadata goes here},
     TimeUUID_2 : { metadata goes here},
     TimeUUID_3 : { metadata goes here},
     ...
}

On 7/31/10 3:42 PM, Benjamin Black wrote:
> The proper way to handle this is to have a row per time interval such
> that the number of columns per row is constrained.
>
> On Thu, Jul 29, 2010 at 2:39 PM, Mark<static.void.dev <at> gmail.com>  wrote:
>    
>> Is there any limitations on the number of columns a row can have? Does all
>> the day for a single key need to reside on a single host? If so, wouldn't
>> that mean there is an implicit limit on the number of columns one can
>> have... ie the disk size of that machine.
>>
>> What is the proper way to handle timelines in this matter. For example lets
>> say I wanted to store all user searches in a super column.
>>
>> <ColumnFamily Name="SearchLogs"
>>                     ColumnType="Super"
>>                     CompareWith="TimeUUIDType"
>>                     CompareSubcolumnsWith="BytesType"/>
>>
>> Which results in a structure as follows
>> {

Re: Upgrading to Cassanda 0.7 Thrift Erlang

Re: Upgrading to Cassanda 0.7 Thrift Erlang

Re: NullPointerException and Java client hang

I was able to reproduce this exception consistently. Then I realized that in /etc/default/cassandra

JVM_START_MEM="256"

It was set to 256 (Bytes). Before It was 128M. I changed it to 256M and made sure that the -Xmx and -Xms in
casandra.in.sh and /etc/init.d/cassandra are all in sync. 

After applying these changes, the exception no longer occurs in my 3 node cluster. BTW, my Xmx is set to 4G.

Hope this helps.

Claire
On Jul 30, 2010, at 1:05 PM, Peter Schuller wrote:

>> I am getting following exception:
>> 
>> java.lang.NullPointerException
>>         at org.apache.cassandra.db.Table.apply(Table.java:407)
> 
> Are you triggering this repeatedly without difficulty?
> 
> Can you run with the attached patch (indentation is messed up in the
> patch though - sorry, no time to fix it now)? Hopefully it should emit
> (at level error) in the log a message indication it failed to obtain a
> column family store, followed by a list of all known column family
> stores, prior to bailing with the same exception.
> 
> Is this happening on every node or just one? When did this start - did
> it start right after a schema change (keyspace addition)?
> 
> (I'm just grasping at straws based on a cursory examination; I may be
> barking up the wrong tree completely.)
> 
> -- 
> / Peter Schuller
> <cassandra-0.6.3-missingcfslog.patch>

Re: Columns limit

Have the TimeUUID as the key, and then index rows named for the time
intervals, each containing columns with TimeUUID names giving the data
in those intervals.

On Sat, Jul 31, 2010 at 5:13 PM, Mark <static.void.dev <at> gmail.com> wrote:
> So have the TimeUUID as the key?
>
> SearchLogs : {
>    TimeUUID_1 : { metadata goes here},
>    TimeUUID_2 : { metadata goes here},
>    TimeUUID_3 : { metadata goes here},
>    ...
> }
>
> On 7/31/10 3:42 PM, Benjamin Black wrote:
>>
>> The proper way to handle this is to have a row per time interval such
>> that the number of columns per row is constrained.
>>
>> On Thu, Jul 29, 2010 at 2:39 PM, Mark<static.void.dev <at> gmail.com>  wrote:
>>
>>>
>>> Is there any limitations on the number of columns a row can have? Does
>>> all
>>> the day for a single key need to reside on a single host? If so, wouldn't
>>> that mean there is an implicit limit on the number of columns one can
>>> have... ie the disk size of that machine.
>>>
>>> What is the proper way to handle timelines in this matter. For example
>>> lets
>>> say I wanted to store all user searches in a super column.
>>>
>>> <ColumnFamily Name="SearchLogs"
>>>                    ColumnType="Super"
>>>                    CompareWith="TimeUUIDType"
>>>                    CompareSubcolumnsWith="BytesType"/>
>>>
>>> Which results in a structure as follows
>>> {
>>>   SearchLogs : {
>>>       "foo" : {
>>>            timeuuid_1 : { metadata goes here}
>>>            timeuuid_2: { metadata goes here}
>>>       },
>>>       "bar" : {
>>>            timeuuid_1 : { metadata goes here}
>>>            timeuuid_2: { metadata goes here}
>>>       }
>>>  }
>>> }
>>>
>>> Couldn't this theoretically run out of columns for the same search term
>>> because for each unique term there can (and will) be many timeuuid
>>> columns?
>>>
>>> Thanks for clearing this up for me.
>>>
>>>
>>>
>
>

RE: kundera: Open source JPA 1.0 compliant ORM for Cassandra

? how to do protocol version negotiation

good morning;

is there somewhere a description of - or code which does, protocol  
version negotiation? particularly client code, across the 0.6/0.7  
transition.

all pointers would be welcome.

Two questions : Server crash during compaction and UnavailableException

What's using my memory?

Re: What's using my memory?

> storms and very slow responses. Also, I'd like to be able to load more data
> to the cluster and I'm hitting the memory wall, which I didn't expect.
> In the cassandra.in.sh you'd notice that I do provide Xmx=12G but given that
> there's so little data I wouldn't expect the process to be using all of
> that. As a matter of fact I wanted to insert more data to the cluster but I

So it seems to be heap space rather than mmap(), given:

> concurrent mark-sweep generation:
>    capacity = 12841320448 (12246.4375MB)
>    used     = 10867324872 (10363.888618469238MB)
>    free     = 1973995576 (1882.5488815307617MB)
>    84.62778353679785% used

The JVM will tend to gobble up the memory you allow it to gobble up
with -Xmx, depending on circumstances. Yes, there are heuristics in
the VM that are intended to prevent it from immediately going to -Xmx
if not needed, but in my experience there are many situations where
these heuristics mostly fail. This is particularly an issue with
CMS/G1, which have to try to stay incremental and avoid pauses while
at the same time trying to do something decent about memory use. The
default throughput collector should be better here (I presume, I
haven't bothered experimenting with it much since it's uninteresting
;)).

The situation is complicated because in general garbage collection
becomes more efficient the more memory you give it, so there is a
direct trade-off between memory use and performance. For this reason
one of the heuristics (which I believe are in place with CMS too;
definitely with G1) is how much time is spent on GC. Certain patterns
can cause this heuristic to cause the heap size to be bumped
(instantly to -Xmx in the case of G1 anyway...).

You can try tweaking CMS settings (there are several, such as forcing
concurrent mark to start at a constant occupancy rate, tweaking
minfree/minused etc), but it is difficult to get right. But by far the
easiest thing to do in your situation is probably to determine roughly
how large the live set is (looking at how much memory is used after a
concurrent mark/sweep as just finished is a good way of doing this)
and then set -Xmx accordingly instead of at 12 GB.

--

-- 
/ Peter Schuller