Paul Fife | 24 Jun 22:51 2016
Picon

Changing a cluster name

I am following the instructions here to attempt to change the name of a cluster: https://wiki.apache.org/cassandra/FAQ#clustername_mismatch
or at least the more up to date advice: http://stackoverflow.com/questions/22006887/cassandra-saved-cluster-name-test-cluster-configured-name

I am able to query the system.local to verify the clusterName is modified, but when I restart Cassandra it fails, and the value is back at the original cluster name. Is this still possible, or are there changes preventing this from working anymore?

I have attempted this several times and it did actually work the first time, but when I moved around to the other nodes it no longer worked.

Thanks,
Paul Fife

Sungju Hong | 24 Jun 06:14 2016
Picon

Question about hector api documentation

Hello,

I'm finding hector java api doc.

I searched though google but couldn't find hector api doc.

This link is broken also.
https://hector-client.github.io/hector/build/html/content/api.html#

Can I know the way to get the doc?

Thanks.

Sungju.
Jean Carlo | 23 Jun 17:22 2016
Picon

Question about sequential repair vs parallel

Cassandra 2.1.12

In the moment of a repair -pr sequential, we are experimenting an exponential increase of number of sstables. For a table lcs.

If someone of you guys kowns if theoritically speaking a sequential repair doing more sstable streaming among replicas than a parallel repair?



Best regards

Jean Carlo

"The best way to predict the future is to invent it" Alan Kay
Mike Heffner | 23 Jun 16:38 2016
Gravatar

Ring connection timeouts with 2.2.6

Hi,

We have a 12 node 2.2.6 ring running in AWS, single DC with RF=3, that is sitting at <25% CPU, doing mostly writes, and not showing any particular long GC times/pauses. By all observed metrics the ring is healthy and performing well.

However, we are noticing a pretty consistent number of connection timeouts coming from the messaging service between various pairs of nodes in the ring. The "Connection.TotalTimeouts" meter metric show 100k's of timeouts per minute, usually between two pairs of nodes for several hours at a time. It seems to occur for several hours at a time, then may stop or move to other pairs of nodes in the ring. The metric "Connection.SmallMessageDroppedTasks.<ip>" will also grow for one pair of the nodes in the TotalTimeouts metric.

Looking at the debug log typically shows a large number of messages like the following on one of the nodes:

StorageProxy.java:1033 - Skipped writing hint for /172.26.33.177 (ttl 0)

We have cross node timeouts enabled, but ntp is running on all nodes and no node appears to have time drift.

The network appears to be fine between nodes, with iperf tests showing that we have a lot of headroom.

Any thoughts on what to look for? Can we increase thread count/pool sizes for the messaging service?

Thanks,

Mike

--

  Mike Heffner <mike <at> librato.com>
  Librato, Inc.

Bienek, Marcin | 23 Jun 10:56 2016
Picon

Upgrade from 3.0.6 to 3.7.

Hi, 

We are short before going in prod with our cassandra cluster. Now I wonder if this maybe  (while still not fully in prod) a good moment to switch from the 3.0.x to the new tick-tock versions.
On planet cassandra the tick-tock article mentions: 

“…We do recognize that it will take some time for tick-tock releases to deliver production-level stability, which is why we will continue to deliver 2.2.y and 3.0.y bugfix releases.  (But if we do demonstrate that tick-tock can deliver the stability we want, there will be no need for a 4.0.y bugfix series, only 4.x tick-tock.)…" 

But the article is from June 2015 so I guess it might not be that valid anymore. What are your experiences with the newer versions ?  
Is there anything to keep in mind before the upgrade from 3.0.x ?  

BR, 
Bieniu 
Bhuvan Rawal | 22 Jun 15:03 2016
Picon
Gravatar

Slow nodetool response time

Hi,

We have been facing slowness in getting response from nodetool for any of its subcommand. On the same version on AWS it responds really fast but on local 1 node machine or local DC cluster it performs very slow.

On Local DC :
$ time nodetool version
ReleaseVersion: 3.0.3

real 0m17.582s
user 0m2.334s
sys 0m0.470s

On AWS:
$ time nodetool version
ReleaseVersion: 3.0.3

real 0m1.084s
user 0m1.772s
sys 0m0.363s

Any way by which its speed can be increased?

Thanks & Regards,
Bhuvan
Joaquin Alzola | 22 Jun 10:58 2016

Spark-Cassandra connector

Hi List

 

I am trying to install the Spark-Cassandra connector through maven or sbt but neither works.

Both of them try to connect to the Internet (which I do not have connection) to download certain files.

 

Is there a way to install the files manually?

 

I downloaded from the maven repository à spark-cassandra-connector_2.10-1.6.0.jar

Which is the version of Scala and Spark that I have …

 

BR

 

Joaquin

This email is confidential and may be subject to privilege. If you are not the intended recipient, please do not copy or disclose its content but contact the sender immediately upon receipt.
Oskar Kjellin | 21 Jun 17:47 2016
Picon
Gravatar

Cluster not working after upgrade from 2.1.12 to 3.5.0

Hi,

We've done this upgrade in both dev and stage before and we did not see similar issues.
After upgrading production today we have a lot issues tho.

The main issue is that the Datastax client quite often does not get the data (even though it's the same query). I see similar flakyness by simply running cqlsh, although it does return it returns broken data.

We are running a 3 node cluster with RF 3.

I have this table

CREATE TABLE keyspace.table (  

  a text,

    b text,

    c text,

    d list<text>,

    e text,

    f timestamp,

    g list<text>,

    h timestamp,

    PRIMARY KEY (a, b, c)

)


Every other time I query (not exactly every other time, but random) I get:


SELECT * from table where a = 'xxx' and b = 'xxx'

 a             | b | c                                 | d | e | f                         | g            | h

---------------------+--------------+-----------------------------------------------+------------------+------------+---------------------------------+-----------------------+---------------------------------

 xxx |          xxx | ccc |             null |       null | 2089-11-30 23:00:00.000000+0000 | ['fff'] | 2014-12-31 23:00:00.000000+0000

 xxx |          xxx |                           ddd |             null |       null | 2099-01-01 00:00:00.000000+0000 | ['fff'] | 2016-06-17 13:29:36.000000+0000


Which is the expected output.


But I also get:

 a             | b | c                                 | d | e | f                         | g            | h

---------------------+--------------+-----------------------------------------------+------------------+------------+---------------------------------+-----------------------+---------------------------------

 xxx |          xxx | ccc |             null |       null |                            null |                  null |                            null

 xxx |          xxx | ccc |             null |       null | 2089-11-30 23:00:00.000000+0000 | ['fff'] |                            null

 xxx |          xxx | ccc |             null |       null |                            null |                  null | 2014-12-31 23:00:00.000000+0000

 xxx |          xxx |                           ddd |             null |       null |                            null |                  null |                            null

 xxx |          xxx |                           ddd |             null |       null | 2099-01-01 00:00:00.000000+0000 | ['fff'] |                            null

 xxx |          xxx |                           ddd |             null |       null |                            null |                  null | 2016-06-17 13:29:36.000000+0000


Notice that the same PK is returned 3 times. With different parts of the data. I believe this is what's currently killing our production environment.


I'm running upgradesstables as of this moment, but it's not finished yet. I started a repair before but nothing happened. The upgradesstables finished now on 2 out of 3 nodes, but production is still down :/


We also see these in the logs, over and over again:

DEBUG [ReadRepairStage:4] 2016-06-21 15:44:01,119 ReadCallback.java:235 - Digest mismatch:

org.apache.cassandra.service.DigestMismatchException: Mismatch for key DecoratedKey(-1566729966326640413, 336b35356c49537731797a4a5f64627a797236) (b3dcfcbeed6676eae7ff88cc1bd251fb vs 6e7e9225871374d68a7cdb54ae70726d)

at org.apache.cassandra.service.DigestResolver.resolve(DigestResolver.java:85) ~[apache-cassandra-3.5.0.jar:3.5.0]

at org.apache.cassandra.service.ReadCallback$AsyncRepairRunner.run(ReadCallback.java:226) ~[apache-cassandra-3.5.0.jar:3.5.0]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_72]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_72]

at java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]


Any help is much appreciated

horschi | 21 Jun 15:31 2016
Picon

C* 2.2.7 ?

Hi,

are there any plans to release 2.2.7 any time soon?

kind regards,
Christian
Atul Saroha | 21 Jun 13:32 2016

MATERIALIZED VIEW difference in 3.0.3 to 3.0.7/3.7

There is  behavioural difference  between 3.0.3 and (3.0.7/3.7) for below schema in materialized view.


CREATE TABLE ks.pa (
    id bigint,
    sub_id text,
    name text,
    class text,
    r_id bigint,
    k_id bigint,
    created timestamp,
    priority int,
    updated timestamp,
    value text,
    PRIMARY KEY (id, sub_id, name)
);

CREATE ks.mv_pa AS
    SELECT k_id, name, value, sub_id, id, class, r_id
    FROM ks.pa
    WHERE k_id IS NOT NULL AND name IS NOT NULL AND value IS NOT NULL AND sub_id IS NOT NULL AND id IS NOT NULL
    PRIMARY KEY ((k_id, name), value, sub_id, id);

We were able to create below MV in 3.0.3 but it fails in 3.0.7/3.7 with following error

InvalidRequest: code=2200 [Invalid query] message="Cannot include more than one non-primary key column 'value' in materialized view partition key"

We are not able to upgrade it.  Also "value" is clustering key and "k_id" is in partition key. Thus, there is only one non-primary key column from main table  in partition key. Then why we are getting this error in 3.0.7/3.7 cassandra.

Help will be appreciated.


---------------------------------------------------------------------------------------------------------------------
Atul Saroha

Lead Software Engineer
M: +91 8447784271 T: +91 124-415-6069 EXT: 12369
Plot # 362, ASF Centre - Tower A, Udyog Vihar,
 Phase -4, Sector 18, Gurgaon, Haryana 122016, INDIA
Joaquin Alzola | 20 Jun 22:47 2016

Spark Cassandra Python Connector

Hi List

Is there a Spark Cassandra connector in python? Of course there is the one for scala ...

BR

Joaquin
This email is confidential and may be subject to privilege. If you are not the intended recipient, please do
not copy or disclose its content but contact the sender immediately upon receipt.


Gmane