Pauls, Gerhard | 9 Feb 09:17 2016
Picon

Distribution of Replicas among nodes

Hi all,
 
i want to setup a Riak DB such that:
 
all nodes have one replica of the data, so that any single node can work on it’s own in case of network failures.
 
I’m intending to reach that goal by:
 
  • Setting n_val equal number of nodes, e.g. 4
  • Making sure that the Ring Size (e.g. 64) can be divided evenly by the number of nodes
 
Is it correct that in this case all the data will always be distributed evenly on all the nodes, that means each node will have a copy of all the data?
 
Kind regards,
 
Gerhard
 
Wichtiger Hinweis: Diese E-Mail kann Betriebs- oder Geschäftsgeheimnisse oder sonstige vertrauliche Informationen enthalten. Sollten Sie diese E-Mail irrtümlich erhalten haben, ist Ihnen eine Kenntnisnahme des Inhalts, eine Vervielfältigung oder Weitergabe der E-Mail ausdrücklich untersagt. Bitte benachrichtigen Sie uns und vernichten Sie die empfangene E-Mail. Vielen Dank.

Important Note: This e-mail may contain trade secrets or privileged, undisclosed or otherwise confidential information. If you have received this e-mail in error, you are hereby notified that any review, copying or distribution of it is strictly prohibited. Please inform us immediately and destroy the original transmittal. Thank you for your cooperation.
_______________________________________________
riak-users mailing list
riak-users <at> lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Joe Olson | 8 Feb 20:36 2016

A Solr join query in Riak KV?

I'm trying to get a Solr join query to work on our Riak KV cluster.

The Solr join query is documented here:

https://wiki.apache.org/solr/Join

Using the example under the "Compared to SQL" heading, I am formatting my http request to Riak as:

curl "http://<cluster IP>:8098/search/query/<riak-index>?wt=json&df=_yz_rk&fl=xxx,yyy&q={%21join+from=<inner_id>+to=<outer_id>}zzz:vvv" | jsonpp

The Riak listener will accept this query, but will return no documents. I've verified each half of the query returns data when queried standalone using the non-join notation.

Is anyone currently using Solr localparms notation succ essfully in Riak KV?
_______________________________________________
riak-users mailing list
riak-users <at> lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Stanislav Vlasov | 8 Feb 14:49 2016
Picon

Cluster balance problem

Hello!

We have riak cluster for riak-cs.
Failed node riak <at> 192.168.0.133 has been replaced via 'riak-admin
replace' by riak <at> 192.168.0.141 (new node), then cleaned up, prepared
and joined to cluster as new.
Member-status after last 'riak-admin cluster commit', when all
transfers complete:

================================= Membership ==================================
Status     Ring    Pending    Node
-------------------------------------------------------------------------------
valid       8.6%      --      'riak <at> 192.168.0.130'
valid       8.2%      --      'riak <at> 192.168.0.131'
valid       8.2%      --      'riak <at> 192.168.0.132'
valid       8.2%      --      'riak <at> 192.168.0.133'
valid       8.2%      --      'riak <at> 192.168.0.134'
valid       8.2%      --      'riak <at> 192.168.0.135'
valid       8.2%      --      'riak <at> 192.168.0.136'
valid       8.2%      --      'riak <at> 192.168.0.137'
valid       8.2%      --      'riak <at> 192.168.0.138'
valid       8.2%      --      'riak <at> 192.168.0.139'
valid       8.2%      --      'riak <at> 192.168.0.140'
valid       9.4%      --      'riak <at> 192.168.0.141'
-------------------------------------------------------------------------------
Valid:12 / Leaving:0 / Exiting:0 / Joining:0 / Down:0

Every node have 3.6T raid.
Free place on disk and used %:

192.168.0.130 996G 73%
192.168.0.131 1.2T 69%
192.168.0.132 1.2T 68%
192.168.0.133 1.1T 70%
192.168.0.134 1.1T 70%
192.168.0.135 1.2T 69%
192.168.0.136 1.2T 68%
192.168.0.137 1.2T 69%
192.168.0.138 1.2T 69%
192.168.0.139 1.2T 69%
192.168.0.140 1.2T 68%
192.168.0.141 808G 78%

Problem: on 192.168.0.141 very little free space compared to rest,
which may affect in merges on this node after uploading some
additional data into cluster.

Software versions:
ii riak 2.1.3-1 amd64 Riak is a distributed data store
ii riak-cs 2.0.0-1 amd64 Riak CS

Can i rebalance riak without adding/removing nodes or any hardware changes?

--

-- 
Stanislav
Dorothy Pults | 5 Feb 01:07 2016

Riak Recap Feb 4, 2016

Here is a summary of what's come over our user list since the last recap.


## Announcements


3 new lager releases are now available. [1]


* 3.1.0 - Introduces a new API to force rotations of a handler, sink or all. It also introduces a new "rotate" message for 3rd party backends - hence the new minor version number.

* 3.0.3 - Contains community-contributed  PRs and various bug fixes

* 2.2.1 - Includes a fix by Shino related to Unicode byte values and a backport of a fix to remove backends properly when stopping traces.


## Recently Answered


* Jason offered insight into a Solr issue [2][3]

* Luke gave guidance on how to troubleshoot high CPU usage using Riak with RabbitMQ [4]

* Russell and Vitaly gave some insight  to a questions about CRDT appending [5][6]

* Fred explained that secondary indexes are not supported if bucket type has write_once option set to true [7]

* Luke and Shino helped John work through an issue he was having with Riak-S2 javascript aws-sdk multi-part uploads[8][9][10]

* Shino made suggestions on how to change the user admin key after it’s been created [11]

* Vitaly added to the conversation about practical limits for bucket number and Solr indexes

original thread [12] new thread [13]

* There’s a question about whether it’s possible to put a map phase in chain of map-reduce job after reduce phase [14]  Ivaylo provided information along with code that pulls keys and loads Riak objects in blocks  [15]

* Russell explains how to get all keys from a bucket using leveldb’s special $bucket index  [16]

* Luke and Richard offered insight into what could be causing high FSM Time [17]

* Zeeshan explained what to do if you are using a CRDT Map bucket with  Riak Search (2.x)  and the search results do not return any fields of the map that are CRDT sets [18]

* Magnus explained how erlang schedulers can cause high CPU usage [19]

* Magnus answered a question on vm.swappiness  [20]


## Open Discussions


* Kinson is looking for guidance on  implementing Basho Bench  [21]

* Luke is helping troubleshoot a mapreduce timeout issue [22]


## Weekend Reads

* Tech Target interviewed Basho’s own Dave McCrory for their article titled, “Choose the Right PaaS for Developing Cloud Applications.” [23]


Have a great weekend,
Dorothy Pults

Basho Technologies

dpults <at> basho.com

<at> deepults


[1] https://github.com/basho/lager

[2] http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-January/017998.html

[3] http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-January/018000.html

[4] http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-January/018034.html

[5] http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-January/018005.html

[6] http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-January/018008.html

[7] http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-January/018010.html

[8] http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-January/018014.html

[9] http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-January/018031.html

[10] http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-January/018035.html

[11] http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-January/018040.html

[12] http://lists.basho.com/pipermail/riak-users_lists.basho.com/2015-November/017856.html

[13] http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-January/018038.html

[14]  http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-January/018037.html  

[15]  http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-January/018041.html

[16] http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-January/018067.html

[17] http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-January/018055.html

[18] http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-January/018060.html

[19] http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-January/018064.html

[20] http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-February/018072.html

[21] http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-January/018039.html

[22] http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-January/018045.html

[23] http://searchcloudapplications.techtarget.com/news/4500271622/Choose-the-right-PaaS-for-developing-cloud-applications


_______________________________________________
riak-users mailing list
riak-users <at> lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Weixi Yen | 1 Feb 20:01 2016

List Keys for a bucket not returning anything

I'm trying delete and rebuild yokozuna search indexes.

One idea was to just list all the buckets and all the keys, and trigger an update to rebuild the index.

However, the following commands don't seem return any buckets or keys.

HTTP/1.1 200 OK
Vary: Accept-Encoding
Transfer-Encoding: chunked
Server: MochiWeb/1.1 WebMachine/1.10.8 (that head fake, tho)
Date: Mon, 01 Feb 2016 18:57:11 GMT
Content-Type: application/json

{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}

curl -i http://localhost:8098/buckets?buckets=true

HTTP/1.1 200 OK
Vary: Accept-Encoding
Server: MochiWeb/1.1 WebMachine/1.10.8 (that head fake, tho)
Date: Mon, 01 Feb 2016 18:56:47 GMT
Content-Type: application/json
Content-Length: 14

{"buckets":[]}

(My bucket_type is 'comment', and bucket is named 'comment')
(This is for Riak 2.1.2-0-gf969bba)

Thanks in advance!

_______________________________________________
riak-users mailing list
riak-users <at> lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Sakoulas, Byron | 29 Jan 19:26 2016
Picon

vm.swappiness

Is it possible that Basho’s recommendation on vm.swappiness is out of date?

The current recommendation (per http://docs.basho.com/riak/latest/ops/tuning/linux/) is still to
set vm.swappiness to 0.

Based on https://www.percona.com/blog/2014/04/28/oom-relation-vm-swappiness0-new-kernel/ it
appears that in kernel 3.5 and up, the behavior of vm.swappiness=0 was changed.

Due to that, should we modify our vm.swappiness to 1?

Thanks

This email and attachments contain information that may be confidential or privileged. If you are not the
intended recipient, notify the sender at once and delete this message completely from your information
system. Further use, disclosure, or copying of information contained in this email is not authorized,
and any such action should not be construed as a waiver of privilege or other confidentiality protections.
_______________________________________________
riak-users mailing list
riak-users <at> lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Russell Brown | 29 Jan 15:06 2016

Re: Re[2]: Get all keys from an bucket

With leveldb you can use the special $bucket index. You can also stream the keys, and paginate them, meaning you can get them in smaller lumps, hopefully this will appear faster and avoid the timeout you're seeing.


On 29 Jan 2016, at 14:03, Markus Geck <zerebokep <at> mail.ru> wrote:

Yes, sorry I forgot to mention that.


Monday, January 25, 2016 10:10 AM UTC from Russell Brown <russell.brown <at> me.com>:

Hi Markus,
Are you using leveldb backend?

Russell

On 22 Jan 2016, at 19:05, Markus Geck <zerebokep <at> mail.ru> wrote:

> Hello,
>
> is there any way to get all keys from an bucket?
>
> I've already tried this guide: http://www.paperplanes.de/2011/12/13/list-all-of-the-riak-keys.html But riak always wents unresponsive with a huge server load.
>
> and "GET /buckets/bucket/keys?keys=stream" returns an timeout error.
>
> Is there any other way?
> _______________________________________________
> riak-users mailing list
> riak-users <at> lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
riak-users <at> lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Eugene Shubin | 28 Jan 14:10 2016
Picon

map reduce on multiple buckets

Is it possible to run mapreduce job on two or more buckets?
I see from documentation that it might be possible if I specify inputs as list of {bucket, key} pairs,
although list of secondary index inputs causes an error:
riakc_pb_socket:mapred(P, [
    {index, Bucket1, Index1, From, To},
    {index, Bucket2, Index2, From, To}
  ], ...
 {error,<<"{inputs,{\"Inputs target tuples must be {B,K} or {{B,K},KeyData}:\",\n

Is it possible using Solr (riak search) indexes?

Evgenii Shubin
_______________________________________________
riak-users mailing list
riak-users <at> lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Fasil K | 28 Jan 10:05 2016
Gravatar

High CPU usage by beam.smp


Hello, I am using riak 2.1.1 for saving some datas (50000 datas). 
I am running riak with a single node so far. my problem is riak is consuming almost 40% of CPU in idle state. Can any one help me to solve this issue.?


With Regards,

Fasil K
_______________________________________________
riak-users mailing list
riak-users <at> lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Weixi Yen | 27 Jan 20:34 2016

Riak Search 2.0 - CRDT Sets within Maps not returned in search results

I have a CRDT Map bucket that I'm running searches on in Yokozuna (using _yz_default index for now)

However, the search results do not return any fields of the map that are CRDT sets.  Is this intended behavior?

Thanks in advance!
_______________________________________________
riak-users mailing list
riak-users <at> lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Alex Wolfe | 26 Jan 20:40 2016
Gravatar

Undiagnosed High FSM Time

We have a 5 node Riak cluster running 2.1.1. This morning FSM Time (99th percentile) went way up. We couldn't
find any clear signs of trouble with the cluster and ultimately chose to move the data files and restart the
nodes. Once we started with an empty DB, the FSM Time normalized. But now it's headed back up again. We're
stumped on how to trouble shoot this issue. Any suggestions?

Gmane