eric | 23 Oct 20:36 2014

http://packages.couchbase.com

Some time in the recent past, couchbase appears to have made some changes to
their http://packages.couchbase.com server, which broke the supermarket
couchbase cookbook (https://supermarket.getchef.com/cookbooks/couchbase).

I've been seeing 403 errors and chef throws this when running the client
recipe:

==> default: STDERR:
http://packages.couchbase.com/rpm/couchbase-rpm.key/repodata/repomd.xml: [Errno
14] PYCURL ERROR 22 - "The requested URL returned error: 403 Forbidden"

Here's the debug output from a failed couchbase::client run.  FYI, it was run
against a slightly modified version of the cookbook that simply replaces :

==> default: * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* * * * * * *
==> default: [2014-10-23T18:12:16+00:00] INFO: *** Chef 11.16.4 ***
==> default: [2014-10-23T18:12:16+00:00] INFO: Chef-client pid: 16014
==> default: [2014-10-23T18:12:22+00:00] INFO: Setting the run_list to
["recipe[couchbase::client]"] from CLI options
==> default: [2014-10-23T18:12:22+00:00] INFO: Run List is
[recipe[couchbase::client]]
==> default: [2014-10-23T18:12:22+00:00] INFO: Run List expands to
[couchbase::client]
==> default: [2014-10-23T18:12:22+00:00] INFO: Starting Chef Run for
rubensteintech-berkshelf
==> default: [2014-10-23T18:12:22+00:00] INFO: Running start handlers
==> default: [2014-10-23T18:12:22+00:00] INFO: Start handlers complete.
==> default: [2014-10-23T18:12:22+00:00] INFO: WindowsPackage lightweight
resource already initialized -- overriding!
(Continue reading)

Justin Dossey | 23 Oct 18:17 2014

Ruby, Chef, proxies

Hi all,

I work at a place that requires that I use a proxy to access the Internet. 

Support for proxies in Ruby stuff is uneven, probably because most people don't have to support proxies.

I wrote a script to test different methods of GETting a public page in Ruby.  It addresses the questions:
  • Does this method respect the http_proxy (or HTTP_PROXY) environment variable?
  • If it does, does it respect the no_proxy (or NO_PROXY) environment variable?

Here's a gist of the code:

https://gist.github.com/justindossey/7c6e0a62ec34ce0bf317

The output on my system is posted as a comment.

I'd love to see other clients in here! I tried to stick to stuff that was installed via ChefDK.

A couple of things I want to highlight about the test:
  • In ruby's standard Net::HTTP, the way you perform the request strongly influences whether your proxy is respected.  Net::HTTP.get_request and Net::HTTP.get do not honor proxies.  Even Net::HTTP.start() doesn't honor your proxy.  You have to do Net::HTTP.new().start in order to get proxy support.
  • open-uri is proxy-friendly (but you give up a bit of control of request headers, etc)
  • Ruby doesn't honor wildcards in the no_proxy or NO_PROXY environment variable.  Chef, however, does (in Chef::HTTP::BasicClient).
  • I didn't test Chef::HTTP::BasicClient because my test runs outside of Chef, but I'm pretty sure it does the right thing.

If you want to run this, you need:

  1. ChefDK, or have the HTTP and Faraday gems installed;
  2. A Proxy server to use

--
Justin Dossey
Practice Owner
New Context Services, Inc
Jeff Blaine | 23 Oct 16:24 2014
Picon

"Last release: 11.14.2"

I don't think I understand Chef releases, or the current CHANGELOG.md
on master is incorrect.

The master branch CHANGELOG.md currently states:

   Unreleased: 12.0.0
   ...
   Last release: 11.14.2

Yet there were no less than 4 official Chef releases between those.

Do I just not know how your process works?

--

-- 
Jeff Blaine
kickflop.net
PGP/GnuPG Key ID: 0x0C8EDD02

Varun Shankar | 23 Oct 12:01 2014
Picon

Not able to use IAM role with knife ec2 command

I am using following version:
knife-ec2-0.8.0
Chef: 11.16.0

--use-iam-profile
Use the Identity and Access Management (IAM) that is assigned to the current machine. Default value: false.

But I don't see this option available in knife ec2 command.

[ec2-user <at> ip-10-1-0-30 ~]$ knife ec2 server list --use-iam-profile
Error: invalid option: --use-iam-profile
USAGE: knife ec2 server list (options)
    -A, --aws-access-key-id KEY      Your AWS Access Key ID
        --aws-credential-file FILE   File containing AWS credentials as used by aws cmdline tools
    -K SECRET,                       Your AWS API Secret Access Key
        --aws-secret-access-key
        --availability-zone          Show availability zones
    -s, --server-url URL             Chef Server URL
        --chef-zero-host HOST        Host to start chef-zero on
        --chef-zero-port PORT        Port to start chef-zero on
    -k, --key KEY                    API Client Key
        --[no-]color                 Use colored output, defaults to false on Windows, true otherwise
    -c, --config CONFIG              The configuration file to use
        --defaults                   Accept default values for all questions
    -d, --disable-editing            Do not open EDITOR, just accept the data as is
    -e, --editor EDITOR              Set the editor to use for interactive commands
    -E, --environment ENVIRONMENT    Set the Chef environment (except for in searches, where this will be flagrantly ignored)
    -F, --format FORMAT              Which format to use for output
    -z, --local-mode                 Point knife commands at local repository instead of server
    -n, --no-name                    Do not display name tag in output
    -u, --user USER                  API Client Username
        --print-after                Show the data after a destructive operation
        --region REGION              Your AWS region
    -t, --tags TAG1,TAG2             List of tags to output
    -V, --verbose                    More verbose output. Use twice for max verbosity
    -v, --version                    Show chef version
    -y, --yes                        Say yes to all prompts for confirmation
    -h, --help                       Show this message
Fouts, Chris | 22 Oct 22:51 2014

Single chef server vs. multiple chef servers - pros and cons?

We have a product comprised of 12-25 nodes with a combination of RHEL and Windows OS’s. Each node has its identity dictated by the set *.msi and *.rpms we install onto it. We can have several deployments of these products throughout our lab, say 5 in the dev lab, 9 in the QA lab, 4 in the Perf lab, etc.  So if at one time we have 20 deployed products, that makes 240-500 nodes we may configure at any given time.

 

We have been exploring two approaches to use Chef to configure our nodes

 

Option 1

We have a single Chef server that contains all our cookbooks that all nodes talk to. I understand the need to segregate cookbooks under development, vs. ones for test or production. I also understand that we may need provision to make this highly-available, etc., so if one server fails we have a standby server.

 

Option 2

Each product is configured with its own chef server, such that the deployment of the product involves first the creation of a chef server, and then the nodes on THIS product can be deployed via this chef server. IOW, if we had 20 products deployed currently, we’ll need 20 chef servers – 1 chef server per product

 

Currently we orchestrate our product deployment via Jenkins

 

Any pros/cons to each approach?

 

Chris

 

Mark Mzyk | 22 Oct 22:08 2014

Chef Server 12 Issue Triage and Progress Update

Hello Chefs,

An action item that came out of last week's community IRC meeting was 
that we should open the Chef Server 12 issue triage process to everyone. 
While we use GitHub issues for most of our issue tracking, that still 
doesn't serve as a great way to communicate where we are in the release 
processes for Chef Server 12.

To provide more transparency, the Chef Server team has created this 
google doc:

https://docs.google.com/a/opscode.com/document/d/1xqVTMt1_8q23VCkQOQGFcMMvSf3ZKj1wjsmx8O7vqRI/edit

Full details are in the document, but it details how we monitor issues, 
the triage lists of issues that are known about and their priority, and 
the target date of releasing Chef Server 12 (We're aiming for early 
November), as well as how to provide feedback on any of the above.

Hopefully this is helpful to you as a way to get more transparency into 
what the Chef Server team is working on and the status of Chef Server 12 
as we work towards the release.

Thanks,

Mark Mzyk

Douglas Garstang | 22 Oct 21:53 2014
Picon

Edelight MongoDB Cookbook

I'm wondering if anyone has actually gotten the Edelight MongoDB cookbook to work, with multiple shards and multiple replicas per shard? I've been playing with it for months now and I'm frustrated beyond belief. We had the folks from mongodb come out and meet us a few months ago and they anecdotally told us they had heard of people who were actually using it.

Since I'm so frustrated, and can't be bothered detailing all the issues from scratch, here's part of an email I sent internally within our organisation today about it...

"Last night, I was able to get the community cookbook to deploy two replica sets, but I had to remove authentication to do so. I had issues getting the sharding to work on the router, so, since the sharding is going to be handled by mongos on the DBC rightscsale instances anyway, and Eugene has said we _might_ not need sharding, I decided to go back to getting replica sets with authentication to work.

When auth is enabled in the cookbook, it automatically tries to authenticate every operation as the admin user when configuring replication, which, if not previously configured will fail. This means the admin user has to be created first. Since the cookbook puts the replset option in the config file automatically on every node, the node knows it's part of a replica set. This means the admin user can't be create on an arbitrary node, but instead has to be created only the primary node. However, there is no primary node until replication has been set up, which it hasn't been yet because there is no admin user.

Someone suggested I work around this by spinning up a data node manually (without the options configured by the cookbook to make it look like it's part of a replica set) and then using the appropriate commands to create the admin user. Not sure how I'd copy the database over to the  instances. In any case, that's hardly automated.

I asked why the admin user can't be added to the router. After all, that's what it's for, right? Apparently db local users are propagated from the router to the data nodes, but system level users (like the admin user) are not. When I attempted to use the cookbook yesterday to create the admin user on the router, mongo failed because the auth option isn't recognized on the router. I don't know why. Maybe because system level users aren't propagated. Even if I get the admin user created on the data nodes, this will stop the admin user (presumably with the same credentials) from being created on the router, and then sharding can't be configured on top of the replica sets.

Once the keyfile option is supplied to the cookbook, then for reasons I don't understand, the cookbook automatically adds the auth option to the config file, which as we know, breaks the router. From what I've been told, they are two different things. One is for internal authentication, and the other is for external user authentication."

All very frustrating stuff. On top of that there's other issues, like the fact that the shard name is passed to the addshard() command instead of the replica set name:
https://github.com/edelight/chef-mongodb/issues/348

The replica set name used in the chef search has a hard coded "rs_" at the front, and this isn't documented anywhere which initially caused a lot of time to be wasted:
https://github.com/edelight/chef-mongodb/issues/347

Is there a better cookbook for mongo?

Doug.





Douglas Garstang | 22 Oct 17:49 2014
Picon

Disable chef-client?

All,

When the chef-client is running as a daemon, is there a way to keep the client running as a daemon, but temporarily put it into NOOP mode?

Doug

Douglas Garstang | 22 Oct 17:10 2014
Picon

chef-metal and chef search.

I've been using chef-metal for a few days, and I've noticed that after the initial instance creation, and chef-client run (initiated by chef-metal), that, in order for the new node to be available in chef search results, I have to manually log in to the instance and run chef-client a _second_ time. Waiting a few minutes isn't a substitute. Why is this required?

Doug

Joe Nuspl | 21 Oct 23:34 2014

chefspec and lazy

I’m trying to write a unit test for a recipe that uses http://docs.getchef.com/resource_common.html#lazy-attribute-evaluation

It fails with:

Failure/Error: runner.converge(described_recipe)
NoMethodError:
 undefined method `lazy' for Chef::Resource::Execute

Any ideas on how to make this work?

Joe
eric | 21 Oct 17:43 2014

Enabling/disabling yum repos

I'm trying to figure out how to enable and disable yum repos with the 3.x yum
and yum-* cookbooks.

I've tried this, but it doesn't appear to be working:

node.default['yum']['rpmforge']['enabled'] = false
node.default['yum']['rpmforge']['managed'] = true
include_recipe 'yum-repoforge'

Can anyone help with this?

Thanks.

Eric


Gmane