Tim Smith | 9 Oct 20:01 2015

Cookbook Engineering Team Update

Today marks one month since Nathen Harvey wrote about our plans for the Chef Supermarket and our newly expanded Cookbook Engineering Team (https://www.chef.io/blog/2015/09/08/the-state-of-the-public-supermarket/).  I thought at this point we should probably give a bit of an update to let everyone know some of the work we’ve accomplished over the last month.

Getting the house in order:

We maintain 130+ community cookbooks at Chef and for several years many of those had not been updated. Our first priority was general cleanup of these cookbooks so we can evaluate fixes and features to work on next:
- All cookbooks have been moved from opscode-cookbooks to chef-cookbooks
- We created a new single repository for Contributing and Testing documents (https://github.com/chef-cookbooks/community_cookbook_documentation).  All repos now have TESTING.MD and CONTRIBUTING.MD files that point here with up to date instructions.  No more references to JIRA!
- We added MAINTAINERS.toml and MAINTAINERS.md files with a list of cookbook maintainers you can contact.  As we loop in community members to help maintain these cookbook they’ll be listed in these files.
- All references to Opscode have been replaced with Chef Software
- Cookbook version badges have been added to all cookbooks so you can easily see the latest published release
- README.md files have been expanded with requirements sections that list support versions of chef, supported platforms, and required cookbooks
- The new issues_url and source_url metadata have been added for Supermarket
- Standardized .gitignore and chefignore files have been added to all repos
- Berksfiles have been updated to the 3.X format
- Apache 2.0 license files have been added to the root of every repo


We at Chef are firm believers in testing infrastructure, but many of our cookbooks were created well before the technology existed to do so.  We’ve begun the process of rolling up to date testing out to all of our community cookbooks. Obvious extensive testing will take time, but this gives us a solid baseline to work from.
- We’ve turned Travis on for all repos, and added Travis badges to all Readme.md files.  Travis had actually been turned off for sometime, but the badges remained in many cookbooks giving a false sense of security. Travis is also using pre-release ChefDK debs now instead of testing via gems, which allows us to test on the latest Chef before we ship.  Expect a lot of red here for the time being as we still have a significant amount of cookbook cleanup to perform.
- We updated all Test Kitchen configs to the new format and added additional platforms so we’re testing common use cases.  We also added Test Kitchen configs to any cookbook that was lacking them.
- All existing Chefspecs have been updated to the 4.X format.  We also added basic convergence specs to any cookbook lacking Chefspecs.
- We released Foodcritic 5.0 and began testing all of our cookbooks with that in Travis.  Supermarket is also using 5.0 to test uploaded cookbooks
- There’s a new standard .rubocop.yml file with a sane set of rules and good portion of the warnings have been resolved.
- For local testing we added a Rakefile with several tasks to speed up the testing process
- All of the development Gems have been updated and broken out into groups in the Gemfiles
- We’ve begun rolling out Test Kitchen running via Docker in Travis so we can run full integration tests on every commit  See https://github.com/chef-cookbooks/activemq for an example of how that works.

Issues and PRs:

With 130 repositories it’s easy to become overwhelmed by the number of issues and PRs we receive. We’re working to stay on top of the existing PRs and to better identify where work is necessary.
- Our Github org is now polled daily and we save off the issue count and PR count per repo so we can compare how we’re doing over time, and see where work is needed.
- We’ve merged a large number of PRs. How many? Well on 9/4 when we began polling we had 380 open PRs and today we have 292 PRs even though a large number of new PRs have been opened in the last month.  With our testing in place we hope to begin rapidly merging PRs where the CLA has been signed.
- We’ve released 46 Chef managed cookbooks in the last month with a large number of bug fixes, improved platform support, and new features.

We think we’ve come fairly far in the last month, but there’s certainly more to come.  Expect our open PR / issue count to drop, and many new cookbooks released to Supermarket in the coming months.  Additionally expect to see higher quality cookbooks with our expanded testing and the Cookbook Quality Metrics (https://github.com/chef-cookbooks/cookbook-quality-metrics).  If you have any questions or advice of where to focus our efforts now feel free to reach out to me.

-Tim Smith
<at> tas50
tsmith <at> chef.io

Cassiano Leal | 9 Oct 16:42 2015

New Custom Resources

Hi list,

I’m having a look into the newly released Chef Client 12.5 and I really like the new streamlined custom resources.

I couldn’t find information on how to deal with multiple providers in this new model though. What’s the
thought around that? Create different custom resources for what used to be different providers?

Cassiano Leal
Anthony Kong | 9 Oct 02:06 2015

Getting a Deprecated warning message

Hi all,

When I vagrant up a box today, I got this warning message. It seems to have come from yum_globalconfig

==> Server-003: Deprecated features used!

==> Server-003:   Cannot specify both default and name_property together on property path of resource yum_globalconfig. Only one (name_property) will be obeyed. In Chef 13, this will become an error. at 1 location:

==> Server-003:     - /var/chef/cache/cookbooks/yum/resources/globalconfig.rb:76:in `class_from_file'

==> Server-003:   version nil currently does not overwrite the value of version. This will change in Chef 13, and the value will be set to nil instead. Please change your code to explicitly accept nil using "property :version, [MyType, nil]", or stop setting this value to nil. at 3 locations:

==> Server-003:     - /var/chef/cache/cookbooks/python/recipes/pip.rb:52:in `block in from_file'

==> Server-003:     - /var/chef/cache/cookbooks/python/recipes/virtualenv.rb:25:in `block in from_file'

==> Server-003:     - /var/chef/cache/cookbooks/python/providers/pip.rb:98:in `load_current_resource'

Is there anything I need to do?


Roberto Leibman | 8 Oct 19:59 2015

REST API question: How to fetch file?

I need to fetch an attribute file from the chef service...

Using the REST API, I call

GET /organizations/NAME/cookbooks/NAME/VERSION

And among other things I get:


I tried to fetch the file using the url given (either directly, or using the same signature schema as other REST api calls), but it doesn't work, I get a 403.

What is the meaning of the URL if it's not to fetch the file? How do I fetch the file? Is there a different REST api call that will give me the file directly (since I know everything else about it)


Roberto Leibman
jeffty | 8 Oct 18:59 2015

How to invoke another recipe with notifies

Hi All,


I have two recipe check_out_code.rb and deploy.rb and want to sync code from git, and them deploy the code to server if the sync successfully (which means that some code in this git project has been changed).


In check_out_code.rb:


git node['git_project_directory’] do

    repository  git_url

    action :sync

    notifies :run, "how to call the deploy recipe here?", :immediately



And in deploy.rb the updated file in the git project directory will be deployed to server once a synchronization is performed successfully.


Is it possible? If not maybe we need to define a global variable like:


synced = false


And change it to true if git sync succeeded. Then


if synced then

  include_recipe “mytest::deploy”



But this seems not work either in the git resource.


Thanks & Regards


Doug Garstang | 8 Oct 18:17 2015

Include recipe if file exists.


Trying to do a conditional include_recipe based on a file existing. Doesn't work due to compile/converge stage stuff.

A workaround was suggested here by the folks that should know:

However, it doesn't work, just as someone else responded to the ticket with. They did not get a reply.

So, just to be clear, as suggested, this does not work:

unless ::File.exists?("/etc/foo/bar")
  include_recipe "foo::bar"

Koert Kuipers | 7 Oct 18:12 2015


lately i been seeing a lot of these:

       Starting Chef Client, version 12.4.3
       Compiling Cookbooks...
       [2015-10-07T16:03:24+00:00] WARN: Using an LWRP provider by its name (Ark) directly is no longer supported in Chef 12 and will be removed.  Use Chef::ProviderResolver.new(node, resource, action) instead.

i got about 50 cookbooks that are part of this run. how do i know where the offending call to Ark is? a little more info like a cookbook name or file name would be helpful.

Nate Meyer | 7 Oct 02:05 2015

Using resources in a library?

Hi everyone,

I'm trying to add a gem dependency to a library. I found a couple conversations talking about using resources in libraries in my travels:

In both, the most reasonable advice is to create an instance of the resource's class and call #run_action(:whatever) to get it to execute immediately. From my testing, this doesn't work, with the problem specifically being this code snippet and error: https://gist.github.com/notnmeyer/04bc1c0be0e41504cd62

My first thought was to pass it run_context... But there's no run_context at this point for me to pass to the resource's constructor.

To demonstrate in further detail, I put together a quick cookbook with a Test Kitchen suite: https://github.com/notnmeyer/gem-dep

Any thoughts? What's the proper way to immediately execute a resource during compile time?

Douglas Garstang | 6 Oct 22:35 2015

Berkshelf sources from chef server

In our Berksfile's, we've been specifying the location of dependent cookbooks as paths to local files. It would seem that a better approach would be to retrieve them from our own chef server. I can't get this to work however.

My Berksfile now has the following:

source "https://api.berkshelf.com"
source "https://chef-004.dev.slicetest.com"

After setting the path to the certificate for our chef server with:
export SSL_CERT_FILE=/Users/doug/.chefvm/configurations/dev/trusted_certs/chef-004_dev_slicetest_com.crt

When I run 'berks install', this is what I get:

Fetching cookbook index from https://api.berkshelf.com...
Fetching cookbook index from https://chef-004.dev.slicetest.com...
Error retrieving universe from source: https://chef-004.dev.slicetest.com
  * [Berkshelf::APIClient::ServiceNotFound] service not found at: https://chef-004.dev.slicetest.com
/opt/chefdk/embedded/lib/ruby/2.1.0/net/http.rb:920:in `connect': SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed (Faraday::SSLError)

What am I doing wrong?

Fabien Delpierre | 6 Oct 22:18 2015

Puppet+Hiera vs. Chef, or handling the occasional special snowflake server

Hello folks,
I'm a user of both Puppet and Chef, I'm more familiar with and prefer the latter overall, but am required to use the former at work. Recently I was toying with the idea of moving to Chef, although it's probably not a realistic option, but it raised a few interesting questions.
While Puppet and Chef do roughly the same thing, they do it in slightly different ways, and Puppet has something in particular that Chef doesn't have, which is Hiera.

If you're not familiar with Hiera, it's a simple way to apply settings to any number of your servers (from a single node to an entire infrastructure of thousands), you define the hierarchy of settings yourself -- in a sense, it's kind of like being able to customize Chef's attribute precedence and having a single environment attached to all your nodes. The link above explains it a lot better than I can.

Anyway, I was wondering how one might deal with losing Hiera when moving from Puppet to Chef. It seems that you could achieve similar results with a mix of cookbooks, roles and environments, like everything else, but at the cost of ending up with a lot more of them than you'd normally have.

I guess a different way I might ask this is:
Imagine I have a group of servers, all configured the same way (same run list, same environment), and I need to change something on just one of them. A common scenario in my environment is having to attach YourKit to a JVM process.
With Puppet/Hiera, I'd just add a two-line YAML file with the same name as that one server's FQDN, and when I run Puppet, that YAML file would be read at compile time and the config change applied without affecting the other nodes.
With Chef, I'd have to modify the node's run list or take its environment file, make a copy of it, modify the copy and assign that copied environment to the node instead of the normal env file, lest I affect the other servers in the same group.

Hiera is also capable of aggregating data from multiple sources into a single hash or array to be applied to a node during a Puppet agent run.

Those are things that I never missed for as long as I used Chef, but now that I know they're out there, I'm wondering how I would handle things without Hiera.

So in short, I was wondering if anybody has thoughts on the matter. Obviously it helps if you have some experience with both tools. I'm far from being a Chef expert so it's possible I'm missing something. Any input is appreciated. Thanks!
kallen | 6 Oct 00:37 2015

oc-id on chef server 11.1.x? for private Supermarket

Hi. I'm exploring running a private Supermarket. tl;dr -- can I run oc-id on our
chef 11.1.x server and point Supermarket at it? Or do I need to spend my time
upgrading to chef server 12?


This doc suggests chef server 12 for oc-id, but doesn't say 11.x is out of the
question: https://www.chef.io/blog/2015/04/21/setting-up-your-private-supermarket-server/

This deprecated repo mentions chef server >= chef server 11.2:

I tried following the instructions in the doc, and added the config bit to

chef-server]# tail -5 chef-server.rb
oc_id['applications'] = {
  'supermarket' => {
    'redirect_uri' => 'https://supermarket.redacted.internal/auth/chef_oauth2/callback'

But, fail:

chef-server]# /opt/chef-server/bin/chef-server-ctl reconfigure

Recipe Compile Error in /opt/chef-server/embedded/cookbooks/chef-server/recipes/default.rb

undefined method `[]=' for nil:NilClass

Cookbook Trace:
  /opt/chef-server/embedded/cookbooks/chef-server/recipes/default.rb:34:in `from_file'

Relevant File Content:

 27:  end.run_action(:create)
 29:  if File.exists?("/etc/chef-server/chef-server.json")
 30:    Chef::Log.warn("Please move to /etc/chef-server/chef-server.rb for configuration -
/etc/chef-server/chef-server.json is deprecated.")
 31:  else
 32:    ChefServer[:node] = node
 33:    if File.exists?("/etc/chef-server/chef-server.rb")
 34>>     ChefServer.from_file("/etc/chef-server/chef-server.rb")
 35:    end
 36:    node.consume_attributes(ChefServer.generate_config(node['fqdn']))
 37:  end
 39:  if File.exists?("/var/opt/chef-server/bootstrapped")
 40:    node.set['chef_server']['bootstrap']['enable'] = false
 41:  end
 43:  # Create the Chef User

chef-server]# cat /opt/chef-server/embedded/cookbooks/cache/chef-stacktrace.out
Generated at 2015-10-05 22:35:04 +0000
NoMethodError: undefined method `[]=' for nil:NilClass
/opt/chef-server/embedded/lib/ruby/gems/1.9.1/gems/mixlib-config-2.1.0/lib/mixlib/config.rb:47:in `from_file'
`block in
/opt/chef-server/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/policy_builder/expand_node_object.rb:73:in `setup_run_context'
/opt/chef-server/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/client.rb:265:in `setup_run_context'
/opt/chef-server/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/client.rb:429:in `do_run'
`block in run'
/opt/chef-server/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/client.rb:207:in `fork'
`block in
/opt/chef-server/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/application.rb:67:in `run'
/opt/chef-server/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/bin/chef-solo:25:in `<top (required)>'
/opt/chef-server/embedded/bin/chef-solo:23:in `load'
/opt/chef-server/embedded/bin/chef-solo:23:in `<main>'