Anthony Kong | 28 Jan 07:13 2015
Picon

Chef-zero and invalid sensu data-bag

Hi,

I am getting data bag error when I tried to install sensu via vargant/chef. Here is some output


==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
==> default: Configuring and enabling network interfaces...
==> default: Mounting shared folders...
    default: /vagrant => /Users/antkong/dev/zeetings/aws/vagrant-setup/01-sensu-server
    default: /tmp/vagrant-cache => /Users/antkong/.vagrant.d/cache/ubuntu/trusty64
    default: /tmp/vagrant-chef/ee2283bdbc846a6f570f99e61af9abd4/cookbooks => /Users/antkong/.berkshelf/vagrant-berkshelf/shelves/berkshelf20150128-2618-1t2vwq6-default
==> default: Installing Chef 12.0.3 Omnibus package...
==> default: Downloading Chef 12.0.3 for ubuntu...
==> default:   to file /tmp/install.sh.2131/metadata.txt
==> default: trying wget...
==> default: md5 76910abba3f7902d1b4d176f7a568c3a
==> default: sha256 3e172853aa0bcc27415a14b1013811dd44e5f523f4ba34714b66191ba365e796
==> default: downloaded metadata file looks valid...
==> default: /tmp/vagrant-cache/vagrant_omnibus/chef_12.0.3-1_amd64.deb already exists, verifiying checksum...
==> default: Comparing checksum with sha256sum...
==> default: checksum compare succeeded, using existing file!
==> default: Installing Chef 12.0.3
==> default: installing with dpkg...
==> default: (Reading database ... 60959 files and directories currently installed.)
==> default: Preparing to unpack .../chef_12.0.3-1_amd64.deb ...
==> default:  * Stopping chef-client chef-client
==> default:    ...done.
==> default: Unpacking chef (12.0.3-1) over (11.8.2-2) ...
==> default: dpkg: warning: unable to delete old directory '/var/log/chef': Directory not empty
==> default: dpkg: warning: unable to delete old directory '/etc/chef': Directory not empty
==> default: Setting up chef (12.0.3-1) ...
==> default: Thank you for installing Chef!


And further down the log, there is a data bag exception


=> default: [2015-01-28T05:29:38+00:00] DEBUG: I am not loading apt, because I have already seen it.
==> default: [2015-01-28T05:29:38+00:00] DEBUG: filtered backtrace of compile error: /tmp/vagrant-chef/ee2283bdbc846a6f570f99e61af9abd4/cookbooks/sensu/libraries/sensu_helpers.rb:39:in `data_bag_item',/tmp/vagrant-chef/ee2283bdbc846a6f570f99e61af9abd4/cookbooks/sensu/recipes/default.rb:66:in `from_file'
==> default: [2015-01-28T05:29:38+00:00] DEBUG: filtered backtrace of compile error: /tmp/vagrant-chef/ee2283bdbc846a6f570f99e61af9abd4/cookbooks/sensu/libraries/sensu_helpers.rb:39:in `data_bag_item',/tmp/vagrant-chef/ee2283bdbc846a6f570f99e61af9abd4/cookbooks/sensu/recipes/default.rb:66:in `from_file'
==> default: [2015-01-28T05:29:38+00:00] DEBUG: backtrace entry for compile error: '/tmp/vagrant-chef/ee2283bdbc846a6f570f99e61af9abd4/cookbooks/sensu/libraries/sensu_helpers.rb:39:in `data_bag_item''
==> default: [2015-01-28T05:29:38+00:00] DEBUG: Line number of compile error: '39'
==> default:
==> default: ================================================================================
==> default: Recipe Compile Error in /tmp/vagrant-chef/ee2283bdbc846a6f570f99e61af9abd4/cookbooks/sensu/recipes/default.rb
==> default: ================================================================================
==> default:
==> default:
==> default: Chef::Exceptions::InvalidDataBagPath
==> default: ------------------------------------
==> default: Data bag path '/tmp/vagrant-chef/ee2283bdbc846a6f570f99e61af9abd4/data_bags' is invalid
==> default:
==> default: Cookbook Trace:
==> default: ---------------
==> default:   /tmp/vagrant-chef/ee2283bdbc846a6f570f99e61af9abd4/cookbooks/sensu/libraries/sensu_helpers.rb:39:in `data_bag_item'
==> default:   /tmp/vagrant-chef/ee2283bdbc846a6f570f99e61af9abd4/cookbooks/sensu/recipes/default.rb:66:in `from_file'
==> default:
==> default: Relevant File Content:
==> default: ----------------------
==> default: /tmp/vagrant-chef/ee2283bdbc846a6f570f99e61af9abd4/cookbooks/sensu/libraries/sensu_helpers.rb:
==> default:
==> default:  32:            "/opt/sensu/embedded/bin/gem"
==> default:  33:          else
==> default:  34:            "gem"
==> default:  35:          end
==> default:  36:        end
==> default:  37:
==> default:  38:        def data_bag_item(item, missing_ok=false)
==> default:  39>>         raw_hash = Chef::DataBagItem.load("sensu", item)
==> default:  40:          encrypted = raw_hash.detect do |key, value|
==> default:  41:            if value.is_a?(Hash)
==> default:  42:              value.has_key?("encrypted_data")
==> default:
==> default:  43:            end
==> default:  44:          end
==> default:  45:          if encrypted
==> default:  46:            secret = Chef::EncryptedDataBagItem.load_secret
==> default:  47:            Chef::EncryptedDataBagItem.new(raw_hash, secret)
==> default:  48:          else
==> default:
==> default:
==> default: [2015-01-28T05:29:38+00:00] DEBUG: Re-raising exception: Chef::Exceptions::InvalidDataBagPath - Data bag path '/tmp/vagrant-chef/ee2283bdbc846a6f570f99e61af9abd4/data_bags' is invalid
==> default: /opt/chef/embedded/apps/chef/lib/chef/data_bag.rb:116:in `block in load'
==> default:   /opt/chef/embedded/apps/chef/lib/chef/data_bag.rb:114:in `each'
==> default:   /opt/chef/embedded/apps/chef/lib/chef/data_bag.rb:114:in `load'
==> default:   /opt/chef/embedded/apps/chef/lib/chef/data_bag_item.rb:146:in `load'
==> default:   /tmp/vagrant-chef/ee2283bdbc846a6f570f99e61af9abd4/cookbooks/sensu/libraries/sensu_helpers.rb:39:in `data_bag_item'
==> default:   /tmp/vagrant-chef/ee2283bdbc846a6f570f99e61af9abd4/cookbooks/sensu/recipes/default.rb:66:in `from_file'
==> default:   /opt/chef/embedded/apps/chef/lib/chef/mixin/from_file.rb:30:in `instance_eval'
==> default:   /opt/chef/embedded/apps/chef/lib/chef/mixin/from_file.rb:30:in `from_file'
==> default:   /opt/chef/embedded/apps/chef/lib/chef/cookbook_version.rb:245:in `load_recipe'
==> default:   /opt/chef/embedded/apps/chef/lib/chef/run_context.rb:169:in `load_recipe'
==> default:   /opt/chef/embedded/apps/chef/lib/chef/run_context/cookbook_compiler.rb:140:in `block in compile_recipes'
==> default:   /opt/chef/embedded/apps/chef/lib/chef/run_context/cookbook_compiler.rb:138:in `each'
==> default:   /opt/chef/embedded/apps/chef/lib/chef/run_context/cookbook_compiler.rb:138:in `compile_recipes'
==> default:   /opt/chef/embedded/apps/chef/lib/chef/run_context/cookbook_compiler.rb:75:in `compile'
==> default:   /opt/chef/embedded/apps/chef/lib/chef/run_context.rb:92:in `load'
==> default:   /opt/chef/embedded/apps/chef/lib/chef/policy_builder/expand_node_object.rb:73:in `setup_run_context'
==> default:   /opt/chef/embedded/apps/chef/lib/chef/client.rb:235:in `setup_run_context'
==> default:   /opt/chef/embedded/apps/chef/lib/chef/client.rb:397:in `run'
==> default:   /opt/chef/embedded/apps/chef/lib/chef/application.rb:261:in `block in fork_chef_client'
==> default:   /opt/chef/embedded/apps/chef/lib/chef/application.rb:249:in `fork'
==> default:   /opt/chef/embedded/apps/chef/lib/chef/application.rb:249:in `fork_chef_client'
==> default:   /opt/chef/embedded/apps/chef/lib/chef/application.rb:215:in `block in run_chef_client'
==> default:   /opt/chef/embedded/apps/chef/lib/chef/local_mode.rb:38:in `with_server_connectivity'
==> default:   /opt/chef/embedded/apps/chef/lib/chef/application.rb:201:in `run_chef_client'
==> default:   /opt/chef/embedded/apps/chef/lib/chef/application/solo.rb:245:in `block in interval_run_chef_client'
==> default:   /opt/chef/embedded/apps/chef/lib/chef/application/solo.rb:234:in `loop'
==> default:   /opt/chef/embedded/apps/chef/lib/chef/application/solo.rb:234:in `interval_run_chef_client'
==> default:   /opt/chef/embedded/apps/chef/lib/chef/application/solo.rb:224:in `run_application'
==> default:   /opt/chef/embedded/apps/chef/lib/chef/application.rb:58:in `run'
==> default:   /opt/chef/embedded/apps/chef/bin/chef-solo:25:in `<top (required)>'
==> default:   /usr/bin/chef-solo:40:in `load'
==> default:   /usr/bin/chef-solo:40:in `<main>'

I checked the code at line 66 in /tmp/vagrant-chef/ee2283bdbc846a6f570f99e61af9abd4/cookbooks/sensu/recipes/default.rb

ssl = Sensu::Helpers.data_bag_item("ssl")

In my host PC data_bags path, the file ssl.json is present. 

$ tree data_bags/
data_bags/
└── sensu
    └── ssl.json

I did not encrypt the ssl.json

I 'vargant ssh' to the guest OS and checked the data bag path. Here is the result:

vagrant <at> vagrant-ubuntu-trusty-64:~$ sudo ls -l /tmp/vagrant-chef/ee2283bdbc846a6f570f99e61af9abd4/data_bags
ls: cannot access /tmp/vagrant-chef/ee2283bdbc846a6f570f99e61af9abd4/data_bags: No such file or directory


It seems like chef-zero failed to create the data bag path in the guest OS? How can I fix it?



Cheers, Tony







Christine Draper | 28 Jan 04:09 2015

Using machine_file resource for upload

Hi,

I am trying to upload a file using chef provisioning machine_file. I am getting the error below. Am I doing something wrong or is there a bug?


The following works OK so I'm not desperate, just curious/keen to learn.

machine 'appserver' do
  files(
        '/twi/pkg/INT/ypo.tar.gz' => '/twi/pkg/INT/ypo.tar.gz',
       )
end

--------------------------------------------------------------------------------

 * machine_file[/twi/pkg/INT/ypo.tar.gz] action upload[2015-01-27T21:05:12-06:00] INFO: Processing machine_file[/twi/pkg/INT/ypo.tar.gz] action upload ( <at> recipe_files::/home/christine/test/provision/deploy.rb line 3)
[2015-01-27T21:05:14-06:00] INFO: Executing sudo ls -d /twi/pkg/INT/ypo.tar.gz on ubuntu <at> 54.200.143.38
[2015-01-27T21:05:17-06:00] INFO: Completed ls -d /twi/pkg/INT/ypo.tar.gz on ubuntu <at> 54.200.143.38: exit status 0
[2015-01-27T21:05:17-06:00] INFO: Executing sudo md5sum -b /twi/pkg/INT/ypo.tar.gz on ubuntu <at> 54.200.143.38
[2015-01-27T21:05:18-06:00] INFO: Completed md5sum -b /twi/pkg/INT/ypo.tar.gz on ubuntu <at> 54.200.143.38: exit status 0
       ================================================================================
    Error executing action `upload` on resource 'machine_file[/twi/pkg/INT/ypo.tar.gz]'
    ================================================================================
   
    TypeError
    ---------
    no implicit conversion of nil into String
   
    Resource Declaration:
    ---------------------
    # In /home/christine/test/provision/deploy.rb
   
      3: machine_file '/twi/pkg/INT/ypo.tar.gz' do
      4:  machine 'appserver'
      5:   path '/twi/pkg/INT/ypo.tar.gz'
      6:   action :upload
      7: end
      8:
----------------------------------------------------------------

Regards,
Christine
Francis Vidal | 27 Jan 22:52 2015
Picon

Chef DK fails in "chef verify"

Hi,

I installed Chef DK on a VM running Ubuntu 12.04 and got the following error message when I ran "chef verify":

root <at> vagrant-ubuntu-precise-64:~# chef verify
Running verification for component 'berkshelf'
Running verification for component 'test-kitchen'
Running verification for component 'chef-client'
Running verification for component 'chef-dk'
Running verification for component 'chefspec'
Running verification for component 'rubocop'
Running verification for component 'fauxhai'
Running verification for component 'knife-spork'
Running verification for component 'kitchen-vagrant'
Running verification for component 'package installation'
..........

/opt/chefdk/embedded/lib/ruby/site_ruby/2.1.0/rubygems/specification.rb:2104:in `raise_if_conflicts': Unable to activate knife-container-0.2.4, because json-1.8.2 conflicts with json (<= 1.8.1, >= 1.4.4) (Gem::ConflictError)

Hope you can help me out. Thanks!
Fouts, Chris | 27 Jan 22:34 2015

Chef Push jobs primer/tutorial?

https://docs.chef.io/install_push_jobs.html

 

I’m trying to use Chef push for orchestrating my node configurations, and the best information I can find is in the above link. However, it’s still lack-luster. Is there a better tutorial somewhere? (I’ve looked)

 

For example, what does item #3 in above link mean?

 

·  Add the following default attributes on all nodes that are managed by Chef push jobs:

"push_jobs": {

  "package_url": "<package_url>",

  "package_checksum": "<checksum>"

}

 

What is <package_url> and <checksum>, or where do I get those?

 

Chris

Koert Kuipers | 27 Jan 20:47 2015

define same service multiple times

what happens if i define the same service multiple times across recipes?

for example:
i define it once just so i can have it restart if a template changes (but in that recipe i do not enable or start it)

i define it again in another recipe to enable and start it.

do these multiple definitions get merged somehow? what if they conflict?

thanks! koert
Douglas Garstang | 26 Jan 23:29 2015
Picon

Creating authorized_keys for LDAP users.

I'm having trouble setting up users authorized keys. A cookbook that runs earlier in the runlist sets up LDAP. However, due to reasons I don't understand, none of that user information is available during the chef run. I previously posted about this once before. As a result, I can't simply create files and directories and use 'owner' and 'group.

I came up with the below idea. I'm iterating over the ssh keys in a data bag and then for each user running a command as this user. That makes PAM do all the home directory setup for me. I create the ~/.ssh directory in a similar fashion, as the user. All works ok. However, I'm having an issue with adding the array of ssh_keys pulled from the data bag to the users authorized keys file.

include_recipe "slice-ldap"
bag = data_bag("ssh-keys")
for item in bag do
  user = data_bag_item('ssh-keys', item)
  user_name = user['id']
  ssh_keys = user['ssh_keys']
  execute "create_home_#{user_name}" do
    command "su - #{user_name} -c \"ls\""
    creates "/home/#{user_name}"
    notifies :run, "execute[create_ssh_dir_#{user_name}]", :immediately
  end
  execute "create_ssh_dir_#{user_name}" do
    command "su - #{user_name} -c \"mkdir /home/#{user_name}/.ssh\""
    notifies :run, "execute[install_public_rsa_#{user_name}]", :immediately
    creates "/home/#{user_name}/.ssh"
  end
  ssh_keys.each_with_index do |k, index|
    log "k = #{k}"
    execute "install_public_rsa_#{user_name}" do
      command "su - #{user_name} -c \"echo '#{k}' >> /home/#{user_name}/.ssh/authorized_keys\""
      action :nothing
    end
  end
end


However, I'm having an issue with adding the array of ssh_keys pulled from the data bag to the users authorized keys file. The loop at the end does this, but chef also gives me this warning:

==> default: [2015-01-26T22:23:47+00:00] WARN: Previous execute[install_public_rsa_doug]: /tmp/vagrant-chef-3/chef-solo-1/cookbooks/slice-ssh-keys/recipes/default.rb:38:in `block (2 levels) in from_file'
==> default: [2015-01-26T22:23:47+00:00] WARN: Current  execute[install_public_rsa_doug]: /tmp/vagrant-chef-3/chef-solo-1/cookbooks/slice-ssh-keys/recipes/default.rb:38:in `block (2 levels) in from_file'



Apart from the warning, only the last ssh keys is being added to the authorized_keys file. Even though I'm using echo and >>, the last one is not there. The log statement shows each key, so I know the loop is iterating over both. What gives?

Doug

Dwayne Forehand | 26 Jan 23:03 2015

Sharing state across an environment

I've got internal and external webserver roles.  In my internal run list I have a recipe for a web service which sets a node attribute with that services end point.  I've got other web app recipes later in the internal run list that use that web service end point attribute in configuring the web apps.  I also have a recipe in the external web server role that wants to discover the end point for that web service, but I'm not sure how to access the other nodes attributes or even if I should.  

Is there a way that I can set the web service end point attribute at an environment level and have my recipes access it from there?  Is there a better way to handle this need?

Thanks,
Dwayne
Douglas Garstang | 26 Jan 20:38 2015
Picon

Determine inside VPC

I'm trying to use ohai to determine if my instance is inside a VPC. There's a key nested down there under node['ec2']['network_interfaces_macs'] when this is the case, called vpc_id, but my ruby is horrible and I'm having trouble fetching it. For reasons I don't understand ohai reports this is a dictionary, but when I attempt to iterate over it inside a chef recipe, it's an array.

Is there an easy/better way to determine in chef if your ec2 instance is inside a VPC?

Doug


M G | 26 Jan 17:37 2015
Picon

Multi-node chef run orchestration

I have node1 and node2. Each node has chef-client running on it.

Is it possible that after chef-client run on node1 is complete, it can it start/trigger a chef run on node2? 

Does chef have the ability to perform orchestration of runs on the basis of some triggers?

Another thing, I am trying to locate documentation for CHef's Rest calls. Couldnt find any. I am using Chef 11 Open Source.

Any help/pointers on the above questions would be appreciated.

regards

mg

Anthony Kong | 25 Jan 03:46 2015
Picon

How to use 'knife data bag create' without chef server?


Hi, 

I intend to install a sensu receipt (https://github.com/sensu/sensu-chef) via chef-zero and vagrant. 

It requires the creation of a data bag. 

In the README.md of the sensu-chef, it says I can run either

cd examples/ssl ./ssl_certs.sh generate knife data bag create sensu
or

knife data bag from file sensu ssl.json

However either way it requires a  chef server URL. 

I do not have access to any chef server. How can get around this restriction to create a data bag?

Thanks, Tony


Julian C. Dunn | 23 Jan 22:55 2015

Re: Querying a Chef server through it's API

Couple things:

* You don't have a colon after 'http'
* If you're running Chef Server 12 or connecting to Hosted Enterprise
Chef, the chef_server_url is going to be something like
https://your-chef-server/organizations/yourorg -- there is no endpoint
'/nodes' at the root of the system.

- Julian

On Fri, Jan 23, 2015 at 12:48 PM, ANGELA EBIRIM <aebirim <at> icloud.com> wrote:
> Hello everyone,
>
> I'm new to Chef so be gentle!
>
> I'm trying to query a Chef server through it's API and have developed the
> following code:-
>
> require 'chef/config'
> require 'chef/log'
> require 'chef/rest'
> chef_server_url = "http//chef.prod.practor.service"
> client_name = "ace67"
> signing_key_filename=File.dirname(__FILE__) + "/.chef/ace67.pem"
>
> rest = Chef::REST.new(chef_server_url, client_name, signing_key_filename)
> nodes = rest.get_rest("/nodes")
>
> nodes.keys.each do |node_name|
>     puts "Node name: #{node_name}"
> end
>
> It's a simple Ruby script to query the Chef server. through it's API, for a
> list of nodes. However when I run the script, I get an error of TypeError:
> can't dup NilClass. It's occurring on the nodes  =  rest.get_rest("/nodes")
> line.
>
> I'm running Chef version 11.16.2
>
> I'd appreciate any assistance with this.
>
> Many thanks
>
> Angela Ebirim
>
> Sent from iCloud

--

-- 
[ Julian C. Dunn <jdunn <at> aquezada.com>          * Sorry, I'm    ]
[ WWW: http://www.aquezada.com/staff/julian    * only Web 1.0  ]
[ gopher://sdf.org/1/users/keymaker/           * compliant!    ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9       ]


Gmane