cheffish icon indicating copy to clipboard operation
cheffish copied to clipboard

Users are not added to new group until 2nd convergence of recipe

Open poliva83 opened this issue 10 years ago • 9 comments

chef_group will create new group in your organization but the users do not appear to be added to group until 2nd chef-client run.

recipe snippet

url_with_org = {
    :chef_server_url => URI.join(chef_server_url, "/organizations/blackberry").to_s,
    :options => {
      :client_name => "pivotal",
      :signing_key_filename => "/etc/opscode/pivotal.pem" } }

chef_group 'ops' do
      users  ["ops-blackberry", "anbrown"]
      complete true
      chef_server url_with_org
      action :create
end

1st chef-client run

* chef_group[ops] action create
 - create group ops at https://127.0.0.1/organizations/blackberry
 -   update users from [] to ["ops-blackberry", "anbrown"]

knife group show ops -c ~/.chef/knife-chef12.rb  (via knife-acl)

actors:
annotated_groups:
clients:
groupname:        ops
groups:
name:             ops
orgname:          blackberry
users:

2nd chef-client run

* chef_group[ops] action create
 - update group ops at https://127.0.0.1/organizations/blackberry
 -   update users from [] to ["ops-blackberry", "anbrown"]
 -   remove orgname

knife group show ops -c ~/.chef/knife-chef12.rb (via knife-acl)

actors:
  anbrown
  ops-blackberry
annotated_groups:
clients:
groupname:        ops
groups:
name:             ops
orgname:          blackberry
users:
  anbrown
  ops-blackberry

poliva83 avatar Oct 27 '15 01:10 poliva83

@jkeiser @tyler-ball We discussed this issue today I'd rate this issue higher on priority as needing to run chef-client twice when adding new group is less then ideal.

poliva83 avatar Oct 27 '15 01:10 poliva83

Phil had mentioned that this issue that i'm seeing with kitchen could be related. When converging the backend recipe in test-kitchen the following error was observed. Reconverging with no changes resulted in a pass:

Manual run:

  • sudo chef-server-ctl user-list
  • ERROR: Server returned error 500 for https://127.0.0.1/users, retrying 1/5 in 3s
  • Pivotal

Test-Kitchen run:

* chef_user[nlam] action create

           ================================================================================
           Error executing action `create` on resource 'chef_user[nlam]'
           ================================================================================

           Net::HTTPFatalError
           -------------------
           500 "Internal Server Error"

           Resource Declaration:
           ---------------------
           # In /tmp/kitchen/cache/cookbooks/bb_chef_server_cluster/recipes/manage_rbac.rb

            43:   chef_user user do
            44:     display_name "#{values['display_name']}"
            45:     email "#{values['email']}"
            46:     password "#{values['password']}"
            47:     source_key "#{values['source_key']}"
            48:     chef_server url_root
            49:     action :nothing
            50:     not_if { values['source_key'].nil? }
            51:     subscribes :create, 'ruby_block[only modify rbac content on master backend]', :immediately
            52:   end
            53: end

           Compiled Resource:
           ------------------
           # Declared in /tmp/kitchen/cache/cookbooks/bb_chef_server_cluster/recipes/manage_rbac.rb:43:in `block in from_file'

           chef_user("nlam") do
             action [:nothing]
             retries 0
             retry_delay 2
             default_guard_interpreter :default
             chef_server {:chef_server_url=>"https://127.0.0.1:443", :options=>{:client_name=>"pivotal", :signing_key_filename=>"/etc/opscode/pivotal.pem", :api_version=>"0"}}
             declared_type :chef_user
             cookbook_name "bb_chef_server_cluster"
             recipe_name "manage_rbac"
             display_name "Nhid Lam"
             email "[email protected]"
             password "passw0rd"
             source_key "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDU+yPffx9TRlctRhja2hBi3780L+xhBlxxi6apkKN6UdlXgBiFNPVIXhkAgOUWRMpbigNOlF+tQ2se9ItG1uMkJkmArp4TvgCckiXMTro8ZD7cU4yvGwLuziBINdaCFXi9Xvjm8HYc3jXQh2oPAEbmheGd+KIVBMaC/9cWAtsddUpelVaaBHnLmm472sxcvfZFdwRlV5MorJeGvUfC0XUbg+sp1Qufdvpd6TBD4qM2TUDl0loUrjR/5AzUvzNulr+R2+EKFPetj8U/DS9v/5m0lxyRCBZORpGYhuVO5SY2ounSKORwvAtzdNE9/5LWpOEz3YD3/Deb0RbOhMMsmhVV nlam@nlam-chef-provisioning-ws"
             not_if { #code block }
           end


       Running handlers:
       [2015-11-02T20:01:37+00:00] ERROR: Running exception handlers
       Running handlers complete
       [2015-11-02T20:01:37+00:00] ERROR: Exception handlers complete
       Chef Client failed. 70 resources updated in 331.736216745 seconds
       [2015-11-02T20:01:37+00:00] FATAL: Stacktrace dumped to /tmp/kitchen/cache/chef-stacktrace.out
       [2015-11-02T20:01:37+00:00] ERROR: chef_user[nlam] (bb_chef_server_cluster::manage_rbac line 43) had an error: Net::HTTPFatalError: 500 "Internal Server Error"
       [2015-11-02T20:01:38+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
>>>>>> Converge failed on instance <backend-ubuntu-1404>.
>>>>>> Please see .kitchen/logs/backend-ubuntu-1404.log for more details
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: SSH exited (1) for command: [sh -c '

sudo -E /opt/chef/bin/chef-client --local-mode --config /tmp/kitchen/client.rb --log_level auto --force-formatter --no-color --json-attributes /tmp/kitchen/dna.json --chef-zero-port 8889
']
>>>>>> ----------------------

cat /tmp/kitchen/cache/chef-stacktrace.out
Generated at 2015-11-02 20:01:37 +0000
Net::HTTPFatalError: chef_user[nlam] (bb_chef_server_cluster::manage_rbac line 43) had an error: Net::HTTPFatalError: 500 "Internal Server Error"
/opt/chef/embedded/lib/ruby/2.1.0/net/http/response.rb:119:in `error!'
/opt/chef/embedded/apps/chef/lib/chef/http.rb:145:in `request'
/opt/chef/embedded/apps/chef/lib/chef/http.rb:126:in `post'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/cheffish-1.5.0/lib/cheffish/actor_provider_base.rb:30:in `block in create_actor'
/opt/chef/embedded/apps/chef/lib/chef/mixin/why_run.rb:52:in `call'
/opt/chef/embedded/apps/chef/lib/chef/mixin/why_run.rb:52:in `add_action'
/opt/chef/embedded/apps/chef/lib/chef/provider.rb:175:in `converge_by'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/cheffish-1.5.0/lib/cheffish/actor_provider_base.rb:29:in `create_actor'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/cheffish-1.5.0/lib/chef/provider/chef_user.rb:15:in `block in <class:ChefUser>'
/opt/chef/embedded/apps/chef/lib/chef/provider/lwrp_base.rb:160:in `instance_eval'
/opt/chef/embedded/apps/chef/lib/chef/provider/lwrp_base.rb:160:in `block in action'
/opt/chef/embedded/apps/chef/lib/chef/provider.rb:140:in `run_action'
/opt/chef/embedded/apps/chef/lib/chef/resource.rb:584:in `run_action'
/opt/chef/embedded/apps/chef/lib/chef/runner.rb:49:in `run_action'
/opt/chef/embedded/apps/chef/lib/chef/runner.rb:57:in `block in run_action'
/opt/chef/embedded/apps/chef/lib/chef/runner.rb:55:in `each'
/opt/chef/embedded/apps/chef/lib/chef/runner.rb:55:in `run_action'
/opt/chef/embedded/apps/chef/lib/chef/runner.rb:81:in `block (2 levels) in converge'
/opt/chef/embedded/apps/chef/lib/chef/runner.rb:81:in `each'
/opt/chef/embedded/apps/chef/lib/chef/runner.rb:81:in `block in converge'
/opt/chef/embedded/apps/chef/lib/chef/resource_collection/resource_list.rb:83:in `block in execute_each_resource'
/opt/chef/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
/opt/chef/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
/opt/chef/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
/opt/chef/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
/opt/chef/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
/opt/chef/embedded/apps/chef/lib/chef/resource_collection/resource_list.rb:81:in `execute_each_resource'
/opt/chef/embedded/apps/chef/lib/chef/runner.rb:80:in `converge'
/opt/chef/embedded/apps/chef/lib/chef/client.rb:654:in `block in converge'
/opt/chef/embedded/apps/chef/lib/chef/client.rb:649:in `catch'
/opt/chef/embedded/apps/chef/lib/chef/client.rb:649:in `converge'
/opt/chef/embedded/apps/chef/lib/chef/client.rb:688:in `converge_and_save'
/opt/chef/embedded/apps/chef/lib/chef/client.rb:269:in `run'
/opt/chef/embedded/apps/chef/lib/chef/application.rb:270:in `block in fork_chef_client'
/opt/chef/embedded/apps/chef/lib/chef/application.rb:258:in `fork'
/opt/chef/embedded/apps/chef/lib/chef/application.rb:258:in `fork_chef_client'
/opt/chef/embedded/apps/chef/lib/chef/application.rb:224:in `block in run_chef_client'
/opt/chef/embedded/apps/chef/lib/chef/local_mode.rb:39:in `with_server_connectivity'
/opt/chef/embedded/apps/chef/lib/chef/application.rb:212:in `run_chef_client'
/opt/chef/embedded/apps/chef/lib/chef/application/client.rb:402:in `block in interval_run_chef_client'
/opt/chef/embedded/apps/chef/lib/chef/application/client.rb:392:in `loop'
/opt/chef/embedded/apps/chef/lib/chef/application/client.rb:392:in `interval_run_chef_client'
/opt/chef/embedded/apps/chef/lib/chef/application/client.rb:382:in `run_application'
/opt/chef/embedded/apps/chef/lib/chef/application.rb:60:in `run'
/opt/chef/embedded/apps/chef/bin/chef-client:26:in `<top (required)>'
/opt/chef/bin/chef-client:55:in `load'

nlam79 avatar Nov 02 '15 20:11 nlam79

@poliva83 what Chef server are you using? I'm seeing things that make me wonder if perhaps the issue is server-side.

jkeiser avatar Nov 04 '15 15:11 jkeiser

@jkeiser We are seeing this on 12.2.0-1 from stable channel.

poliva83 avatar Nov 04 '15 15:11 poliva83

@jkeiser Not sure this matters but worth mentioning its using a HA topology setup (BYOHA model).

poliva83 avatar Nov 04 '15 15:11 poliva83

I see the above bug (chef/chef-server#605) on Hosted; not 100% sure if it's the same bug, because it would imply that both create and update were busted. Will investigate more.

jkeiser avatar Nov 04 '15 16:11 jkeiser

@jkeiser Did your investigation bear fruit? Any news on this issue?

poliva83 avatar Nov 09 '15 15:11 poliva83

@poliva83 We're following up on this in https://github.com/chef/chef-server/issues/605

pburkholder avatar Nov 17 '15 21:11 pburkholder

I can work around it in cheffish by having it create and then immediately update the group.

jkeiser avatar Dec 10 '15 18:12 jkeiser