Add ability to specify top level node attributes in Chef Search

[jeremiahsnapp]

Customers want to be able to specify top level attributes in Chef Search to avoid the problems caused by extracting nested fields to the top level.

When nested fields are present in a JSON structure, the chef-client [actually opscode-expander right?] will extract those nested fields to the top-level, flattening them into compound fields that support wildcard search patterns.

Some very important attributes are at the top level such as chef_environment. Customers have cookbooks or other tools that rely on Chef Search to return all nodes in a particular chef_environment but all it takes to cause unexpected results (and a production outage) is having another chef_environment attribute with a different value nested anywhere at all in the node data.

I discussed this briefly with one of the developers and thought it could be possible (maybe not even hard) to enable this functionality for users but that a feature request for it would likely get triaged as a low priority. I would like to see this kind of improvement in reliability be made a much higher priority. Especially if it isn't even that hard to do.

I've seen time and time again that two separate Chef Server features can give completely correct results and yet cause customers serious trouble when those results aren't at all what they expected. The two features are Cookbook Depsolver and Chef Search. Currently all we can say is the server is "right" even though you're not getting what you expect. And then we have to train the customer to learn how to do their best to avoid these problems in the future.

I love the direction Policyfiles is taking to make it easier for customers to explicitly declare what set of recipes will be run on a node ahead of time rather than being bitten by the complexity of cookbook depsolving during a deploy. I think adding the ability to specify top level attributes in a Chef Search would be another great win in the battle to make things more sane for customers.

[jrwesolo]

Here is an example to reproduce the issue with nested attributes named the same as root level attributes: https://github.com/jrwesolo/chef_search_woes

[rlaveycal]

I'd like to add a voice for this. We've just had an issue where we deleted the wrong VM on AWS because the knife ec2 server delete -N command uses search to resolve the instance_id from the given node name. We had additional name attributes with values corresponding to valid nodes deep within our attribute data. Consequently the search returned multiple answers and the ec2 plugin chose the first one. The user didn't spot this.

This is a ticking time bomb and really needs fixing so that everything that uses search can start to be migrated to safe searches. I appreciate that the default will have to remain as-is for some considerable time.

[kamaradclimber]

Hello, we've been hitting that bug frequently and have to avoid any attribute ending with role, roles, name and other frequent words of chef vocabulary.

Is this ticket being in the roadmap?

Would you have pointers if we wanted to implement a fix?

[mattwhite]

This is really quite dangerous:

$ knife node flip server123 decom
Looking for an fqdn of server123 or name of server123
Setting environment to decom
23 items found
...

Any use of name, role, or roles as an attribute name can cause unexpected behavior.

[gsantandrea]

We hit in the same problem. Are there any updates on this ?

[PrajaktaPurohit]

Thank you for reporting the issue. We are in the transition to move from solr to elasticsearch. We will bring this up for prioritization with product once that transition is complete. Thank you for your patience.