cloudtools
Using self.context.hook_data['some_key'] does not work when you are trying to diff a stack
I'm using the release-1.0 branch and a common workflow for me is to make edits to our template.yaml and the blueprints and then run 'stacker diff' to see the changes that would be emitted.
If you have a pre_build hook(specifically the aws_lambda.upload_lambda_functions hook) then it is impossible to diff a stack that reaches into the context.hook_data to retrieve the results(in this case the troposphere Code object being generated in the hook) because the hook_data object is not populated when you run the diff command.
I had not considered this case when writing the hook. They are not usually run in diff mode, and would need special support for it, to generate relevant data without actually applying any changes. Do you have any suggestions on how it should work in that case?
The way I managed to work around it for now was simply providing a mock troposphere Code object like the following in my blueprint. It still creates entries in the diff but at least that is manageable when scanning the output.
if 'lambda' in self.context.hook_data:
code = self.context.hook_data['lambda']['BuildsAccessLogParser']
else:
code = Code("JunkCode", S3Bucket="JUNK", S3Key="JUNK")
Yeah, I'm not sure there's a good way to genericize this - if the hook_data was somehow typed, we could potentially do something, but that's a bigger change.
The best I believe can be done is to let the hook signal that it knows how to run in diff-only mode, and run it. It can't be done by default or existing hooks may run twice.
Em dom, 29 de jan de 2017 17:12, Michael Barrett [email protected] escreveu:
Yeah, I'm not sure there's a good way to genericize this - if the hook_data was somehow typed, we could potentially do something, but that's a bigger change.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/remind101/stacker/issues/296#issuecomment-275937946, or mute the thread https://github.com/notifications/unsubscribe-auth/AAtnToKNoU-jmYbMj_PHZeYQov83Hmwrks5rXOSEgaJpZM4LmTdU .
@mchristen So after talking with @danielkza and others on the slack, I think the plan going forward will be to change diff so that it no longer does our own version of diffing, but instead uses CFN changesets, which would be more "complete". We already use changesets when using --interactive in the stacker build action, so it should be easy-ish to implement.
This doesn't fix any issues we might have in stacker build --dump, namely that the hooks are actually ran whenever you run the dump (which isn't really what most people would expect is my guess). Going to try and talk this out some more - I have, in the past, advocated for doing away with the --dump argument, but maybe it's used by too many people.
Opinions?
Sorry for the delay in getting back, I was distracted with other work and haven't had much time to look at the Lambda stuff again. I've been loosely following the discussion about pulling in blueprints via git and I think that the lambda stuff could benefit from something similar. I also hacked in some requirements.txt parsing as well to pull in the runtime dependencies and package them up with the zip file since lambda has no concept of setup.py or requirements.txt.
In general I don't need to execute the lambda hook unless code has actually changed.
Ideally, I think the lambda hook could look at the Python dependency tree of the lambda function code and not do anything unless either the lambda function code itself has changed or a dependency needs to be updated. If the hook has knowledge of the current version of the lambda function deployed and the version of the current code for the function (via. git hash or something) then it could do something different when running in 'diff' mode, like print out that the lambda hook needs to be updated and what it is going to update when executing the hook in 'build' mode.
Regarding the --dump command
I personally don't use --dump at all and prefer to manage our stack changes through the diff command.
I think my ideal workflow would be something like:
- Make changes to blueprint code and/or configuration files
- Inspect all the changes that CloudFormation is going to make to the templates
- It would be nice if the resource lifecycle could be incorporated into this somehow. For example, some resource types force a replacement of the entire resource instead of updating the specific parameter. So sometimes what looks like a simple configuration change might in turn tear down entire resources and stacks to rebuild them completely. The only way to know is to go look up the specific properties in the CFN Documentation and try and piece together what CloudFormation will do.
- Issue the build command that updates the stacks
Hah! @mchristen - re: the diff command, see my response to your other issue. I plan to change diff to use actual CFN changesets, so that should make it easier to have full confidence in it, since the changesets can tell you if there's a replacement required :)