Azure-Player
Add public function to delete objects from ADF instance
It looks like when an ADF object is excluded from deployment (via the Excludes property of the options) it is still considered to be "in source" and DeleteNotInSource does not apply to it. For instance: if a given ADF object is already in the target ADF, and if a new deployment excludes that object from deployment and DeleteNotInSource flag is set to true, the excluded object will not be deleted from the target. I would expect it to be deleted after the deployment.
Yes, this is exactly how the feature should work like.
'Excludes' property means you want to skip deployment of particular (set of) object(s), but definitely won't delete any of excluded objects. As latter property says "Delete Not in Source" - but the object is still in the source (repo) and should NOT be removed from ADF service.
I consider exposing the method to delete some objects explicitly if needed. Let me know if that would help in your scenario.
Hey Kamil,
Thanks for your note.
-
I see what you mean, Excluded does not mean that is no longer "in source" (they are considered to be in source), as a result "Delete not in source" (true) will skip the deletion of them if they were present at the target. This is not a big deal.
-
Yes, a way of explicit delete of objects in a given folder would definitely be helpful. Again, just nice to have. This could be extended by the object type (pipeline, dataset, etc.), recursive flag to delete subfolders, perhaps even a pattern to match folders to delete.
We normally use ARM template deployment, and it works pretty well, except when we need to automatically rebuild templates. There is no API for that, and Microsoft is very slow in adding it. I know they are planning to revamp ADF, but not sure when. We keep enquiring them for improvements related to remote ADF calls, better workflow capabilities, ability to fail a pipeline etc, all slated to come but no one knows when.
Anyway, nice package you have created. Well done. Probably the most comprehensive that I have seen.
Thanks Maciej
On Sun, Sep 6, 2020 at 4:13 AM Kamil Nowinski [email protected] wrote:
Yes, this is exactly how the feature should work like. 'Excludes' property means you want to skip deployment of particular (set of) object(s), but definitely won't delete any of excluded objects. As latter property says "Delete Not in Source" - but the object is still in the source (repo) and should NOT be removed from ADF service. I consider exposing the method to delete some objects explicitly if needed. Let me know if that would help in your scenario.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/SQLPlayer/azure.datafactory.tools/issues/30#issuecomment-687747274, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMI26ROGXX652GNRAKROGHLSENOE5ANCNFSM4Q4DGOVQ .
@NowinskiK @gidziu indeed, this feature will be really cool. At the moment I'm migrating a project to a different Team Data Factory, and I thought about something related to this. It would be great to have a "delete deployment" feature (all included adf objects in the deployment config files). Kind of a "reverse deployment" thing to be able clean ADF.
