Support of different policies per project

[buxtonpaul]

Hi, I initially read the Policies documentation as meaning that you could supply a policies.yml as a file in the root of the codebase being scanned, after some experimenting and then re-reading I realised that policies.yml is globally applied.

It would be useful to be able to either pass the policies.yml along with the project to be scanned. Or support having multiple policy files that the server knows about (probably specified based on outgoing license), and allow specifying which one applies for the project.

This is to allow for the use case where an organisation has multiple types of project which are distributed under different licenses where a single blanket policy will not work.

The first option is likely easiest to manage and would allow small teams to manage the policy checking for their project as and when they come across new licences. The 2nd option would allow the organisation to centrally manage the policy and would allow it to be pre-populated so that new projects get the benefit without having to create the files from the start.

An expanded approach might be to combine those so that there is both an ability to select the outgoing license, or provide a project specific policy file. Pehaps the policy file could be provided in the file hierarchy such that the policy applies to files that are in the same folder or below and are overridden by policy files below them.

Paul

[pombredanne]

This is to allow for the use case where an organisation has multiple types of project which are distributed under different licenses where a single blanket policy will not work.

This makes perfect sense. In particular it would be typical to have a very different policy for a web-based, externally facing app, an embedded device firmware, a mobile app, some desktop app, or some internally used app.

An expanded approach might be to combine those so that there is both an ability to select the outgoing license, or provide a project specific policy file. Pehaps the policy file could be provided in the file hierarchy such that the policy applies to files that are in the same folder or below and are overridden by policy files below them.

I like the idea. We could likely expand it not only to license policies but also to other scan settings, I am thinking about for instance skipping some directories or files entirely or pointing to some pre-concluded definitive license determination that would override scanning... and the ability to optionally ignore this entirely and/or applying another policy not stored in the code.

Let's work out the design here!

[buxtonpaul]

Agree, Could supply .scan_ignorefiles using the .gitignore format Aboutfiles could be treated like a 'concluded license'

So my thinking in python inspired pseudocode

# default license policy read from existing policies 
proj_policy = server_policy 


# select a policy from pre-configured policy per outgoing license
....
if proj_outgoing_license is not None 
    if proj_outgoing_license in license_policies:
        proj_policy = license_policies
    else
       alert user chose license not handled

for each folder in folders:
    file_policy = proj_policy
    local_policy_file = find_nearest_policy(folder)
    if local_policy_file is not None:
        file_policy = local_policy_file
    for each file in folder
        apply_policy(file,file_policy)

If we prescanned the tree to locate policy.yml files then sort it by length then the policy.yml to apply would be the one that has the longest string match with the folder path

[pombredanne]

We are slowly and surely implementing parts to support this:

1. We now support scancode_config.yml file that can be added to the scan inputs (or in the new project settings) to selectively ignore some files
2. In one of the pipelines (the new/WIP deploy_to_devel), we have a new pipe/step that uses ABOUT files for origin and license as some curation file

We should add policy to this next.

[tdruez]

Right, we now have the structure in place to implement per-project policies.