Deploy central authentication system : Openstack Keystone

[vincentnam]

The central authentication system is the main security service for data security. The tool choosed is Openstack Keystone for several reason :

• API are available in Python (and other language could be used if really needed)
• Can be invoked standalone and can be integrate in other services pipeline
• Natively integrated with Openstack Swift
• LDAP / NIS authentication system can be natively integrated in Openstack Keystone authentication database

Objectives are to deploy the service and create the authentication mechanisms for the differents services. LDAP and NIS are often used authentication system. As theses services are often already deployed, the objective is to integrate it in and not overide and create a brand new system (and redo what has already been done before -> waste of time / ressources / probably datas).

• [ ] Initial deployment
  • [ ] Design general authentication system and access
  • [ ] Add LDAP support
  • [ ] Add NIS support
  • [ ] Integrate Openstack Keystone with Openstack Swift
• [ ] Create authentications mechanisms for Mongodb
• [ ] Create authentications mechanisms for Airflow
• [ ] Create authentications mechanisms for database in consumption area
• [ ] Create authentications mechanisms for neo4J
• [ ] Create authentications mechanisms for web GUI service
• [ ] Create authentications mechanisms for Kafka
• [ ] Create authentications mechanisms for data consumption services

[vincentnam]

Deployment Openstack Keystone :

Openstack Keystone is the central authentication system choosed in the architecture to unify the authentication among the whole architecture. As it is often already deployed in information system, LDAP and NIS has to be integrated as former authentication database. It should be able to extend the database with new user easily.

• [ ] Deploy Openstack Keystone
  • [ ] Deploy Openstack Keystone
  • [ ] Deploy with Openstack Swift
    • [ ] Redefine Openstack Swift configuration (with distributed configuration over several servers (at least 1 per service + duplication / replication services + files parallelized execution)
    • [ ] Integrate Openstack Swift as default authentication service (v3.0)
  • [ ] Integrate LDAP database in Openstack Keystone
  • [ ] Integrate NIS database in Openstack Keystone
• [ ] Test the deployment (for automation) and authentication with test user

[vincentnam]

Authentication for Neo4J :

• [ ] Deploy Neo4J in metadata management area
• [ ] Create a custom authentication plugin for neo4j (https://neo4j.com/docs/java-reference/4.2/extending-neo4j/security-plugins/)

[vincentnam]

Authentication for MongoDB

There is no native solution to deploy an external authentication solution in MongoDB. It can be done through LDAP, SCRAM or x.509 Certificate Authentication (https://docs.mongodb.com/manual/core/authentication-mechanisms/).

• [x] Deploy MongoDB in metadata management area
• [ ] State of art of solution that could be possible for MongoDB
  • [ ] Through Openstack Keystone as LDAP solution ? (https://www.percona.com/doc/percona-server-for-mongodb/ext-auth.html) : with SASL daemon or SASL lib custom implementation ?
  • [ ] Through SCRAM mechanism ?
  • [ ] ... ?
• [ ] Design a solution
• [ ] Deploy this solution

[vincentnam]

Authentication for Airflow

• [ ] Deploy Airflow 1.14
• [ ] Deploy Airflow 2.0
• [ ] "State of art" for Airflow tools and look if a solution is directly through Airflow service
• [ ] If not, implement a "authentication" task
  • [ ] See if metadata as to be added in messages cross-services

[vincentnam]

Authentication for Kafka

As stream are mainly defined through services and process, the process has to be linked to an user. May be the subject of a design task.

• [ ] Deploy Kafka
• [ ] Integrate the Openstack Keystone authentication in Kafka ?
  • [ ] Design a solution
  • [ ] Deploy a solution

[vincentnam]

Authentication for data consumption services

May be useless, indeed, authentication could have been done in other services (as in the access zone). To discuss.

[vincentnam]

Authentication for web GUI

• [ ] Integration in the web GUI of the Openstack Keystone authentication mechanism (JS library should be available)
  • [ ] Log-in page