[WIP] Simple REST API with OUATH authorization support

[mdzik]

Proof-of-concept for rest api. (#384)

For this to work you need to makemigrations for outh2 plugin and install it ( pip install django-oauth-toolkit djangorestframework==3.13.1 )

Then run locally, and add 'application': https://django-oauth-toolkit.readthedocs.io/en/latest/getting_started.html#authorization-code

Then copy ClientID and Secret to simple CLI app in Go, from here: https://github.com/mdzik/coldfront-cli

To just test json output go to

http://localhost:9000/users-api/slurm/RESOURCENAME/

[
    {
        "id": 497,
        "all_public_attributes_as_list": [
            {
                "id": 14, //attribute type ID
                "name": "slurm_account_name",
                "value": "XXXX",
                "usage": ""
            },
            {
                "id": 3,
                "name": "Core Usage (Hours)",
                "value": "1000000",
                "usage": "86048.52"
            }
        ],
        "project_id": 2695
    }
]

[mdzik]

I will clean this bit more and add some more detailed instruction asap :)

[mdzik]

CLI Output, login is required first time, than auth token is stored locally

go run ./coldfront-cli.go

Visit the following URL for the auth dialog (ctrl_click is often enough):
 http://localhost:9000/users-api/o/authorize?response_type=codeXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Paste the token obtained from web-page:
XXXXXXXXXXXXX


+----------------------------------+---------------+-----------+-----------+-----------+---------+----------------+
| PROJECT                          | ALLOCATION ID | CPUH      |           | GPUH      |         | ACCOUNT        |
+----------------------------------+---------------+-----------+-----------+-----------+---------+----------------+
|                                  |               | Available | Used      | Available | Used    |                |
+----------------------------------+---------------+-----------+-----------+-----------+---------+----------------+
| https://xxxxxxxxxxxxxxxxx/p/3501 | 866           | 2200000   | 2045.7667 | 2045.7667 | 2200000 | XXXX        |
| https://xxxxxxxxxxxxxxxxx/p/3284 | 1113          | 2000000   | 89718.83  | 89718.83  | 2000000 | XXX      |
| https://xxxxxxxxxxxxxxxxx/p/3562 | 1190          | 2000000   | 2529.0933 | 2529.0933 | 2000000 | XXXX       |
| https://xxxxxxxxxxxxxxxxx/p/2352 | 491           | 1000000   | 6274.5    | 6274.5    | 1000000 | XXXX  |
| https://xxxxxxxxxxxxxxxxx/p/2695 | 495           | 5000000   | 630785.7  | 630785.7  | 5000000 | XXXX |
+----------------------------------+---------------+-----------+-----------+-----------+---------+----------------+

[aebruno]

@mdzik Thanks so much for this. Looks like a great start, we'll give this a quick test internally and get back to you with any feedback. Thanks again!

[knikolla]

@aebruno @dsajdak @mdzik are there any plans to continue this work? We're about to start similar work to expose a simple JSON view of the currently active projects, allocations and users.

[mdzik]

We moved "list of active allocations of the user" to production today, and use it for MOTD. I will updage this PR so you cold use it as starting point?

czw., 2 mar 2023, 15:57 użytkownik Kristi Nikolla @.***> napisał:

@aebruno https://github.com/aebruno @dsajdak https://github.com/dsajdak @mdzik https://github.com/mdzik are there any plans to continue this work? We're about to start similar work to expose a simple JSON view of the currently active projects, allocations and users.

— Reply to this email directly, view it on GitHub https://github.com/ubccr/coldfront/pull/425#issuecomment-1452008022, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA3A5RYTVDVZVCA4DZIHVQDW2CYNNANCNFSM52V5JHCA . You are receiving this because you were mentioned.Message ID: @.***>

[aebruno]

We moved "list of active allocations of the user" to production today, and use it for MOTD. I will updage this PR so you cold use it as starting point?

@mdzik That would be great!

@knikolla We haven't started work on this other than what @mdzik has done in this PR. This is a highly requested feature that's been on our radar for a while. So would definitely be open to collaborating on this.

[knikolla]

@mdzik That would be great, thank you!

@aebruno Sound great! I'll wait until @mdzik pushes an updated version and use that as a starting point.

[mdzik]

@knikolla we use this:

https://github.com/mdzik/coldfront-cli/blob/main/motd/coldfront-motd.go

to generate MOTD at ssh login

[knikolla]

I did some work in implementing this for coldfront-plugin-cloud in a way that's not specific to SLURM (as this PR filters on resources of type Cluster and the path terminology is slurm. In my case, I need to list out all active Resource Allocations and their attributes. https://github.com/nerc-project/coldfront-plugin-cloud/pull/93/files

One of the issues I ran into is that there's no mechanism for plugins that are out of tree to include urlpatterns, as I had to fork ColdFront and change /coldfront/config/urls.py.

What would be the best way in moving forward in a way that we can satisfy both of our use cases and get some form of generic API merged upstream?

[lcrownover]

Hey all, it's been a while since I've looked at ColdFront and this PR seems to be working towards the goal that has kept me from implementing CF. While my programming is decent, my django experience is pretty slim, so I'm wondering what the status is on this, and if there's anything specific you might need extra hands on?

[dsajdak]

@lcrownover this is currently our top priority on this project; however, we are only able to dedicate a few hours a week on it. Would you be able to provide some use cases you're interested in using this API for so that we can ensure we accommodate those?

[lcrownover]

@dsajdak Sure!

Some background on us: at University of Oregon, we use a combination of Active Directory and Ansible inventory data to maintain these associations. It's slow, manual, and error-prone to have a human add users to multiple relevant groups in AD, then go over to Ansible and run one or more playbooks. I've had to write several thousand lines of Powershell (ugh) to put guardrails around this process, and tooling to detect inconsistencies in group structure and fix or report on them.

Based on what I've seen and folks we've talked to at SC, Coldfront is hands-down the best Configuration Management Database (CMDB) designed specifically for HPC centers. Having a central system to track associations between users, projects, storage allocations, etc, is so incredibly important for consistent access.

A few immediate issues:

• The loaddata CLI tool requires both access to the host system as well as administrative access to Coldfront.
• Lack of API means that we're always at the mercy of multiple cron job timing variability. All we can say is "This runs every 10min, this other one runs every 5min, so changes could be made in 1-15min". That, or using a service account to SSH trigger jobs, but that requires SSH access from wherever the request is coming from.
• In a cron-based automated system, how would you know if your ingest data was malformed? I guess...mail...

Having an API would allow users of the product to be able to build custom tools to extend the functionality.

One of several use cases today:

We have several classes that run on our cluster, each with a hundred or so students each. That's around 300 students per term that we have to onboard. Using the loaddata method, they'd have to contact us, we'd have to verify their data is correct, both from a formatting standpoint, as well as security, (they could just slip in an association to another project), then we'd administratively add those users. If there was a REST API with proper role-based access control, we could provide documentation and examples on how to add users to a project and they can take the time to import the students without our help, and it would be secure, as they'd assumedly get a 401 if they tried to access something they shouldn't.

In Coldfront's current state, my plan is to create an API "companion" app that will do several things:

• Properly handle a limited amount of administrative auth for security (azure oauth probably)
• (POST) Have a set of endpoints that will accept association data and serialize it into the format Coldfront expects, then call the loaddata tool to import it.
• (GET) Have a set of endpoints that act as "exporters", so that tools like Active Directory can simply run a scheduled task every minute or so to ping the API and check for any new changes. If there's a change, it can get an entire user/project association dump and fill out AD groups with the correct users.

Having this functionality means I can now have immediate feedback on changes. Pushing data means it gets done and reports success, all in one call. Getting data means that it's not inconsistent because the ingestor cron job runs 2min from now and my call now will have different results from my call in 2min.

If Coldfront had a proper API, this tool would not need to exist 😊

I'm happy to elaborate on any points.

I'm also potentially interested in contributing if there's some legwork that needs done, I just don't want to step on any toes if you're still deciding on the how/why/when's for implementation.

edit: I originally read your post as "help justify allocation of dev time by providing examples", so I framed my post in that way. If you're just looking for examples to satisfy customer needs, here's a simple list 😊:

• Role based access control (jwt) so users can use the API to make modifications to their own resources, but not others
• Endpoints for management (get, post, put, delete) of all the usual suspects: users, projects, allocations, etc.
• A CLI tool like above that uses the API would be pretty awesome for administrative use.

[knikolla]

@dsajdak Sure!

Some background on us: at University of Oregon, we use a combination of Active Directory and Ansible inventory data to maintain these associations. It's slow, manual, and error-prone to have a human add users to multiple relevant groups in AD, then go over to Ansible and run one or more playbooks. I've had to write several thousand lines of Powershell (ugh) to put guardrails around this process, and tooling to detect inconsistencies in group structure and fix or report on them.

Based on what I've seen and folks we've talked to at SC, Coldfront is hands-down the best Configuration Management Database (CMDB) designed specifically for HPC centers. Having a central system to track associations between users, projects, storage allocations, etc, is so incredibly important for consistent access.

A few immediate issues:

• The loaddata CLI tool requires both access to the host system as well as administrative access to Coldfront.
• Lack of API means that we're always at the mercy of multiple cron job timing variability. All we can say is "This runs every 10min, this other one runs every 5min, so changes could be made in 1-15min". That, or using a service account to SSH trigger jobs, but that requires SSH access from wherever the request is coming from.
• In a cron-based automated system, how would you know if your ingest data was malformed? I guess...mail...

Having an API would allow users of the product to be able to build custom tools to extend the functionality.

One of several use cases today:

We have several classes that run on our cluster, each with a hundred or so students each. That's around 300 students per term that we have to onboard. Using the loaddata method, they'd have to contact us, we'd have to verify their data is correct, both from a formatting standpoint, as well as security, (they could just slip in an association to another project), then we'd administratively add those users. If there was a REST API with proper role-based access control, we could provide documentation and examples on how to add users to a project and they can take the time to import the students without our help, and it would be secure, as they'd assumedly get a 401 if they tried to access something they shouldn't.

In Coldfront's current state, my plan is to create an API "companion" app that will do several things:

• Properly handle a limited amount of administrative auth for security (azure oauth probably)
• (POST) Have a set of endpoints that will accept association data and serialize it into the format Coldfront expects, then call the loaddata tool to import it.
• (GET) Have a set of endpoints that act as "exporters", so that tools like Active Directory can simply run a scheduled task every minute or so to ping the API and check for any new changes. If there's a change, it can get an entire user/project association dump and fill out AD groups with the correct users.

Having this functionality means I can now have immediate feedback on changes. Pushing data means it gets done and reports success, all in one call. Getting data means that it's not inconsistent because the ingestor cron job runs 2min from now and my call now will have different results from my call in 2min.

If Coldfront had a proper API, this tool would not need to exist 😊

I'm happy to elaborate on any points.

I'm also potentially interested in contributing if there's some legwork that needs done, I just don't want to step on any toes if you're still deciding on the how/why/when's for implementation.

edit: I originally read your post as "help justify allocation of dev time by providing examples", so I framed my post in that way. If you're just looking for examples to satisfy customer needs, here's a simple list 😊:

• Role based access control (jwt) so users can use the API to make modifications to their own resources, but not others
• Endpoints for management (get, post, put, delete) of all the usual suspects: users, projects, allocations, etc.
• A CLI tool like above that uses the API would be pretty awesome for administrative use.

We've implemented a subset of that as a separate plugin to ColdFront, though I don't know how useful it may be for you given that it's somewhat prescriptive about the setup for our immediate needs (which were for an admin level API to: get allocation info to find PI and associated openstack/openshift project, mass registering students to classes.)

• Admin-only
• Can query (read-only) allocation information.
• Can get/create users.
• Can add/remove user from allocation.

[lcrownover]

@knikolla Thanks for that. I took a look at the openapi.yaml per the README's instruction and I don't see the necessary controls to work with user objects. Is there other updated documentation you might have? Looking at the scripts in /tools I do see some examples for usage.

I think I'd probably prefer to wait/contribute for a proper implementation though.