Bulk load candidate profiles from a CSV

[christinerogers]

Feature request from user is Spain: https://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/2022-May/001126.html :

Add ability to load of candidate profiles from a CSV (or other tabular format)

• Note that developers can already consult this Tutorial to create candidates in bulk via API. (see Section 3) https://github.com/xlecours/loris-api-presentation/blob/master/LORIS-API_Part2-Python-script.ipynb

To facilitate this for new projects as a setup step without (much) developer time, a few options:

• Add a front-end feature that (first) generates a template/format recommendation and then (validates then) ingests a completed version of the template.
• Provide a script that will load 2 Raisinbread candidates based on a simple CSV (probably using API not php libraries), which can be adapted.
• Add a few steps written up in the api-docs, and visible on Demo (note: api-docs module not currently visible on Demo), elaborating on the tutorial linked above

[christinerogers]

@zaliqarosli Are there any redcap-related scripts that might be used for this? or could be easily recycled such that this task it more or less covered?

[zaliqarosli]

@christinerogers i don't think so as this issue doesn't necessarily depend on redcap's CSV format, and should just be contained to utilizing the loris api candidate schema and running an API request with a click of a button.

the redcap related script i have is a php script that uses the swagger schema generated PHP client to connect to the API. I can leave some bits of code here in case it's useful (this is off of LORIS 21.0 and API v0.0.3-dev):

require_once __DIR__ . '/../libraries/SwaggerClient-php/vendor/autoload.php';
$clientConfig   = new Swagger\Client\Configuration();

createCandidate(
    $project,
    $pscid,
    $site,
    $dob,
    $sex,
    $clientConfig
);

function createCandidate(
    string                       $project,
    string                       $pscid,
    string                       $site,
    string                       $do_b,
    string                       $sex,
    Swagger\Client\Configuration $clientConfig
) : ?\Swagger\Client\Model\Candidate {
    $apiInstance = new Swagger\Client\Api\CandidatesApi(
        new GuzzleHttp\Client(),
        $clientConfig
    );

    $candidate = new \Swagger\Client\Model\NewCandidateCandidate([
        'project' => $project,
        'pscid'   => $pscid,
        'site'    => $site,
        'do_b'    => $do_b,
        'sex'     => $sex,
    ]);

    $body = new \Swagger\Client\Model\NewCandidate([
        'candidate' => $candidate
    ]);

    try {
        $result = $apiInstance->candidatesPost($body);
    } catch (Exception $e) {
        echo "\nException when calling CandidatesApi->candidatesPost on $pscid: ", $e->getMessage(), PHP_EOL;
    }

    return $result ?? null;
}

[zaliqarosli]

i also find this function useful to convert csvToArray:

/**
 * Convert a comma separated file into an associated array.
 * The first row should contain the array keys.
 *
 * Example:
 *
 * @param string $filename Path to the CSV file
 * @param string $delimiter The separator used in the file
 * @return array
 * @link http://gist.github.com/385876
 * @author Jay Williams <http://myd3.com/>
 * @copyright Copyright (c) 2010, Jay Williams
 * @license http://www.opensource.org/licenses/mit-license.php MIT License
 */
function csvToArray(string $filename='', string $delimiter=','): array
{
    if(!file_exists($filename) || !is_readable($filename)) {
        return array();
    }
    $header = null;
    $data   = array();
    if (($handle = fopen($filename, 'r')) !== FALSE)
    {
        while (($row = fgetcsv($handle, 1000, $delimiter)) !== FALSE)
        {
            if (!$header) {
                $header = $row;
            } else {
                $data[] = array_combine($header, $row);
            }
        }
        fclose($handle);
    }
    return $data;
}

[christinerogers]

FYI @jeffersoncasimir in case useful

[adamdaudrich]

this is being addressed in CBIG currently, with extensive validation based on user config. It also submits for GUID projects and submits external IDs.