help wanted - quickstart / examples that work

[heepspray]

Hi guys

I am Looking for a way to automatically import scan files

I have followed the quickstart on https://github.com/aaronweaver/defectdojo_api/tree/master/examples and I am using the dojo_ci_cd.py with scan files that are imported perfectly when using the GUI. It does not work. I have tried every advice on here, including using defectdojo_api==1.1.3

Am I using the wrong tool or method ? Is there a prerequisite that I don't know of ? Am I just stupid, or is the API broken ? Can someone help me with a basic setup to make this work ?

I have posted in a number of issues on here, but there seems to be no resolution or updates in a good while, in particular:

• TypeError: string indices must be integers from https://github.com/aaronweaver/defectdojo_api/issues/13

• AttributeError: 'NoneType' object has no attribute 'success' from https://github.com/aaronweaver/defectdojo_api/issues/14

Any feedback appreciated

[MaroonBeret]

@aaronweaver having the same problem here, mind looking into this? Thanks

[MaroonBeret]

@heepspray been able to fix the problem? You could use curl to post a report as well. Btw, Retire.js seems not to be supported by Dojo anymore, that might be your problem.

[heepspray]

Hey @MaroonBeret

No, I am still stuck. The amount of errors I ran into while trying made me believe that either a) the wrapper is broken OR b) Everything I do is wrong and I need help

All the issues are stale, so either the users have quietly resolved their problems, or they are using a different method for uploading

So.. are you saying I should just drop the wrapper, and shoot the upload straight at the API ? Does that work for you ?

By the way, should I use API v1 or v2 ?

And what do you mean Retire.js may be a problem ? I only know that as a burp extension..

Thanks for the reply :) sorry about all the questions

[aaronweaver]

I don't have the bandwidth at the moment to look into this. I will say I use this daily with no issues so sounds like some misconfiguration. I would suggest proxying the request to see what the error is. Choose a sample burp or zap file and try loading it.

On Mon, Oct 29, 2018, 5:23 PM heepspray [email protected] wrote:

Hey @MaroonBeret https://github.com/MaroonBeret

No, I am still stuck. The amount of errors I ran into while trying made me believe that either a) the wrapper is broken or b) Everything I do is wrong and I need help

All the issues are stale, so either the users have quietly resolved their problems, or they are using a different method for uploading

So.. are you saying I should just drop the wrapper, and shoot the upload straight at the API ? Does that work for you ?

By the way, should I use API v1 or v2 ?

And what do you mean Retire.js may be a problem ? I only know that as a burp extension..

Thanks for the reply :) sorry about all the questions

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/aaronweaver/defectdojo_api/issues/20#issuecomment-434085439, or mute the thread https://github.com/notifications/unsubscribe-auth/AANepnLjkFoR_yTYmp4dppwzG2D6ZYVDks5up3HDgaJpZM4XykBr .

[MaroonBeret]

@heepspray okay gotcha, I've found a way to report my scan results and I'm gonna try to help you. No problem at all btw, I ran into a lot of issues as well :P.

It will sound weird but fill in all user information for the user you use to post the scan. This includes first name, last name, email address, etc. Dojo needs this info to correctly link the uploaded scan to the uploading user.

If this doesn't work you could try posting with curl. Let me know if thats the case.

[aaronweaver]

BTW - the user info bug has been fixed in dev and hoping to push the release shortly.

On Tue, Oct 30, 2018 at 5:52 AM MaroonBeret [email protected] wrote:

@heepspray https://github.com/heepspray okay gotcha, I've found a way to report my scan results and I'm gonna try to help you. No problem at all btw, I ran into a lot of issues as well :P.

It will sound weird but fill in all user information for the user you use to post the scan. This includes first name, last name, email address, etc. Dojo needs this info to correctly link the uploaded scan to the uploading user.

If this doesn't work you could try posting with curl. Let me know if thats the case.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/aaronweaver/defectdojo_api/issues/20#issuecomment-434237148, or mute the thread https://github.com/notifications/unsubscribe-auth/AANeptLX-svjb7v5KG7RRa3ZER1j50etks5uqCFogaJpZM4XykBr .

-- Aaron Weaver Philadelphia OWASP Chapter Lead OWASP AppSec Pipeline Lead https://www.owasp.org/index.php/OWASP_AppSec_Pipeline

[MaroonBeret]

@aaronweaver Nice, it's appreciated!

[heepspray]

Thanks for trying to help guys, I did another concentrated attempt and will try to explain :

My main struggle is dojo_ci_cd.py - the file does not run and there is no request sent to the server unless I make a series of horrible hacks in it. I will not be posting the web request here before I get to a point where I can confidently run the example script

The example on https://github.com/aaronweaver/defectdojo_api/tree/master/examples reads : dojo_ci_cd.py --product=1 --file "/tests/scans/Bodgeit-burp.xml" --scanner="Burp Scan" --high=0 --host=http://localhost:8000 --api_key=<api_key> --user=admin

The command I am using is : python dojo_ci_cd.py --product=10 --file "/xx/xx/xx/burp_ad/a9.xml" --scanner="Burp Scan" --high=0 --host=http://localhost:8000 --api_key=xxxxxxxxxxxx --user=xxxxxxxxx

(--product is an actual product id on my server. And the scan file is a working file that can be imported through the UI)

When I run the file it results in the following failure :

Traceback (most recent call last):
  File "dojo_ci_cd.py", line 242, in <module>
    class Main:
  File "dojo_ci_cd.py", line 283, in Main
    test_ids = processFiles(dd, engagement_id, file, scanner=scanner)
  File "dojo_ci_cd.py", line 145, in processFiles
    if test_id.success == False:

On line 86 in dojo_ci_cd.py I change scannerName from None to "Burp Scan". I have no clue why scannerName is set to None in the first place. When I run the file again I get the following error :

Traceback (most recent call last):
  File "dojo_ci_cd.py", line 242, in <module>
    class Main:
  File "dojo_ci_cd.py", line 283, in Main
    test_ids = processFiles(dd, engagement_id, file, scanner=scanner)
  File "dojo_ci_cd.py", line 146, in processFiles
    print "Upload failed: Detailed error message: " + test_id.data
TypeError: cannot concatenate 'str' and 'NoneType' objects

I can keep hacking the dojo_ci_cd.py file, but it is obvious that what I am doing is wrong. And it will probably worsen the more I try, so I will stop right here :)

[MaroonBeret]

@heepspray You might wanna use curl to post your request instead of the Python script, like this:

curl --fail
--header "Authorization: ApiKey $DEFECT_DOJO_API_KEY"
--form "file=@<REPORT_FILE>"
--form "scan_type=<SCAN_TYPE>"
--form "engagement=/api/v1/engagements/<ENGAGEMENT_ID>/" \ --form "active=true"
--form "verified=False"
--form "scan_date=$DATE"
$DEFECT_DOJO_DOMAIN/api/v1/importscan/"

[heepspray]

@MaroonBeret YES! - thank you :) that works just fine

I see no problem in using this going forward, not sure what to do with the original Issue though.. if you believe that all my problems have been caused by my misunderstandings and general incompetence then it can probably be closed :/

Thanks again, see you on the next hurdle ;)

[MaroonBeret]

@heepspray Good to know that it worked! As I said, this doesn't mean you are incompetent, I had the same issue and someone else helped me out as well. Doesn't mean this issue doesn't persist though, we just use another approach now. Good luck!

[ylorgat]

@MaroonBeret I'm having issues using the curl method could you paste an example on how to use it correctly.

My scan files are in Downloads/sample-scan-files and I'm using zap

Thank you very much

[heepspray]

@ylorgat this is a working example :

curl --fail --header "Authorization: ApiKey xxxxxxxx:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" --form "file=@/path/to/report/report.xml" --form "scan_type=ZAP Scan" --form "engagement=/api/v1/engagements/123/" --form "active=true" --form "verified=False" --form "scan_date=2018-11-14" example.com/api/v1/importscan/ --verbose

[valentijnscholten]

I think some of the problems may be caused by the pip version lagging behind what is on github, see #22

[wheelq]

Thanks for trying to help guys, I did another concentrated attempt and will try to explain :

My main struggle is dojo_ci_cd.py - the file does not run and there is no request sent to the server unless I make a series of horrible hacks in it. I will not be posting the web request here before I get to a point where I can confidently run the example script

The example on https://github.com/aaronweaver/defectdojo_api/tree/master/examples reads : dojo_ci_cd.py --product=1 --file "/tests/scans/Bodgeit-burp.xml" --scanner="Burp Scan" --high=0 --host=http://localhost:8000 --api_key=<api_key> --user=admin

The command I am using is : python dojo_ci_cd.py --product=10 --file "/xx/xx/xx/burp_ad/a9.xml" --scanner="Burp Scan" --high=0 --host=http://localhost:8000 --api_key=xxxxxxxxxxxx --user=xxxxxxxxx

(--product is an actual product id on my server. And the scan file is a working file that can be imported through the UI)

When I run the file it results in the following failure :

Traceback (most recent call last):
  File "dojo_ci_cd.py", line 242, in <module>
    class Main:
  File "dojo_ci_cd.py", line 283, in Main
    test_ids = processFiles(dd, engagement_id, file, scanner=scanner)
  File "dojo_ci_cd.py", line 145, in processFiles
    if test_id.success == False:

On line 86 in dojo_ci_cd.py I change scannerName from None to "Burp Scan". I have no clue why scannerName is set to None in the first place. When I run the file again I get the following error :

Traceback (most recent call last):
  File "dojo_ci_cd.py", line 242, in <module>
    class Main:
  File "dojo_ci_cd.py", line 283, in Main
    test_ids = processFiles(dd, engagement_id, file, scanner=scanner)
  File "dojo_ci_cd.py", line 146, in processFiles
    print "Upload failed: Detailed error message: " + test_id.data
TypeError: cannot concatenate 'str' and 'NoneType' objects

I can keep hacking the dojo_ci_cd.py file, but it is obvious that what I am doing is wrong. And it will probably worsen the more I try, so I will stop right here :)

After hardcoding those values I get:

Upload failed: Detailed error message:
==============================================
Traceback (most recent call last):
  File "dd_import.py", line 239, in <module>
    class Main:
  File "dd_import.py", line 288, in Main
    summary(dd, engagement_id, test_ids, max_critical, max_high, max_medium)
  File "dd_import.py", line 164, in summary
    print "Total Number of Vulnerabilities: " + str(findings.data["meta"]["total_count"])
TypeError: string indices must be integers

[wurstbrot]

This works (based on securecodebox): https://github.com/wurstbrot/defectdojo-java-api It will be moved to DefectDojos repo space as soon as @aaronweaver will create a repo in that space.

cURL/Bash-based works also, but is no longer maintained: https://github.com/wurstbrot/defectdojo_api

[dougmcdorman]

Not using the Python API, but it is a Python example of uploading a scan.

from defectdojo_api import defectdojo
from datetime import datetime, timedelta
import os, sys
import argparse
import time
import requests

host = 'http://localhost:8080'
authn_value = 'Token aaaaabbbbbcccccddddddddeeeeeeeffffffff'

filename = '/home/username/zap-report.xml'

headers = {'Authorization': authn_value}
base_url = host
url = base_url + '/api/v2/import-scan/'

engagement_id = '7'
scan_type = 'ZAP Scan'
lead = 'admin'
active = 'true'
verified = 'true'
close_old_findings = 'true'
skip_duplicates = 'true'
scan_date = '2019-09-04'
tags = ''
build_id = ''
minimum_severity = ''
with open(filename, 'rb') as filehandle:
              filedata = filehandle.read()
              data = {
                               'file': filedata,
                               'engagement': ('', engagement_id),
                               'scan_type': ('', scan_type),
                               'active': ('', active),
                               'verified': ('', verified),
                               'close_old_findings': ('', close_old_findings),
                               'skip_duplicates': ('', skip_duplicates),
                               'scan_date': ('', scan_date),
                               #'tags': ('', tags), # tags gets upset if it is empty
                               'build_id': ('', build_id),
                               'minimum_severity': ('', minimum_severity)
                           }
              my_request = requests.Session()
              response = my_request.post(url=url, files=data, headers=headers)
              print(response.text)
              print(response.status_code)
print(requests.status_codes._codes[response.status_code])`

[willnil]

Thanks for trying to help guys, I did another concentrated attempt and will try to explain : My main struggle is dojo_ci_cd.py - the file does not run and there is no request sent to the server unless I make a series of horrible hacks in it. I will not be posting the web request here before I get to a point where I can confidently run the example script The example on https://github.com/aaronweaver/defectdojo_api/tree/master/examples reads : dojo_ci_cd.py --product=1 --file "/tests/scans/Bodgeit-burp.xml" --scanner="Burp Scan" --high=0 --host=http://localhost:8000 --api_key=<api_key> --user=admin The command I am using is : python dojo_ci_cd.py --product=10 --file "/xx/xx/xx/burp_ad/a9.xml" --scanner="Burp Scan" --high=0 --host=http://localhost:8000 --api_key=xxxxxxxxxxxx --user=xxxxxxxxx (--product is an actual product id on my server. And the scan file is a working file that can be imported through the UI) When I run the file it results in the following failure :

Traceback (most recent call last):
  File "dojo_ci_cd.py", line 242, in <module>
    class Main:
  File "dojo_ci_cd.py", line 283, in Main
    test_ids = processFiles(dd, engagement_id, file, scanner=scanner)
  File "dojo_ci_cd.py", line 145, in processFiles
    if test_id.success == False:

On line 86 in dojo_ci_cd.py I change scannerName from None to "Burp Scan". I have no clue why scannerName is set to None in the first place. When I run the file again I get the following error :

Traceback (most recent call last):
  File "dojo_ci_cd.py", line 242, in <module>
    class Main:
  File "dojo_ci_cd.py", line 283, in Main
    test_ids = processFiles(dd, engagement_id, file, scanner=scanner)
  File "dojo_ci_cd.py", line 146, in processFiles
    print "Upload failed: Detailed error message: " + test_id.data
TypeError: cannot concatenate 'str' and 'NoneType' objects

I can keep hacking the dojo_ci_cd.py file, but it is obvious that what I am doing is wrong. And it will probably worsen the more I try, so I will stop right here :)

After hardcoding those values I get:

Upload failed: Detailed error message:
==============================================
Traceback (most recent call last):
  File "dd_import.py", line 239, in <module>
    class Main:
  File "dd_import.py", line 288, in Main
    summary(dd, engagement_id, test_ids, max_critical, max_high, max_medium)
  File "dd_import.py", line 164, in summary
    print "Total Number of Vulnerabilities: " + str(findings.data["meta"]["total_count"])
TypeError: string indices must be integers

I am having the same TypeError now when using the defectdojo_api example python script: dojo_ci_cd.py trying to import my json scan files. I guess the problem might due to the json parser. I tried to hack the python file, like adding json.loads() or json.dumps() to the data from findings, but did not success.

Traceback (most recent call last):
  File "/home/ubuntu/persistant_storage/defectdojo_api/examples/dojo_ci_cd.py", line 244, in <module>
    class Main:
  File "/home/ubuntu/persistant_storage/defectdojo_api/examples/dojo_ci_cd.py", line 293, in Main
    summary(dd, engagement_id, test_ids, max_critical, max_high, max_medium)
  File "/home/ubuntu/persistant_storage/defectdojo_api/examples/dojo_ci_cd.py", line 168, in summary
    print "Total Number of Vulnerabilities: " + str(findings.data["meta"]["total_count"])
TypeError: string indices must be integers

@heepspray I have the same problem. Did you solve this or u try another approach to import your scans automatically?

Thank you all :)