=========================== smtp-url-analysis

This package/policies have been updated to work with zeek (new broker framework).

Primary scope of these zeek policies is to give more insights into smtp-analysis esp to track phishing events.

This is a subset of phish-analysis repo and doesn't use any backed postgres database. So relieves the user from postgres dependency while getting basic phishing detection up and running very quickly.

Following functionality are provided by the script

:: 1) Works in a cluster and standalone mode 2) extracts URLs from Emails and logs them to smtpurl_links.log 3) Tracks these SMTP urls in http analyzer and logs if any of these SMTP URL has been clicked into a file smtp_clicked_urls.log 4) Reads a file for malicious indicators and generates an alert of any of those inddicators have a HIT in smtp traffic (see below for more details) 5) Generates alerts if suspicious strings are seen in URL (see below for details) 6) Generates alerts if a SMTP URL is clicked resulting in a file download

Installation

zkg install initconf/smtp-url-analysis
or
@load smtp-url-analysis/scripts

Upgrade

$ zkg upgrade zeek/initconf/smtp-url-analysis.git
The following packages will be UPGRADED:
  zeek/initconf/smtp-url-analysis.git (master)

Proceed? [Y/n] y
Running unit tests for "zeek/initconf/smtp-url-analysis.git"
all 7 tests successful
Upgraded "zeek/initconf/smtp-url-analysis.git" (master)

Detailed Notes:

All the configuration variables are in the following file. Please modify as needed:

    configure-variables-in-this-file.zeek

    Note: Make sure you replace "site.org" in the file with your domain name(s)

Detail Alerts and descriptions: Following alerts are generated by the script:

Heuristics in smtp-malicious-indicators.zeek are used to flag known sensitve IoC's (sort of your local smtp intel feed).

This should generate following Kinds of notices:

- Malicious_MD5,
- Malicious_Attachment,
- Malicious_Indicator,
- Malicious_Mailfrom,
- Malicious_Mailto,
- Malicious_from,
- Malicious_reply_to,
- Malicious_subject,
- Malicious_rcptto,
- Malicious_path,
- Malicious_Decoded_Subject

To activate these notices a sample smtp_malicious_indicators.out is provided in "scripts/feeds" directory. You either need to populate that or redef smtp_indicator_feed in configure-variables-in-this-file.zeek:

redef Phish::smtp_indicator_feed = "/feeds/BRO-feeds/smtp_malicious_indicators.out" ;

I have a cron job which scraps various email indicators (senders, subject, attachment, md5 hash etc) from various phish related feeds/notices and periodically creates this one file: /feeds/BRO-feeds/smtp_malicious_indicators.out. Bro reads this file using input-framework ie new additions/append/removal to this file doesn't requre zeek to be restarted.

Note: 1) Make sure the fields inthefile are seperated. 2) Make sure format of above feed file complies to:

Here is a sample format

#fields indicator       description
"At Your Service" <[email protected]>	Some random comment
[email protected]	some random comment
f402e0713127617bda852609b426caff	some bad hash
HelpDesk	some bad subject

Example alert:

• Phish::Malicious_rcptto

  Aug 24 11:26:06 CPLZuO3KTSDHx9mCC1 174.15.3.146 36906 18.3.1.10 25 tcp Phish::Malicious_rcptto Malicious rectto :: [[email protected], description=random test ], [email protected] [email protected] 174.15.3.146 18.3.1.10 25 zeek Notice::ACTION_EMAIL,Notice::ACTION_LOG 60.000000 F - - - - -

smtp-sensitive-uris.zeek will generate following alerts

- SensitiveURI
- Dotted_URL
- Suspicious_File_URL
- Suspicious_Embedded_Text
- WatchedFileType
- BogusSiteURL

Example Alert: BogusSiteURL

1503599166.565855       CPLZuO3KTSDHx9mCC1      1.1.1.1    36906   2.2.2.2    25      -       -       -       tcp     Phish::BogusSiteURL     Very similar URL to site: http://www.site.org.blah.com/ from  1.1.1.1       -       1.1.1.1    2.2.2.2  25      -       zeek     Notice::ACTION_EMAIL,Notice::ACTION_LOG 3600.000000     F       -       -       -       -       -

Again see configure-variables-in-this-file.zeek for tweaking and tunning

Example Alert: FileDownload

Malicious file download: If a link in an email is clicked and results in a file download, this module can generate an alert of that as well.

1481499234.568566       CQa9SJ1adwAqlPDcKj      1.1.1.1      49067   46.43.34.31     80      FxrREO3dgcnSlAQZO8      application/x-dosexec   http://the.earth.li/~sgtatham/putty/0.67/x86/putty.exe  tcp     Phish::FileDownload     [ts=1481431889.562629, uid=CX5ROKa8g7WcfnET4, from=Bad Guy <[email protected]>, to=John Doe <[email protected]>, subject=putty.exe, referrer=[]]        http://the.earth.li/~sgtatham/putty/0.67/x86/putty.exe  1.1.1.1      46.43.34.31     80      -       zeek     Notice::ACTION_LOG    3600.000000     F

Example Alert: Phish::DottedURL

Watch for URLs which only have IP address instead of domain names in them - another sign of maliciousness

1483418588.406004       CNDcli3Oo5dFqrJNhi      198.124.252.166 46134   128.3.41.120    25      -       -       -       tcp     Phish::DottedURL        Embeded IP in URL http://183.81.171.242/c.jpg from  198.124.252.166     -       198.124.252.166 128.3.41.120 25       -       zeek     Notice::ACTION_LOG      3600.000000     F

Example Alert: SensitiveURI

Generates an Alert when a string in URL matches signature defined in "suspicious_text_in_url" available in configure-variables-in-this-file.zeek

1351714828.429308       CAmJxI1WlO5E5bWxCj      128.3.41.133    1277    209.139.197.113 25      -       -       -       tcp     Phish::SensitiveURI     Suspicious text embeded in URL http://www.foxterciaimobiliaria.com.br/corretor/565/ from  CAmJxI1WlO5E5bWxCj -128.3.41.133    209.139.197.113 25      -       zeek     Notice::ACTION_LOG      3600.000000     F

Example Alert: Phish::WatchedFileType

Simple regexp match on file extensions. This is a noisy notice but useful for logging. for critical files flagging use (3) above which is malicious file download based on mime-types.

1481431889.683598       CxGUuzDvWCpUdFI27       74.125.83.52    35030   128.3.41.120    25      -       -       -       tcp     Phish::WatchedFileType  Suspicious filetype embeded in URL http://the.earth.li/~sgtatham/putty/0.67/x86/putty.exe from  74.125.83.52 -74.125.83.52    128.3.41.120    25      -       zeek     Notice::ACTION_LOG      3600.000000     F

Example Alert: SensitivePOST

This is generated when a URL in an email is clicked and results in a HTTP Post request. Often this is how passwords are transmitted on phishing sites.

1449085047.857802       COuvQB1n4JF3MILQUa      128.3.10.69     57106   67.227.172.217  80      -       -       -       tcp     Phish::HTTPSensitivePOST        Request: /cli/viewd0cument.dropboxxg.20gbfree.secure.verfy.l0gin.user0984987311111-config-l0gin-verfy.user763189713835763/validate.php - Data: type=G+Mail&[email protected]&tel=me&password=me&frmLogin:btnLogin1=&frmLogin:btnLogin1=      -       128.3.10.69     67.227.172.217  80      -       zeek     Notice::ACTION_LOG      3600.000000     F


Notice in alert below: [email protected]&tel=me&password=me

Example Alert: SensitivePassword

Alert is triggered when a password transmitted in HTTP SensitivePost is associated with a username related to sites' domain and the password meets the site's password complexity.

1467998894.642754       Ce3m7XMMIuScmhJu9       128.3.2.5    64310   104.16.58.61    80      -       -       -       tcp     HTTP::SensitivePasswd   Request: /electacta/login_action.asp - Data: [email protected]&password=Popiszcze$11&rememberMe=on&role=editor&bypass=&rememberUser=1&ignoreWarning=0       -       128.3.2.5    104.16.58.61    80      -       zeek     Notice::ACTION_LOG      3600.000000     F

========================= Logging

This module should generate two different logs - smtpurl_links.log - smtp_clicked_urls.log

smtpurl_links.log

This is a log of all URLs extracted from emails. A sample looks like this

smtp_clicked_urls.log

This is log of URLs from email which are 'clicked' on - ie which are later seen by the HTTP analyzer.

#fields	ts	uid	id.orig_h	id.orig_p	id.resp_h	id.resp_p	host	url	mail_ts	mail_uid	from	to	subject	referrer
#types	time	string	addr	port	addr	port	string	string	time	string	string	string	string	string

1449081495.794583	CtxTCR2Yer0FR1tIBg	131.243.195.188	61291	67.227.172.217	80	proposito.net	http://proposito.net/cli/viewd0cument.dropboxxg.20gbfree.secure.verfy.l0gin.user0984987311111-config-l0gin-verfy.user763189713835763.htm	1449081435.863394	CHhAvVGS1DHFjwGM9	Maggie Stoeva <[email protected]>	undisclosed-recipients:;	(2) Important Document from Maggie Stoeva	(empty)
1449085026.214280	CPhDKt12KQPUVbQz06	128.3.10.69	57064	67.227.172.217	80	proposito.net	http://proposito.net/cli/viewd0cument.dropboxxg.20gbfree.secure.verfy.l0gin.user0984987311111-config-l0gin-verfy.user763189713835763.htm	1449081435.863394	CHhAvVGS1DHFjwGM9	Maggie Stoeva <[email protected]>	undisclosed-recipients:;	(2) Important Document from Maggie Stoeva	(empty)