pyeti icon indicating copy to clipboard operation
pyeti copied to clipboard

Pull data from YETI

Open rraj1996 opened this issue 7 years ago • 6 comments

Can anyone share a working example of yeti api that pulls data from Yeti.

I don't know what arguments to be where can i find them in my local hosted yeti.

Sharing Screenshots would be more helpful.

rraj1996 avatar Apr 18 '19 12:04 rraj1996

The examples in the read me all work. Is there something more you are looking for as far as examples ? There is also documentation here that might help https://yeti-platform.readthedocs.io/en/latest/api.html

threathive avatar Apr 20 '19 03:04 threathive

Yup @rraj1996 please let us know what you're trying to achieve specifically - I'm happy to add examples for these in the documentation if the ones already there aren't enough.

tomchop avatar Apr 20 '19 05:04 tomchop

First of all I would like to thank you guyz for replying on my post.

What is Yeti according to me?

According to my information gathering Yeti is a central TI source which stores in it TI feeds from different TI source so that managing of feeds and integration with different softwares would be easier.

What I want from yeti?

I want list of all Malicious IP's and Malicious url stored in Yeti's db.For more clarification I want to pull data from all the dataflows shown in the image. image

What is the better way to get this data?Making API call would be better or pulling data from mongo would be better?

Please provide solution with an example.

rraj1996 avatar Apr 23 '19 09:04 rraj1996

Well you can use the API to lookup data about a specific domain or IP. If you want to just dump raw lists of domains or really anything you could do this via an export in the exports tab you have in your screen shot. An example of this can be found in the documentation as well: https://yeti-platform.readthedocs.io/en/latest/use-cases.html#creating-a-blocklist. A really broad rule to get a CSV formatted export would be to do something like this.. go create a new template :

value,created
{%for obs in elements%}{{obs.value}},{{obs.created}}
{%endfor%}

Then you create a new export with that template. In this case I am dumping all active urls in the system. You could do the same for IP type as well. However it will continue to get huge so keep that in mind. It is much better to filter down the results by apply filters on the export including tags or exclusion rules.

Screenshot_20190423_110456

You end up with something like this in the end.

value,created
hxxps:/\/\onedrive.live.com/\redir?resid=355EB95EAF7742D3!105&authkey=!AOQhGMd2qywxpQM&ithint=file.xlsx&page=survey&wdFormId={B5E4E918-EA8D-4706-8725-8891137E4374},2019-04-22 21:42:56.209000

The export will run on a normal frequency and updates the output each time. So you can download this either via the api ( see the documentation ) or even a simple curl request would work as the links are static. Hopefully that gets you started.

threathive avatar Apr 23 '19 18:04 threathive

Like @threathive said, there are many ways to extract data from a running Yeti instance, and exports is one of them. You can also use

  • the REST API and the /api/observablesearch/ endpoint for this; a simple HTTP request with the appropriate parameters will do (you can use curl, wget, httpie...). The JSON object you need to pass in the POST body is described in the documentation.
  • The Python API (pyeti), which is basically a python wrapper around the REST API. See this example.

tomchop avatar Apr 23 '19 18:04 tomchop

Thanks:)

rraj1996 avatar Apr 24 '19 10:04 rraj1996