Data Collection

Scrapy Crawler

This is a firmware scraper that aims to download firmware images and associated metadata from supported device vendor websites. The original version is designed by https://github.com/firmadyne/scraper. The spiders are extended and updated.

Dependencies

• psycopg2
• scrapy

Usage

To run a specific scraper, e.g. dlink:

scrapy crawl dlink

To run all scrapers with maximum 4 in parallel, using GNU Parallel:

The scriper is not fully tested when running in parallel. parallel -j 4 scrapy crawl ::: `for i in ./firmware/spiders/*.py; do basename ${i%.*}; done`

Metadata

The ./metadata folder contains the meta data of firmware images used in the paper. The json files include the download links, firmware categories, release time and etc.

• 360: /metadata/360.json
• AVM: /metadata/avm.json
• belkin: /metadata/belkin.json
• buffalo: /metadata/buffalo.json
• camius: /metadata/camius.json
• dlink: /metadata/dlink.json
• hikvision: /metadata/hikvision.json
• linksys: /metadata/linksys.json
• mercury: /metadata/mercury.json
• mikrotik: /metadata/mikrotik.json
• netcore: /metadata/netcore.json
• netgear: ```/metadata/netgear.json`
• openwrt: /metadata/openwrt.json
• router-tech: /metadata/routertech.json
• supermirco: /metadata/supermirco.json
• Tenda: /metadata/tenda-zh.json /metadata/tenda-en.json
• tenvis: /metadata/tenvis.json
• tomato-shibby: /metadata/tomato.json
• Tp-link: /metadata/tplink-en.json /metadata/tplink-zh.json
• Trendnet: /metadata/trendnet.json
• ubiquiti: /metadata/ubiquiti.json
• ublox: /metadata/ublox.json
• zyxel: /metadata/zyxel.json

Analyzer

ELF Analyzer

The ELF analyzer takes one file as input to check the mitigations adopted by the ELF binary. We use Binaryninja to disassemble statically-linked binary to verify Stack Canary and Fortify Source. We separate this functionality from elf_analyzer.py since BinaryNinja is a commercial tool unavailable to general public.

Dependencies

• elftools
• pwntools
• Binaryninja (optional)

Usage

Run with following command:

python3 elf_analyzer.py file_path

The output includes the mitigations adopted by the ELF binary

File name: /bin/ls
Arch: amd64-64
Dynamic linked: Yes
PIE/PIC: Yes
Protected with Canary: Yes
Protected with NX: Yes
Protected with full RELRO
Protected with Fortify Source: Yes

To accurately verify Stack Canary & Fortify Source in statically-linked binary, run following command with Binaryninja correctly installed:

python3 stc_stack.py file_path

Kernel Analyzer

The kernel analyzer takes ELF format Linux kernel as input to check the Linux kernel version and the existence of kernel attack mitigations. vmlinux-to-elf can be used to transfer raw format Linux kernel to a fully analyzable ELF file.

Dependencies:

• vmlinux-to-elf

Usage:

The kernel_example folder contains a raw format Linux kernel its corresponding recovered ELF format Linux kernel. To check the mitigations, run the example with following:

bash kernel_analyzer.sh -f kernel_example/elf_kernel

Citation

If your research find one or several components of this work useful, please cite the following paper:

@article{yu2022building,
  title={Building Embedded Systems Like It's 1996},
  author={Yu, Ruotong and Del Nin, Francesca and Zhang, Yuchen and Huang, Shan and Kaliyar, Pallavi and Zakto, Sarah and Conti, Mauro and Portokalidis, Georgios and Xu, Jun},
  journal={arXiv preprint arXiv:2203.06834},
  year={2022}
}