Hayden User Blog

[sky-cake]

Hi @bbepis, I have just started using the Hayden Scraper again for archiving select 4chan threads with the Hayden database schema. It is working very well 😄 Thank you for this great tool!

I wanted to share some information which could help other users (and myself) in the future.

Hayden Version: (not sure where to find this) OS: Ubuntu Server 22 LTS MySQL: 8.0.34 .NETCore: 6.0.18

/etc/systemd/system/hayden.service

[Unit]
Description=Hayden Scraper
After=network-online.target mysql.service

[Service]
Type=simple
ExecStart=/mnt/hayden_asagi/Hayden scrape /mnt/hayden_asagi/config.json
WorkingDirectory=/mnt/hayden_asagi
User=m
Group=www-data
Restart=always
RestartSec=600
StandardOutput=null #append:/home/user/hayden_info.log
StandardError=append:/home/user/hayden_error.log
SyslogIdentifier=hayden

[Install]
WantedBy=multi-user.target

/mnt/hayden_asagi/config.json

{
	"source": {
		"type": "4chan",
		"boards": {
			"g": {
				"AnyFilter": "battlestation",
				"AnyBlacklist": "stable diff|dall.*e.*3"
			},
			"ck": {}, # download everything
		},
		"apiDelay": 5.5,
		"boardScrapeDelay": 45
	},
	"readArchive": false,
	"proxies": [],
	"consumer": {
		"type": "Asagi",
		"databaseType": "MySQL",
		"connectionString": "Server=127.0.0.1;Port=3306;Database=hayden;Uid=USER;Pwd=PASSWORD;",
		"downloadLocation": "/mnt/ayase_quart/src/static/hayden_asagi",
		"fullImagesEnabled": true,
		"thumbnailsEnabled": true
	}
}

My Hayden Scraper instance has run for 3 days now. I can confirm that it will continue archiving existing threads after several hours downtime -- restarting the Hayden Scraper service is no issue.

I gtg now, but I plan to add to this blogpost. I've also added some Hayden Scraper instructions at https://github.com/sky-cake/ayase-quart#hayden. Let me know what other information I should include here.