Problems

[lupedsagaces]

📋 Problem Description

During production usage, I identified scenarios where JSMon can fail or behave unstably:

• Missing files: Script crashes if files in downloads/ directory are removed
• Network failures: No automatic retry for temporary connection issues
• SSL certificates: Expired/invalid certificates cause crashes
• Bot detection: Basic headers may be blocked by anti-scraping systems
• No timeouts: Requests can hang indefinitely

🔧 Proposed Solutions

1. Robust Retry System

retry = Retry(
    total=3,
    backoff_factor=1,
    status_forcelist=[500, 502, 503, 504],
    raise_on_status=False,
)
adapter = HTTPAdapter(max_retries=retry)
session.mount("https://", adapter)
session.mount("http://", adapter)

2. Protection Against Missing Files

try:
    oldlines = open(f"downloads/{old}", "r").readlines()
except FileNotFoundError:
    oldlines = []

try:
    newlines = open(f"downloads/{new}", "r").readlines()
except FileNotFoundError:
    newlines = []
    
if not oldlines or not newlines:
    return ""
3. Realistic HTTP Headers
pythonheaders = {
    "User-Agent": (
        "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
        "AppleWebKit/537.36 (KHTML, like Gecko) "
        "Chrome/120.0.0.0 Safari/537.36"
    ),
    "Accept": "*/*",
    "Connection": "keep-alive",
}

4. Enhanced SSL Handling

try:
    response = session.get(endpoint, headers=headers, timeout=10, verify=False)
    response.raise_for_status()
    return response.text
except requests.exceptions.SSLError as ssl_err:
    print(f"[SSL ERROR] SSL failure accessing {endpoint}: {ssl_err}")
    return ""
except requests.exceptions.RequestException as e:
    print(f"[ERROR] Failed to access {endpoint}: {e}")
    return ""
5. File Size Protection
pythondef notify(endpoint, prev, new):
    diff = get_diff(prev, new)
    if not diff:
        print(f"[!] Could not generate diff for {endpoint} (missing file)")
        return

    try:
        prevsize = get_file_stats(prev).st_size
    except FileNotFoundError:
        prevsize = 0

    try:
        newsize = get_file_stats(new).st_size
    except FileNotFoundError:
        newsize = 0

✅ Benefits

Zero breaking changes - maintains full backward compatibility Higher reliability - works even with connectivity issues Automatic recovery - continues working after downloads folder cleanup Reduced blocking - realistic headers avoid bot detection Modern Python - f-strings and best practices Better logging - clearer error messages Graceful degradation - continues monitoring even when individual endpoints fail

🧪 Testing I have implemented and tested these improvements in a production environment where they successfully resolved:

Intermittent network connectivity issues SSL certificate problems with some monitored endpoints Accidental deletion of historical files Detection and blocking by Cloudflare and similar services

🎯 Implementation I have a complete, tested version ready. I can create a Pull Request if there's interest. All improvements maintain 100% compatibility with existing configurations and workflows.