recon-pipeline/docs/checklist.md

Recon Pipeline — Ops Checklist

---

Daily

• Check Matrix for nuclei findings — any high/critical? Triage immediately. Medium can wait.
• Check all 3 Jenkins jobs ran without FAILURE — subfinder (if scheduled), httpx, nuclei. Red build = investigate.
• Check nuclei queue size — growing = nuclei falling behind. Stable/shrinking = healthy.

  cat /var/jenkins_home/recon-state/nuclei/nuclei-queue.txt | wc -l
  

• Check httpx metadata — new vs changed ratio — too many CHANGEDs vs NEWs = state still healing. Normal after 2 cycles.

  cat /var/jenkins_home/recon-state/httpx/metadata.txt
  

• Glance at daily digest for interesting new hosts — login panels, admin, API gateways, staging environments.

  cat /var/jenkins_home/recon-state/httpx/daily-digest.txt
  

---

Weekly

• Verify httpx completed at least 1 full cycle — pointer should have hit 0 and wrapped around at least once. ~57 runs = 1 cycle.

  cat /var/jenkins_home/recon-state/httpx/chunk-pointer.txt
  

• Check CHANGED count trending down — after 2 cycles: CHANGEDs should be 0–5 per run, not 40–170. Still high = state bug.
• Review cumulative nuclei findings for patterns — same template firing on many hosts = systemic issue worth reporting.

  sort /var/jenkins_home/recon-state/nuclei/nuclei-findings-cumulative.txt | uniq -c | sort -rn | head -20
  

• Check disk usage of recon-state — history folders grow unbounded. Prune if over a few hundred MB.

  du -sh /var/jenkins_home/recon-state/
  

• Review grep-interesting for new high-value targets — new admin panels, Jenkins, Grafana, Swagger UIs that appeared this week.

  cat /var/jenkins_home/workspace/*/current-run/results/grep-interesting.txt
  

• Update NUCLEI_BLACKLIST if noisy domains appeared — domains producing only false positives should be blacklisted to keep queue clean.

---

Monthly

• Review and update DOMAINS list in subfinder — new acquisitions? Domains removed from scope? Keep it current.
• Prune history folders — keep last 30 builds, delete older ones to save disk.

  ls -t /var/jenkins_home/recon-state/nuclei/history/ | tail -n +31 | xargs rm
  ls -t /var/jenkins_home/recon-state/httpx/history/ | tail -n +31 | xargs rm
  

• Update nuclei templates manually and check for new template categories — new high-signal template sets worth adding? Check ProjectDiscovery releases.
• Review GREP_PATTERNS — any new tech stacks worth adding? — new tools in the target's stack that weren't there last month.
• Check httpx + nuclei versions, update if needed — new releases often fix false positives and add detection.

  httpx -version && nuclei -version
  

• Full pipeline health check — run all 3 jobs manually — trigger subfinder → wait → trigger httpx → check queue → check nuclei. End-to-end smoke test.