Questions? Feedback? powered by Olark live chat software
Knowledgebase
Author Avatar

Locate a scraper on cPanel servers

Written by: on 27 November 2017 01:59 PM 27 November 2017 01:59 PM

In case your Apache struggling with a lot of "scrapers" requests, which leads to abnormally high LA (load average) please do as described below:

  1. Login to your VPS via SSH as root (link here - https://www.vpsblocks.com.au/support/Knowledgebase/Article/View/41/0/how-do-i-use-ssh)
  2. Type the command (without brackets):

    awk '{ print $1 }' /usr/local/apache/domlogs/<yourdomain>  | sort -n | uniq -c | sort -nr | head -20

    == OR, if your traffic is mostly HTTPS ==

    awk '{ print $1 }' /usr/local/apache/domlogs/<yourdomain>-ssl  | sort -n | uniq -c | sort -nr | head -20
  3. If you see thousands of hits from particular IP address(es), they are most likely scrapers or bruteforcing bots, both deserve to be blocked.
  4. Login to WHM as root, go to "ConfigServer Security & Firewall", click on "Firewall Deny IPs" and add harvested IP addresses (or whole subnets if you wish to the end of the list. Then click "Change" and then "Restart csf+lfd".
(0 vote(s))
Helpful
Not helpful