fail2banでreferrer spam domainをアクセス拒否

ロシアからのreferrer spamが酷くて対処方法を探していたら下記のサイトを見つけました｡

GitHub desbma/referer-spam-domains-blacklist
Referer Spam Blacklist
https://github.com/desbma/referer-spam-domains-blacklist

Piwik’s referrer spam blacklistから分岐したReferer Spam Blacklistで頻繁に更新されているようです｡

有り難いことにfail2banでの使用方法を記載されていましたので導入してみました｡

1. スクリプトのダウンロード
Shell

# curl https://raw.githubusercontent.com/desbma/referer-spam-domains-blacklist/master/update-fail2ban-referer-filter > /etc/cron.weekly/update-fail2ban-referer-filter

1	# curl https://raw.githubusercontent.com/desbma/referer-spam-domains-blacklist/master/update-fail2ban-referer-filter > /etc/cron.weekly/update-fail2ban-referer-filter

2. 実行権限の追加
Shell

# chmod +x /etc/cron.weekly/update-fail2ban-referer-filter # ls -l /etc/cron.weekly/update-fail2ban-referer-filter -rwxr-xr-x 1 root root 1401 May 26 16:00 /etc/cron.weekly/update-fail2ban-referer-filter

1 2 3

# chmod +x /etc/cron.weekly/update-fail2ban-referer-filter # ls -l /etc/cron.weekly/update-fail2ban-referer-filter -rwxr-xr-x 1 root root 1401 May 26 16:00 /etc/cron.weekly/update-fail2ban-referer-filter

3. jail.local設定

jail.localにフィルタを設定します｡

[apache-referer] enabled = true maxretry = 1 bantime = 604800 port = http,https filter = apache-referer logpath = %(apache_access_log)s

1 2 3 4 5 6 7

[apache-referer] enabled = true maxretry = 1 bantime = 604800 port = http,https filter = apache-referer logpath  = %(apache_access_log)s

4. update-fail2ban-referer-filterの実行

update-fail2ban-referer-filterの実行します｡これは下記の動作をするようです｡
1. spammers.txtをダウンロード
2. spammers.txtの差分を確認して同じなら終了
3. 差分があるなら/etc/fail2ban/filter.d/apache-referer.localを作成
4. fail2banのreload

以降は週で自動更新されます｡

Shell

# /etc/cron.weekly/update-fail2ban-referer-filter 1682 domains added, 0 removed Redirecting to /bin/systemctl reload fail2ban.service # cat /etc/fail2ban/filter.d/apache-referer.local # Fail2Ban configuration file # # Autogenerated by /etc/cron.weekly/update-fail2ban-referer-filter @ 2017-05-26 16:01:23+09:00 [Definition] failregex = ^<HOST> -.*"(GET|POST).*HTTP.*".+"https?://(.*\.)?(000Free\.us|00author\...... ignoreregex =

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

# /etc/cron.weekly/update-fail2ban-referer-filter 1682 domains added, 0 removed Redirecting to /bin/systemctl reload  fail2ban.service   # cat /etc/fail2ban/filter.d/apache-referer.local   # Fail2Ban configuration file #   # Autogenerated by /etc/cron.weekly/update-fail2ban-referer-filter @ 2017-05-26 16:01:23+09:00   [Definition]   failregex = ^<HOST> -.*"(GET|POST).*HTTP.*".+"https?://(.*\.)?(000Free\.us|00author\...... ignoreregex =

5. フィルタチェック
Shell

# fail2ban-regex --print-all-matched /var/log/httpd/ssl_access_log /etc/fail2ban/filter.d/apache-referer.local Running tests ============= Use failregex filter file : apache-referer, basedir: /etc/fail2ban Use log file : /var/log/httpd/ssl_access_log Use encoding : ANSI_X3.4-1968 Results ======= Failregex: 1 total |- #) [# of hits] regular expression | 1) [1] ^<HOST> -.*"(GET|POST).*HTTP.*".+"https?://(.*\.)?(000Free\.us|00author\.com|00go\.com|00it\.......*" ".*"$ `- Ignoreregex: 0 total Date template hits: |- [# of hits] date format | [225902] Day(?P<_sep>[-/])MON(?P=_sep)Year[ :]?24hour:Minute:Second(?:\.Microseconds)?(?: Zone offset)? `- Lines: 225902 lines, 0 ignored, 1 matched, 225901 missed [processed in 30.78 sec] |- Matched line(s): | 5.79.68.161 - - [22/May/2017:06:46:39 +0900] "GET / HTTP/1.1" 200 13434 "http://burger-imperia.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.120 Safari/537.36" `- Missed line(s): too many to print. Use --print-all-missed to print all 225901 lines

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

# fail2ban-regex --print-all-matched /var/log/httpd/ssl_access_log /etc/fail2ban/filter.d/apache-referer.local   Running tests =============   Use   failregex filter file : apache-referer, basedir: /etc/fail2ban Use         log file : /var/log/httpd/ssl_access_log Use         encoding : ANSI_X3.4-1968   Results =======   Failregex: 1 total |-  #) [# of hits] regular expression |   1) [1] ^<HOST> -.*"(GET|POST).*HTTP.*".+"https?://(.*\.)?(000Free\.us|00author\.com|00go\.com|00it\.......*" ".*"$ `-   Ignoreregex: 0 total   Date template hits: |- [# of hits] date format |  [225902] Day(?P<_sep>[-/])MON(?P=_sep)Year[ :]?24hour:Minute:Second(?:\.Microseconds)?(?: Zone offset)? `-   Lines: 225902 lines, 0 ignored, 1 matched, 225901 missed [processed in 30.78 sec]   |- Matched line(s): |  5.79.68.161 - - [22/May/2017:06:46:39 +0900] "GET / HTTP/1.1" 200 13434 "http://burger-imperia.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.120 Safari/537.36" `- Missed line(s): too many to print.  Use --print-all-missed to print all 225901 lines

先程確認したらbanしてました｡