User Tools

Site Tools


howtos:dspam_ham-spam_learning_script

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

howtos:dspam_ham-spam_learning_script [d/m/Y H:i] (current)
Line 1: Line 1:
 +====== Training script for Dspam ======
 +This script will go through all mail users in a predefined maildir directory and look for two imap folders: Spam and NotSpam.
  
 +Mail in the Spam folder will be processed as missed spam and dealt with accordingly by Dspam. The opposite is the case for mail put into NotSpam.
 +
 +===== Usage =====
 +Create the appropriate folders in the users maildir (or tell them to do so if they want to play along) and when ever a mail is missed or miss interpret move/copy it to the Spam or NotSpam folder.
 +
 +This is the beauty of the script, if you don't make the folders you opt-out and if you only make one of the folders you only choose to that kind of training.
 +
 +You then setup a cron job running once a day (or more frequent if you like), which will run through the folders, train and clean them up. 
 +
 +Remember to copy HAM to the NotSpam folder as you otherwise will loose the mail.
 +
 +===== train-spam.sh =====
 +
 +
 +<​file>​
 +#!/bin/bash
 +#
 +# train-spam.sh
 +#
 +# Description:​ Checks each user's virtual maildir for the 
 +# the folder "​.Spam"​ and "​.NotSpam"​ directories to see if 
 +# the user placed any "​missed"​ spam/ham messages which got 
 +# through Dspam to their INBOX.
 +# If there are messages in this directory, then the script
 +# invokes dspam to update the site-wide tokens to try
 +# and improve the defenses for next time...
 +#
 +
 +# learn_spam - Function which takes a directory and a user as
 +# arguments, and then feeds that directory to our anti-spam
 +# applications for further SPAM training.
 +#
 +# Arguments:
 +# $1 - Directory name containing SPAM emails. Required
 +# $2 - Dspam user name. Required
 +#
 +# Example:
 +# learn_spam /​usr/​local/​virtual/​domingo.dk/​domingo/​.Spam/​cur domingo@domingo.dk
 +#
 +function learn_spam {
 +
 +# set file location for dspam binary
 +DSPAM=/​usr/​local/​bin/​dspam
 +
 +# loop through all emails in given directory
 +for email in $(ls $1); do
 +
 +# process SPAM email using DSPAM
 +$DSPAM --source=error --class=spam --user $2 < $1/$email
 +echo -n "​."​
 +
 +# delete SPAM email
 +rm $1/$email
 +
 +done # end of email loop
 +
 +} # end function learn_spam
 +
 +# learn_ham - Function which takes a directory and a user as
 +# arguments, and then feeds that directory to our anti-spam
 +# applications for further HAM training.
 +#
 +# Arguments:
 +# $1 - Directory name containing HAM emails. Required
 +# $2 - Dspam user name. Required
 +#
 +# Example:
 +# learn_spam /​usr/​local/​virtual/​domingo.dk/​domingo/​.NotSpam/​cur domingo@domingo.dk
 +#
 +function learn_ham {
 +
 +# loop through all emails in given directory
 +for email in $(ls $1); do
 +
 +# process HAM email using DSPAM
 +/​usr/​local/​bin/​dspam --source=error --class=innocent --user $2 < $1/$email
 +echo -n "​."​
 +
 +# delete HAM
 +rm $1/$email
 +
 +done # end of email loop
 +
 +} # end function learn_ham
 +
 +#
 +# Script starts here!
 +#
 +# Our maildir location
 +maildir=/​usr/​local/​virtual
 +# Static Dspam user
 +user="​domingo@domingo.dk"​
 +
 +# loop through all users maildir ​
 +for domain in $(ls $maildir); do
 +
 +        for userdir in $(ls $maildir/​$domain);​ do
 +        # if there is a Spam/Missed maildir
 +                if [ -d $maildir/​$domain/​$userdir/​.Spam/​cur ]; then
 +                # then process any missed SPAM
 +                echo -n "​missed spam for $maildir/​$domain/​$userdir:​ "
 + #If using individual spam/ham settings ​               ​
 + #​learn_spam $maildir/​$domain/​$userdir/​.Spam/​cur $userdir@$domain
 +                #learn_spam $maildir/​$domain/​$userdir/​.Spam/​new $userdir@$domain
 + #I'm only using one user for spam learning
 + learn_spam $maildir/​$domain/​$userdir/​.Spam/​cur $user
 +                learn_spam $maildir/​$domain/​$userdir/​.Spam/​new $user
 +                ​
 +                echo ""​
 +                fi
 + # if there is a Spam/​NotSpam dir
 + if [ -d $maildir/​$domain/​$userdir/​.NotSpam/​cur ]; then
 +
 + # then process any falsely identified spam, i.e. HAM
 + echo -n "false positives for $maildir/​$domain/​$userdir:​ "
 + #If using individual spam/ham settings ​               ​
 + #​learn_ham $maildir/​$domain/​$userdir/​.NotSpam/​cur $userdir@$domain
 +                #learn_ham $maildir/​$domain/​$userdir/​.NotSpam/​new $userdir@$domain
 + #I'm only using one user for spam learning
 + learn_ham $maildir/​$domain/​$userdir/​.NotSpam/​cur $user
 +                learn_ham $maildir/​$domain/​$userdir/​.NotSpam/​new $user
 + echo ""​
 +
 + fi # end if
 +        done
 +
 +
 +
 +done # end for loop
 +
 +echo "​Done!"​
 +
 +</​file>​
 +
 +
 +===== Original script by Alan =====
 +
 +
 +<​file>​
 +
 +#!/bin/sh
 +
 +# train-spam.sh
 +#
 +# Description:​ Checks each user's /​home/​Maildir/​.Spam.Missed
 +# directories to see if the user placed any "​missed"​ spam
 +# messages which got through SpamAssassin to their INBOX.
 +# If there are messages in this directory, then the script
 +# invokes sa-learn to update the site-wide tokens to try
 +# and improve the defenses for next time...
 +#
 +
 +# learn_spam - Function which takes a directory and a user as
 +# arguments, and then feeds that directory to our anti-spam
 +# applications for further SPAM training.
 +#
 +# Arguments:
 +# $1 - Directory name containing SPAM emails. Required
 +# $2 - User name. If it is not provided, $USER will be used.
 +#
 +# Example:
 +# learn_spam /​home/​alank/​Maildir/​.Spam.Missed/​cur alank
 +#
 +function learn_spam {
 +
 +# loop through all emails in given directory
 +for email in $(ls $1); do
 +
 +# process SPAM email using DSPAM
 +/​usr/​local/​bin/​dspam --mode=teft --source=error --class=spam --feature=chained,​noise --user $2 < $1/$email
 +echo -n "​."​
 +
 +# delete SPAM email
 +rm $1/$email
 +
 +done # end of email loop
 +
 +} # end function learn_spam
 +
 +# learn_ham - Function which takes a directory and a user as
 +# arguments, and then feeds that directory to our anti-spam
 +# applications for further HAM training.
 +#
 +# Arguments:
 +# $1 - Directory name containing HAM emails. Required
 +# $2 - User name. If it is not provided, $USER will be used.
 +#
 +# Example:
 +# learn_ham /​home/​alank/​Maildir/​.Spam.NotSpam/​cur alank
 +#
 +function learn_ham {
 +
 +# loop through all emails in given directory
 +for email in $(ls $1); do
 +
 +# process HAM email using DSPAM
 +/​usr/​local/​bin/​dspam --mode=teft --source=error --class=innocent --feature=chained,​noise --user $2 < $1/$email
 +echo -n "​."​
 +
 +# delete HAM
 +rm $1/$email
 +
 +done # end of email loop
 +
 +} # end function learn_ham
 +
 +#
 +# Script starts here!
 +#
 +
 +# loop through all user home directories
 +for file in $(ls /home); do
 +
 +# if there is a Spam/Missed maildir
 +if [ -d /​home/​$file/​Maildir/​.Spam.Missed/​cur ]; then
 +
 +# then process any missed SPAM
 +echo -n "​missed spam for $file: "
 +learn_spam /​home/​$file/​Maildir/​.Spam.Missed/​cur $file
 +learn_spam /​home/​$file/​Maildir/​.Spam.Missed/​new $file
 +echo ""​
 +
 +fi # end if
 +
 +# if there is a Spam/​NotSpam dir
 +if [ -d /​home/​$file/​Maildir/​.Spam.NotSpam/​cur ]; then
 +
 +# then process any falsely identified spam, i.e. HAM
 +echo -n "false positives for $file: "
 +learn_ham /​home/​$file/​Maildir/​.Spam.NotSpam/​cur $file
 +learn_ham /​home/​$file/​Maildir/​.Spam.NotSpam/​new $file
 +echo ""​
 +
 +fi # end if
 +
 +done # end for loop
 +
 +echo "​Done!"​
 +</​file>​
 +
 +
 +----
 +Source: http://​c600g.blogspot.com/​2008/​02/​by-request-my-dspam-training-script.html
howtos/dspam_ham-spam_learning_script.txt · Last modified: d/m/Y H:i (external edit)