User Tools

Site Tools


howtos:dspam_ham-spam_learning_script

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

howtos:dspam_ham-spam_learning_script [d/m/Y H:i] (current)
Line 1: Line 1:
 +====== Training script for Dspam ======
 +This script will go through all mail users in a predefined maildir directory and look for two imap folders: Spam and NotSpam.
  
 +Mail in the Spam folder will be processed as missed spam and dealt with accordingly by Dspam. The opposite is the case for mail put into NotSpam.
 +
 +===== Usage =====
 +Create the appropriate folders in the users maildir (or tell them to do so if they want to play along) and when ever a mail is missed or miss interpret move/copy it to the Spam or NotSpam folder.
 +
 +This is the beauty of the script, if you don't make the folders you opt-out and if you only make one of the folders you only choose to that kind of training.
 +
 +You then setup a cron job running once a day (or more frequent if you like), which will run through the folders, train and clean them up. 
 +
 +Remember to copy HAM to the NotSpam folder as you otherwise will loose the mail.
 +
 +===== train-spam.sh =====
 +
 +
 +<file>
 +#!/bin/bash
 +#
 +# train-spam.sh
 +#
 +# Description: Checks each user's virtual maildir for the 
 +# the folder ".Spam" and ".NotSpam" directories to see if 
 +# the user placed any "missed" spam/ham messages which got 
 +# through Dspam to their INBOX.
 +# If there are messages in this directory, then the script
 +# invokes dspam to update the site-wide tokens to try
 +# and improve the defenses for next time...
 +#
 +
 +# learn_spam - Function which takes a directory and a user as
 +# arguments, and then feeds that directory to our anti-spam
 +# applications for further SPAM training.
 +#
 +# Arguments:
 +# $1 - Directory name containing SPAM emails. Required
 +# $2 - Dspam user name. Required
 +#
 +# Example:
 +# learn_spam /usr/local/virtual/domingo.dk/domingo/.Spam/cur domingo@domingo.dk
 +#
 +function learn_spam {
 +
 +# set file location for dspam binary
 +DSPAM=/usr/local/bin/dspam
 +
 +# loop through all emails in given directory
 +for email in $(ls $1); do
 +
 +# process SPAM email using DSPAM
 +$DSPAM --source=error --class=spam --user $2 < $1/$email
 +echo -n "."
 +
 +# delete SPAM email
 +rm $1/$email
 +
 +done # end of email loop
 +
 +} # end function learn_spam
 +
 +# learn_ham - Function which takes a directory and a user as
 +# arguments, and then feeds that directory to our anti-spam
 +# applications for further HAM training.
 +#
 +# Arguments:
 +# $1 - Directory name containing HAM emails. Required
 +# $2 - Dspam user name. Required
 +#
 +# Example:
 +# learn_spam /usr/local/virtual/domingo.dk/domingo/.NotSpam/cur domingo@domingo.dk
 +#
 +function learn_ham {
 +
 +# loop through all emails in given directory
 +for email in $(ls $1); do
 +
 +# process HAM email using DSPAM
 +/usr/local/bin/dspam --source=error --class=innocent --user $2 < $1/$email
 +echo -n "."
 +
 +# delete HAM
 +rm $1/$email
 +
 +done # end of email loop
 +
 +} # end function learn_ham
 +
 +#
 +# Script starts here!
 +#
 +# Our maildir location
 +maildir=/usr/local/virtual
 +# Static Dspam user
 +user="domingo@domingo.dk"
 +
 +# loop through all users maildir 
 +for domain in $(ls $maildir); do
 +
 +        for userdir in $(ls $maildir/$domain); do
 +        # if there is a Spam/Missed maildir
 +                if [ -d $maildir/$domain/$userdir/.Spam/cur ]; then
 +                # then process any missed SPAM
 +                echo -n "missed spam for $maildir/$domain/$userdir: "
 + #If using individual spam/ham settings                
 + #learn_spam $maildir/$domain/$userdir/.Spam/cur $userdir@$domain
 +                #learn_spam $maildir/$domain/$userdir/.Spam/new $userdir@$domain
 + #I'm only using one user for spam learning
 + learn_spam $maildir/$domain/$userdir/.Spam/cur $user
 +                learn_spam $maildir/$domain/$userdir/.Spam/new $user
 +                
 +                echo ""
 +                fi
 + # if there is a Spam/NotSpam dir
 + if [ -d $maildir/$domain/$userdir/.NotSpam/cur ]; then
 +
 + # then process any falsely identified spam, i.e. HAM
 + echo -n "false positives for $maildir/$domain/$userdir: "
 + #If using individual spam/ham settings                
 + #learn_ham $maildir/$domain/$userdir/.NotSpam/cur $userdir@$domain
 +                #learn_ham $maildir/$domain/$userdir/.NotSpam/new $userdir@$domain
 + #I'm only using one user for spam learning
 + learn_ham $maildir/$domain/$userdir/.NotSpam/cur $user
 +                learn_ham $maildir/$domain/$userdir/.NotSpam/new $user
 + echo ""
 +
 + fi # end if
 +        done
 +
 +
 +
 +done # end for loop
 +
 +echo "Done!"
 +
 +</file>
 +
 +
 +===== Original script by Alan =====
 +
 +
 +<file>
 +
 +#!/bin/sh
 +
 +# train-spam.sh
 +#
 +# Description: Checks each user's /home/Maildir/.Spam.Missed
 +# directories to see if the user placed any "missed" spam
 +# messages which got through SpamAssassin to their INBOX.
 +# If there are messages in this directory, then the script
 +# invokes sa-learn to update the site-wide tokens to try
 +# and improve the defenses for next time...
 +#
 +
 +# learn_spam - Function which takes a directory and a user as
 +# arguments, and then feeds that directory to our anti-spam
 +# applications for further SPAM training.
 +#
 +# Arguments:
 +# $1 - Directory name containing SPAM emails. Required
 +# $2 - User name. If it is not provided, $USER will be used.
 +#
 +# Example:
 +# learn_spam /home/alank/Maildir/.Spam.Missed/cur alank
 +#
 +function learn_spam {
 +
 +# loop through all emails in given directory
 +for email in $(ls $1); do
 +
 +# process SPAM email using DSPAM
 +/usr/local/bin/dspam --mode=teft --source=error --class=spam --feature=chained,noise --user $2 < $1/$email
 +echo -n "."
 +
 +# delete SPAM email
 +rm $1/$email
 +
 +done # end of email loop
 +
 +} # end function learn_spam
 +
 +# learn_ham - Function which takes a directory and a user as
 +# arguments, and then feeds that directory to our anti-spam
 +# applications for further HAM training.
 +#
 +# Arguments:
 +# $1 - Directory name containing HAM emails. Required
 +# $2 - User name. If it is not provided, $USER will be used.
 +#
 +# Example:
 +# learn_ham /home/alank/Maildir/.Spam.NotSpam/cur alank
 +#
 +function learn_ham {
 +
 +# loop through all emails in given directory
 +for email in $(ls $1); do
 +
 +# process HAM email using DSPAM
 +/usr/local/bin/dspam --mode=teft --source=error --class=innocent --feature=chained,noise --user $2 < $1/$email
 +echo -n "."
 +
 +# delete HAM
 +rm $1/$email
 +
 +done # end of email loop
 +
 +} # end function learn_ham
 +
 +#
 +# Script starts here!
 +#
 +
 +# loop through all user home directories
 +for file in $(ls /home); do
 +
 +# if there is a Spam/Missed maildir
 +if [ -d /home/$file/Maildir/.Spam.Missed/cur ]; then
 +
 +# then process any missed SPAM
 +echo -n "missed spam for $file: "
 +learn_spam /home/$file/Maildir/.Spam.Missed/cur $file
 +learn_spam /home/$file/Maildir/.Spam.Missed/new $file
 +echo ""
 +
 +fi # end if
 +
 +# if there is a Spam/NotSpam dir
 +if [ -d /home/$file/Maildir/.Spam.NotSpam/cur ]; then
 +
 +# then process any falsely identified spam, i.e. HAM
 +echo -n "false positives for $file: "
 +learn_ham /home/$file/Maildir/.Spam.NotSpam/cur $file
 +learn_ham /home/$file/Maildir/.Spam.NotSpam/new $file
 +echo ""
 +
 +fi # end if
 +
 +done # end for loop
 +
 +echo "Done!"
 +</file>
 +
 +
 +----
 +Source: http://c600g.blogspot.com/2008/02/by-request-my-dspam-training-script.html
howtos/dspam_ham-spam_learning_script.txt · Last modified: d/m/Y H:i (external edit)