@(#) This page contains man pages from the Procmail modules
@(#) I have written and that can be used as includerc plug-ins:
@(#) some can be used as subroutines to your own programs and
@(#) some are self containing recipes. There is no user or site
@(#) specific settings in these modules.
@(#) I have tried to minimize the use of external processes and
@(#) use the procmail way as much as possible (eg. to get date and
@(#) directly from message without calling expensive shell date).
@(#) It is important that you concentrate on this performance factor
@(#) too; remember that procmail is run on every incoming message and
@(#) every tick you spend counts.
@(#) $Id: pm-code.txt,v 1.14 1999/04/23 10:21:17 jaalto Exp $
$Contactid: jari.aalto@poboxes.com $
$URL: http://www.procmail.org/jari/ $
$HomePageURL: http://poboxes.com/jari.aalto/ $
$Keywords: procmail library includerc subroutine recipe $
Document control
This document has been automatically generated from the procmail rc files with 2 small perl scripts in the following manner
% ripdoc.pl `ls pm-ja*.rc|sort` | \ t2html.pl --name-uniq --simple > pm-code.htmlThe perl program assume that the documentation sections have been written in Technical Text Format and you are looking at TF layout in this file. The perl programs can be found from CPAN http://www.perl.com/CPAN-local//scripts/
Downloading procmail code
http://www.procmail.org/jari/pm-code.zipAutomatic URL change notification service
If you want to have automatic notification whenever my page changes, please visit this link and register any ULR in your reminder list. See also procmail module pm-janetmind.rc
File id
Copyright (C) 1997-98 Jari Aalto
$Contactid: jari.aalto@poboxes.com $
$Created: 1997-11 $
$keywords: procmail extractEmailAddress subroutine $
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
This includerc extracts the various components of email address from variable INPUT. You can do quite a lot interesting things with your email address. Refer to http://www.qz.to/~eli/faqs/addressing.html for more information. One of the tricks that you could use if you don't have sendmail plus addressing capabilities, is that you put the additional infomation to the RFC comment. Eg. If you read and followup to posts in usenet games groups, you could use:From: login@site.com (John Doe+usenet.games)Or if your email address's localpart (that's characters before @) already signify your First and surname, you don't need to repeat it in comment. However, place special marker "+" to mark additional information part for your procmail recipes:
From: first.surname@site.com (+usenet.games)The use of RFC comment should work everywhere because RFC requires that comments are preserved along with the address information. If you would have sendmail plus addressing capabilities you would have used:
From: login+usenet.games@site.com (John Doe)The idea is that the list infomation is readily available from the email. The following recipe will derive the plus information and use it directly as a mailbox where to drop the message. I The Editor: Emacs, means anything to you, you can program it to generate the appropriate From headers automatically when you send mail from Gnus Mail/Newsreader MUA. Drop me a message if you need an examle how a piece of Emacs lisp code makes those magic RFC plus addresses in the background while you compose the body of the message.
RC_EMAIL = $PMSRC/pm-jaaddr.rc TOME = "(login1|login2)":0 *$ ^TO\/.*$TOME.* { INPUT = $MATCH INCLUDERC = $RC_EMAIL PLUS = $COMMENT_PLUS# If COMMENT_PLUS was defined, we found "+" # address which contained "usenet.games", save it to # folder.:0 : * PLUS ?? [a-z] $PLUS }Notes
1998-05 David Hunt dh@west.net also mentioned that "you need to remember that some MTAs, (qmail for one, and soon vmail) use a dash ( - ) as the subaddress delimiter. So you'll want to allow for that in your code". For this reason the email part accepts both "-" and "+". The RFC comment however accepts only "+".Example input
"From: foo+procmail@this.site.com (Mr. foo)" traditional "From: foo-procmail@this.site.com (Mr. foo)" new styledNOTE: M$SOFT mailers tend to send idiotic smart quotes "'Mr. foo'" and this recipe ignores these two quotes ["'] as if message had only the standard ["]
Returned values
ADDRESS "foo+procmail@this.site.com" containing the email address without <>ACCOUNT "foo+procmail" all characters before @ACCOUNT1 "foo" characters before plus: account1+account2@site Note, if there is no "+", this is same as ACCOUNT.ACCOUNT2 "procmail" _only_ set if plus found: account1+account2@siteSITE "this.site.com" all characters after @DOMAIN "site.com" the main domain, preceding words in site are considered subdomain (local) addresses.sub.sub.domain.netSUB "this.site" all the sub-domain names without the NET part.SUB1 "site" The first subdomain counted from the _RIGHT_ after NETSUB2 "this" Second subdomain.SUB3 "" Third subdomain.SUB4 "" Fourth subdomain.NET "com" last characters after last period ( net,com,edu ...)COMMENT Anything unside parenthesis (Mr. Foo) or if no parentheses found, then anything between quotes "Mr. Foo"COMMENT_PLUS Anything after the "+" in the comment, like "Mr Foo+mail.usenet" --> "mail.usenet"Note: some MTA's don't allow + character, this alternatively '--' can be used: "Mr Foo--mail.usenet" --> "mail.usenet"Additionally there is variables DOT1 DOT2, which behave like ACCOUNT1 and ACCOUNT2, but in respect to dotted firstname.surname type address:
john.doe@site.comACCOUNT1 = john.doe ACCOUNT2 = <empty> DOT1 = john DOT2 = doeIf there is plus, the ACCOUNT2 is defined
john.doe+foo@site.comACCOUNT1 = john.doe ACCOUNT2 = foo DOT1 = john (in respect to ACCOUNT1) DOT2 = doe (in respect to ACCOUNT1)Variable ERROR is set to "yes" if INPUT wasn't recognized or parsing the address failed.
Required settings
PMSRC must point to source direcry of procmail code. This subroutine will include pm-javar.rc from there.Call arguments (variables to set before calling)
INPUT = string-to-parseUsage example
Read From field and address from it. This is lot faster than using external formail call.PMSRC = $HOME/pm RC_ADDR = $PMSRC/pm-jaaddr.rc:0 * ^From:\/.*@.* { INPUT = $MATCH# Turn off the logging while executing this part VERBOSE="off" INCLUDERC = $RC_ADDR VERBOSE="on":0 * ERROR ?? yes { # Hmm, no std email address found. Any other ideas? } }
File id
Copyright (C) 1997-98 John Gianni, Jari Aalto
IdeaBy 1995-01 John Gianni jjg@cadence.com
Maintainer: Jari Aalto
Created: 1997-12
Contactid: jari.aalto@poboxes.com
$Keywords: procmail backup recipe $
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
Preserve last N arriving messages in a separate sub-directory. This should be your safety-belt recipe that you put to the beginning of your .procmailrc.Procmail saves the backup files with names like: msg.rcG msg.scG msg.3YS1, msg.4YS1, msg.VYS1, msg.fYS1 to the backup directory.
Note: this recipe will alawys call shell commands for each message you recive. That is needed because cleaning of the backup directory. If you receive only small number of messages per day, the performance drop of your .procmailrc is not crucial. But if you store many messages per day, then the shell calls may be a performance problem.
In that case, consider moving the cleanup to the pm-jacron.rc module (The cleanup is run only once a day, not for every message)
John Gianni send his simple bsckup script to Jari, who packaged and generalized the code. The code is reused with John's permission and maintaining responsibility was transferred to Jari
Required settings
(none)Call arguments (variables to set before calling)
- JA_BUP_MAX, How many messages to keep at maximum. 32 is default
- JA_BUP_DIR, Where to store the messages. $HOME/Mail/bup by default
- JA_BUP_FILES, regexp to match the saved files. Procmail default.
- JA_BUP_CHECK_DIR. Once you have verified that this recipe works, that directories are ok, please set this flag to "no" to prevent running unnecessary test command for each email.
Usage example
You only want to keep backup of messages that are not from mailing lists. You may want to use TO_ macro to detect addresses better, this example matches against all headersLISTS = "(procmail|list-1|list-2)" JA_BUP_DIR = $HOME/Mail/backup/. # Create the path too JA_BUP_MAX = 42 # this should be enough:0 c *$ ! $LISTS { INCLUDERC = $PMSRC/pm-jabup.rc }If you get many messages, please don't use this module. Instead see pm-jacron.rc where similar backup work is done better.
File id
Copyright (C) 1997-98 Jari Aalto
Contactid: jari.aalto@poboxes.com
Created: 1997-12
keywords: procmail cookieConfirm recipe
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Overview of features
- Each user must register himself to cookie cache before he is considered "known"
- Unless user return the generated cookie string; which is typically a decimal or hex number, he is not considered as known and should not have access to services you provide.
- Can be used as a "doorbell" spam/UBE shield
Description
This recipe handles generating the cookie to new users, comparing the returned cookie against the original one and passing known users through if they already had returned their cookie.When you run automatised scripts, eg. to manage mailing lists where users can subscribe and unsubscribe, you have better to install safety measure so that someone can not subscribe his enemy to 30 mailing lists.
The cookie is any continuous block of random characters that is sent to person who wanted to use the service. He must send back the cookie before the service starts or action, like subscribe will take in effect. If someone forges the From address to pretend to be someone else and then subscribes as-beeing-someone-else to a mailing list, the cookie protects this from happening.
The cookie is sent to someone-else, and he must return the cookie before the "subscribe" service is activated. Obviously this someone-else will not be interested in sending back the cookie and thus the forgery fails. Isn't that simple, but efective protection against misuse?
And nowadays when UBE (aka Spam) is crawling from every possible domain thinkable, you can also start requesting that people "join" to your white list, before you start discussing with them. Bulk email shotguns do not reply to you, so you won't get cookies back from them, but individual peoples that want to talk with you, may want to return the cookie. For start you should add your friends to your cookie database by hand, so that they don't get upset when you activate "cookie-shield".
It's burden, I admit, to have to response to the cookies, but if that is the best way so far after the spam filters, to keep out uninvited people. Think this as a "doorbell", you don't usually let people in to your house without invitation or your approval.
If you have strong spam shield already, then you may not want to install cookie confirmation for your mates. But it's possible with this module.
How it works
By default, the cookie generated uses CRC 32 cksum, but if you have md5, you should use it. The cookie is generated from the reply address and immediately stored to cookie database file with entryDATE FROM-a COOKIE-a DATE FROM-b COOKIE-bIf this was a new user or an old user, who has not registered his cookie yet, then original message is sent back to the sender with instructions: "please place the magic string to Subject line and resent the message."
When cookie is returned back, a new line to the database is added, simply by adding a duplicate entry. The file now looks like this:
DATE FROM-a COOKIE-a DATE FROM-b COOKIE-b DATE FROM-a COOKIE-aWhen there is two or more same entries, like FROM-a, the address is supposed to be known and person behind it "cleared".
Required settings
PMSRC must point to source direcry of procmail code. This subroutine will include
- pm-javar.rc
- pm-jadate.rc
- pm-jacookie1.rc
- pm-jastore.rc
Call arguments (variables to set before calling)
- JA_COOKIE_SEND, flag. Default is "yes". Set to "no" if you want to take full control of the message returned to user. You can check variable ERROR and use key which holds the unique cookie
- JA_COOKIE_CACHE, cache to determine if this is new user or not.
- JA_COOKIE_AUTO_KEY, flag. If set to "yes"; the cookie is initially put to the Subject when the message is bounched back. Receiver only has to press "r" to reply to send the cookie and message back (convenient). You set this flag to "no" when you want to avoid accidnebts eg. when receiver is about to subscribe to a mailing lists: he has to manually insert the cookie into subject. But keep flag to "yes" if you use this module to get your friends registered easily.
- JA_COOKIE_KEYS, the cookie database. Email address and person's access cookie.
- JA_COOKIE_MBOX, if contains [a-z] characters, then the message that got bounched due to unknown user, is stored into this mailbox. Only used if JA_COOKIE_SEND is set to "yes".
- JA_COOKIE_RC, dubroutine to generate the cookie id from INPUT. By default uses CRC 32.
- JA_COOKIE, the string from which the cookie will be generated. If you already have the return addres for the sender derived, you should assing a value to this to save unnecessary formail call.
Returned values
ERROR will contain the efective action when this recipe file ends
- "new-user", This is first message from sender.
- "known-user", message has email that has been "cleared" ie. cookie had been returned and user registered.
- "key-mismatch", This is at least second message from sender. But he dind't send the confirmation in this message.
key is an internal variable in this recipe file and will hold the cookie id in case of "new-user" and "key-mismatch". You may want to use it if you generate your own reply.
Example usage for UBE shield
This is what I use to prevent unknown people from sending me UBE. It takes a bit extra, but they can easily return the message. Fill in the missing variables, this won't work out of the box for you.WORK = "(domain1|domain2|domain3)" LISTS = "(procmail|list-2|list-3|list-4)" VALID = "(postmaster|abuse|$LISTS|$WORK)" RC_COOKIE = $PMSRC/pm-jacookie.rc:0 *$ ! From:.*$VALID *$ ! $from_daemon { JA_COOKIE_SEND = "no" # Take over the resend control INCLUDERC = $RC_COOKIE:0 wc: $RC_COOKIE.lock * mail ?? yes * ! ^X-Loop: $JA_COOKIE_XLOOP * ^Subject:\/.* | ( $FORMAIL -rkt -b \ -A "X-Authentication-key: $key [$ERROR]" \ -A "X-Loop: $JA_COOKIE_XLOOP"; \ -I "Subject: $MATCH $key" \ echo "You Authentication key is: $key"; \ echo "$JA_COOKIE_MSG"; \ ) | $SENDMAIL; \ echo "[my; cookie; $ERROR; $JA_COOKIE]" >> $HIN:0 * mail ?? yes $COOKIE_SPOOL }Example usage for subscriptions
:0 c: # keep backups, expired by cron $BACKUP$RC_COOKIE = $PMSRC/pm-jacookie.rc...Mailing lists handled here... ...Your work messages filed here..TO = `formail -rt -zxTo:` # We need this elswhere JA_COOKIE_TO = $TO # Update this, it is known# I run private List-X, and all subscribe requests must # be confirmaed* ^TO_()list-x * ^Subject: +subscribe\> { JA_COOKIE_SEND = "no" INCLUDERC = $RC_COOKIE:0 * ERROR ?? known-user { # User sent the subsribe request again, allow joining # immediately. } :0 E { # Because the Send was set to "no"; we're in charge # to send a reply to the user. # ...generate suitable message with formail -rt }}# Any other messages may be UBE JA_COOKIE_SEND = "yes" INCLUDERC = $RC_COOKIE
File id
Copyright (C) 1997-98 Jari Aalto
$Contactid: jari.aalto@poboxes.com $
$Created: 1997-12 $
$keywords: procmail cookieMake subroutine $
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
When given a string, this subroutine returns a unique number representing a string, a cookie.Required settings
(none)Call arguments (variables to set before calling)
- INPUT, String from which the magic-cookie is calculated
- JA_COOKIE_CMD: shell command to read INPUT and return decimal or hex cookie string as one continuous block of characters. It decaults to HP-UX cksum, but your system may have md5 or chksum
Return values
- Variable OUTPUT will contain the cookie.
Example usage
INPUT = "foo@site.com" JA_COOKIE_CMD = "md5" # or chksum INCLUDERC = $PMSRC/pm-jacookie1.rc cookie = $OUTPUT
File id
Copyright (C) 1997-1999 Jari Aalto
Contactid: jari.aalto@poboxes.com
Created: 1997-12
Keywords: procmail cron framework recipe
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
Framework for all cron tasks that can be run once a day. This is a wrapper recipe to your cron task list: when the day changes, you cron includerc is called.Required settings
PMSRC must point to source directory of procmail code. This recipe will include
- pm-javar.rc
- pm-jadate.rc
Call arguments (variables to set before calling)
- JA_CRON_RUN_FLAG, You must define this flag file.
- JA_CRON_DATE_FILE, File where the date information, last cron run, is kept. Defaults to $HOME/.yymmdd
- JA_CRON_RC, your includerc which is run when cron triggers.
A file JA_CRON_RUN_FLAG which defaults to ~/.yymmdd.run is created when your includerc, that contains list of cron tasks, is run. If new mail arrives while your cron recipes are still running, you should prevent invoking the cron again by checking if this file exists. When all the cron tasks have been run, this flag file is removed. Remember to use "w" flag in your cron recipes where necessary to serialize the work.
Return values
(none)Usage example
Save backups to separate directory, but do cleaning only once a day We do not keep backups from mailing list messagesLISTS = "(procmail|list-1|list-2)" BACKUP_DIR = "$HOME/Mail/backup/."# Store backups: separate files to directory:0 c: *$ ! $LISTS $BACKUP_DIR# Run JA_CRON_RC once a day. It contains all daily cron tasksCRON_RC = $PMSRC/pm-jacron.rc # the framework JA_CRON_RC = $PMSRC/pm-mycron.rc # the tasks to do JA_CRON_RUN_FLAG = $HOME/.cron-running # define this!# Do not enter here if message arrived at the same day when # the cron is already running. The CRON_RC takes care # of deleting the file when cron has finished.:0 *$ ! ? $IS_EXIST $JA_CRON_RUN_FLAG { INCLUDERC = $CRON_RC }The pm-mycron.rc file may contain anything. For example to clean the backup directory; you add these statements there
# rm dummy: if ls doesn't return files, make sure rm has # at least one argument. # # ls -t: list files; newest first # # sed: chop $max newest files from the listing, leaving the # old onesmax = 32:0 hwic | cd $BACKUP_DIR && $RM -f dummy `ls -t msg.* | $SED -e 1,${max}d`# End of file pm-mycron.rc
File id
Copyright (C) 1997-98 Jari Aalto jari.aalto@poboxes.com
Created: 1997-11
Contactid: jari.aalto@poboxes.com
Keywords: procmail recipe daemon messages
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to Contactid with subject "send <FILENAME>"
Description
When you send a message to a address that had delivery troubles, you get a DAEMON message back explaining the error problem. I usually want to save these daemon mesaages to a different folder and check the folder from time to time. A typical daemon message is like this (shortened)From: Mail Delivery Subsystem MAILER-DAEMON@my.domain.com Subject: Warning: could not send message for past 4 hoursThe original message was received at... ----- Transcript of session follows ----- Deferred: Connection timed out ----- Original message follows ----- [YOUR MESSAGE AS YOU SENT IT WITH HEADERS]Well, when I read the subjects, I do not like the standard error messages, but I also like to know to which address the delivery failed and what was the original subject. This small recipe changes the daemon message's Subject to
Subject BRIEF-ERROR-REASON, SENT-TO-ADDRESS, ORIGINAL-SUBJECTand from that you can immediately tell if you should be worried Eg. if SENT-TO-ADDRESS was your friend's, then you want to take actions immediately, but if it were your complaint to UBE message to postmaster, you don't want to bother reading that daemon message. Here are some real examples:
fatal errors,postmaster,ABUSE (Was: Super Cool Site!) Host unknown,postmaster,ABUSE (Was: A-Credit Information) undeliverable,postmaster,Could you investigate this spam Warning-Returned,friend,Have you looked at thisRequired settings
PMSRC must point to source direcry of procmail code. This subroutine needs scrips
- pm-javar.rc
Call arguments (variables to set before calling)
- JA_DAEMON_REGEXP, which messages to trigger
Return values
- Variable ERROR will be set to "yes" if daemon message was handled otherwise; value is "no"
Usage example
Just add this recipe somewhere in your .procmailrc. The place where you would put this daemon message trapper subroutine is crucial: think carefylly how you order your recipes. One suggested order could be: backup important messages, cron-subroutine, handle duplicates, DAEMON MESSAGES, plus addressed message, server message (file server, ping responder...), MAILING LISTS, send possible vacation replies only after all above, apply kill file, detect mime, save private messages and las FILTER UBE.PMSRC = $HOME/pm RC_DAEMON = $PMSRC/pm-jadaemon.rc DAEMON_MBOX = $HOME/Mail/junk.daemon.mbox...INCLUDERC = $RC_DAEMON:0 : # If that was a daemon message, put it to folder * ERROR ?? yes $DAEMON_MBOX
File id
Copyright (C) 1997-98 Jari Aalto jari.aalto@poboxes.com
Contactid: jari.aalto@poboxes.com
Created: 1997-12
keywords: procmail dateFromMessage recipe
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
This recipe will scan several headers to find the date string. When suitable header is found and the parsing has succeeded, the return variables are set. The Date values reflects the arrive time of the message; not the sending time. If nothing works, a shell call date is used as a last resort.Returned values
YYYY = 4 digits YY = 2 digits MON = 3 characters MM = 2 digits DAY = 3 characters DD = 2 digits hh = 2 digits if available mm = 2 digits if available ss = 2 digits if availableRequired settings
PMSRC must point to source direcry of procmail code. This subroutine will include
- pm-javar.rc
- pm-jadate1.rc
- pm-jadate3.rc
- pm-jadate4.rc
Call arguments (variables to set before calling)
(none)Usage example
INCLUDERC = $PMSRC/pm-jadate.rc # now we have all date variables that we need # $TODAY = $YYYY-$MM-$DD
File id
Copyright (C) 1997-98 Jari Aalto jari.aalto@poboxes.com
Created: 1997-11
Contactid: jari.aalto@poboxes.com
Keywords: procmail subroutine date parsing
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
This includerc parses date from variable INPUT which has string"Week, Daynbr Month Year"Example input
"Tue, 31 Dec 1997" -- without comma "Tue 31 Dec 1997" -- with commaReturned values
YYYY = 4 digits YY = 2 digits MON = 3 characters MM = 2 digits DAY = 3 characters DD = 2 digits hh = 2 digits If available mm = 2 digits If available ss = 2 digits If available TZ = 5 characters If availableVariable ERROR is set to yes if it couldn't recognize the INPUT and couldn't parse the basic YYYY, YY, MM, DD variables.
Required settings
PMSRC must point to source direcry of procmail code. This subroutine will include pm-javar.rc from there.Call arguments (variables to set before calling)
INPUT = string-to-parseThe INPUT can have anything after "Week, dayNbr Month Year", or before it: you can pass a string like "Thu, 13 Nov 1997 11:43:23 +0200".
Usage example
The first Received header will tell when the message was received by your mailserver. We parse the date and avoid calling expensive date command.PMSRC = $HOME/pm RC_DATE_WDMY = $PMSRC/pm-jadate1.rc #Week-Day-Month-Year parser# Get time from first header, it ends like this: # # Received: ... ; Thu, 13 Nov 1997 11:43:50 +0200:0 *$ ^Received:.*;$s+\/...,$s+$d.* { INPUT = $MATCH# Turn off the logging while executing this partVERBOSE=off INCLUDERC = $RC_DATE_WDMY VERBOSE=on:0 * ERROR ?? yes { # Use some other way to get the time or shout loudly } }
File id
Copyright (C) 1997-98 Jari Aalto jari.aalto@poboxes.com
Created: 1997-11
Contactid: jari.aalto@poboxes.com
Keywords: procmail subroutine body-empty-test
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
This includerc parses date in format "YYYY-MM-DD hh:mm:ss" like 1997-12-01 and sets following variables whenever calledYYYY = 4 digits YY = 2 digits MON = 3 characters MM = 2 digits DD = 2 digits hh = 2 digits If avaliable mm = 2 digits If avaliable ss = 2 digits If avaliableVariable ERROR is set to yes if it couldn't recognize the INPUT and couldn't parse the basic YYYY, YY, MM, DD variables.
Required settings
PMSRC must point to source direcry of procmail code. This subroutine will include pm-javar.rc from there.Call arguments (variables to set before calling)
INPUT = string-to-parseLast string in INPUT that matches number sequence NNNN-NN-NN is parsed.
Usage example
PMSRC = $HOME/pm RC_DATE_ISO = $PMSRC/pm-jadate2.rc # ISO date parserINPUT = "This is 1800-10-11, a very old date"# Turn off the logging while executing this partVERBOSE="off" INCLUDERC=$RC_DATE_ISO VERBOSE="on"
File id
Copyright (C) 1997-98 Jari Aalto jari.aalto@poboxes.com
Created: 1997-11
Contactid: jari.aalto@poboxes.com
Keywords: procmail subroutine date parsing
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
This includerc parses date from variable INPUT which has string"Week, Month dayNbr hh:mm:ss yyyy",Example
Tue Nov 25 19:32:57 1997Returned values
YYYY = 4 digits YY = 2 digits MON = 3 characters MM = 2 digits DAY = 3 characters DD = 2 digits hh = 2 digits mm = 2 digits ss = 2 sigitsVariable ERROR is set to "yes" if it couldn't recognize the INPUT.
Required settings
PMSRC must point to source direcry of procmail code. This subroutine will include pm-javar.rc from there.Call arguments (variables to set before calling)
INPUT = string-to-parseUsage example
The first Received header will tell when the message was received by the mailserver. Parse the date and avoid calling expensive date command.PMSRC = $HOME/pm RC_DATE_WMDT = $PMSRC/pm-jadate4.rc #Week-Month-Day-Time parser# Get time from X-From-Line: Which was added by my MDA # X-From-Line: procmail-request@informatik.rwth-aachen.de \ # Tue Nov 25 19:32:57 1997:0 c *$ ^X-From-Line:\/.* { INPUT = $MATCH# Turn off the logging while executing subroutineVERBOSE=off INCLUDERC = $RC_DATE_WMDT VERBOSE=on:0 * ERROR ?? yes { # Use some other way to get the time or shout loudly } }
File id
Copyright (C) 1997-98 Jari Aalto jari.aalto@poboxes.com
Contactid: jari.aalto@poboxes.com
Created: 1997-12 $
keywords: procmail dateFromSehll subroutine
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
This subroutine calls shell command date once and prses the values. This should be your last resort if you haven't got the date values by any other means. This subroutine assumes that the DATE command knows the following % specifier formats (HP-UX)Y NNNN year h MON month d NN day a WEEK Like "Mon" H NN hour M NN min S NN secReturned values
DATE = RFC date in format "Mon, 1 Dec 1997 17:41:09" This is same as what you would see in From_YYYY = 4 digits YY = 2 digits MON = 3 characters MM = 2 digits DAY = 3 characters DD = 2 digits hh = 2 digits mm = 2 digits ss = 2 sigitsVariable ERROR is set to "yes" if values couldn't be set
Required settings
PMSRC must point to source directory of procmail code. This subroutine will include
- pm-javar.rc
- pm-jadate1.rc
Call arguments (variables to set before calling)
(none)Usage example
The First Received line will tell when the message was received by the MDA. If thata fails, then get date from the system. If you send test messages to # yourself, you don't usually put From_ header in it and thus there is # no date information in 'dry run' tests.# Get time from first eader, which is always same in my system # Received: ... ; Thu, 13 Nov 1997 11:43:50 +0200INCLUDERC = $PMSRC/pm-javar.rc # to get $s $d definitions TODAY # Clear it:0 *$ ^Received:.*;$s+\/...,$s+$d.* { INPUT = $MATCH INCLUDERC = $PMSRC/pm-jadate1.rc TODAY = "$YYYY-$MM-$DD" }# Check that variable did get set, if not then we have to call # another date subroutine: Call shell then to find out date # # You could also do this with ':0 E', but this is more # educational:0 *$ ! $TODAY^0 { INCLUDERC = $PMSRC/pm-jadate4.rc # Get date from Shell then TODAY = $YYYY-$MM-$DD }
File id
Copyright (C) 1997-98 Jari Aalto jari.aalto@poboxes.com
Created: 1998-06
Contactid: jari.aalto@poboxes.com
Keywords: procmail subroutine date parsing
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
This includerc parses date from variable INPUT which has string"WeekDay Month dayNbr Year"Example input
"Fri Jun 19 18:51:56 1998" -- without comma "Fri, Jun 19 18:51:56 1998" -- with commaReturned values
YYYY = 4 digits YY = 2 digits MON = 3 characters MM = 2 digits DAY = 3 characters DD = 2 digits hh = 2 digits If available mm = 2 digits If available ss = 2 digits If available TZ = 5 characters If availableVariable ERROR is set to "yes" if it couldn't recognize the INPUT and couldn't parse the basic YYYY,YY,MM,DD variables.
Required settings
PMSRC must point to source direcry of procmail code. This subroutine will include pm-javar.rc from there.Call arguments (variables to set before calling)
INPUT = string-to-parseThe INPUT can have anything after "Week, dayNbr Month Year", or before it: you can pass a string like "Fri Jun 19 18:51:56 1998 11:43:23 +0200".
Usage example
The first Received header will tell when the message was received by your mailserver. We parse the date and avoid calling expensive date command.PMSRC = $HOME/pm RC_DATE_WDMY = $PMSRC/pm-jadate5.rc #Week-Day-Month-Year parser# Get time from first header, it ends like this::0 *$ ()\/From .* { INPUT = $MATCH # Turn off the logging while executing this partVERBOSE=off INCLUDERC = $RC_DATE_WDMY VERBOSE=on:0 * ERROR ?? yes { # Use some other way to get the time or shout loudly } }
File id
Copyright (C) 1997-98 Jari Aalto
Contactid: jari.aalto@poboxes.com
Created: 1997-11
Keywords: procmail filter duplicates recipe
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
This recipe stores duplicate messages to separate folderRequired settings
PMSRC must point to source directory of procmail code. This subroutine will include
- pm-javar.rc
- pm-jastore.rc
Call arguments (variables to set before calling)
- JA_ID_CACHE, Where to keep the Message-Id cache.
- JA_ID_CACHE_SIZE, how big cache, defualt is 8192
- JA_ID_MBOX, where to store duplicate messages when delivering message to duplicate mbox.
- JA_ID_IGNORE, if set to "yes", then ignore duplicate check
Return values
- Variable ERROR is set to "yes" if duplicate message was trapped, otherwise value is "no"
Usage Example
For simple usage, just put this somewhere after backup recipesRC_DUP = $PMSRC/pm-jadup.rc ... INCLUDERC = $RC_DUPWhen you are testing messages, you send them over and over to procmailrc; which means that same message should not be trapped by
duplicate check. You can call procmail with option "-a test" which will set pseudo variable $1. The recipe below sets flag JA_ID_IGNORE to "yes" if test is on going and the duplicate filter should be bypassed.RC_DUP = $PMSRC/pm-jadup.rc ARG = $1 # Copy pseudo variable to $ARG:0 * ARG ?? test { JA_ID_IGNORE = "yes" }# Some microsoft product is known to send same message ids # over and over. If we detect one, tunr off the duplicate test, # because it would trash every message. # <MAPI.Id.0016.00666479202020203030303430303034@MAPI.to.RFC822>:0 * ! ^X-msmail * ! ^Message-ID: *<MAPI.*@MAPI.to.RFC822> { JA_ID_IGNORE = "yes" }# Run this command every time a duplicate message is found. # It writes a small log entry to MY_LOGINCLUDERC = $RC_DUP:0 hwic: * ERROR ?? yes | echo " [duplicate]" >> $BIFF
File id
Copyright (C) 1997-98 Jari Aalto
$Contactid: jari.aalto@poboxes.com $
$Created: 1997-11 $
$keywords: procmail filter duplicates recipe $
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
This simple includerc will define variable BODY_EMPTY to "yes" or "no" when called like thisINCLUDERC = $PMSRC/pm-jaempty.rcRequired settings
(none)
File id
Copyright (C) 1997-98 Jari Aalto $Contactid: jari.aalto@poboxes.com $ $Created: 1997-12 $ $keywords: procmail extracFROM subroutine $This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
This includerc extracts the most likely FROM address from the message. The order of the search is Reply-to, From_, Sender, From and if none found, then as a last resort, call formail. You would usually use the returned value for logging purposes.Avoiding extra formail call could be usefull if you receive lot of messages per day.
Example input
(none)Returned values
OUTPUT, containing the derived FROM fieldRequired settings
PMSRC must point to source direcry of procmail code. This subroutine will include pm-javar.rc from there. You nee procmail 3.11pre7 in order to use this subroutine. (due to formail -z switch)Call arguments (variables to set before calling)
(none)Usage example
INCLUDERC = $PMSRC/pm-jafrom.rc FROM = $OUTPUT # now we have the 'best' FROM field
File id
Copyright (C) 1997 Jari Aalto jari.aalto@poboxes.com
Created: 1997-10
Maintainer: Jari Aalto jari.aalto@poboxes.com
keywords: procmail forward mail
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Overview of features
- Requires procmail 3.11pre7+ (formail -z switch)
- You can send forward-on and forward-off control messages via email to control the forwarding in remote site.
Description
This includerc makes it possible to control your message forwarding via simple remote email message. Thanks to Era Eriksson and Timothy J Luoma who gave the initial idea to this forwarding module in the procmail mailing list 1997-10-07.Activating the forwarding by hand
If you want to activate the forwarding from the local site where this module is, then you could simply write the forward address to the file pointed by JA_FWD_FILE which is ~/.forward-address by default.% echo Me@somewhere.com > ~/.forward-addressand when you no longer need forwarding, then remove that file. But really, this module is not used for that purpose, because it is lot easier to write
:0 ! Me@somewhere.comas a first statement in your .procmailrc when you want to forward your mail to another account.
Activating the forwarding by remote email
Suppose you're on the road and suddenly realise that you want your mail forwarded to the current account, then you send following control messageSubject: forward-on password new-address@bar.com To: my-account@bar.com From: onTheRoad@some.comThat message is is enough to get the mail forwarded to the address new-address@bar.com This script will repond to address From that the current forwarding is now pointing to address "new-address@bar.com".
Deactivating forwarding by remote email
The message is very similar, but the Subject header saysSubject: forward-off passwordAnd no other fields are checked. Not even Reply-To. In this case the confirmation message is sent directly back to From address.
Activating forwarding via body message
If for some reason you have no control over the headers of email, eg when you send GSM-Mail message from your phone to your account:EMAIL foo@bar.com FORWARD-ON PASSWORD new-address@bar.comThe email message looks like this:
From: GenEmail sms@FooBar.net Date: Thu Sep 17, 11:42am +0200 To: "'Foo.Bar'" foo@bar.com Subject: Message 03384874987FORWARD-ON PASSWORD new-address@bar.comInstead of looking at the Subject field, you can get this module to look at the first words in the body field. See variable JA_FWD_CONTROL_FIELD which you want to set to "body".
Restricting the control message aceptance
If you only have persistent accounts, then you should set the JA_FWD_FROM_MUST_MATCH to match those addresses that you have. The following setting says that only control messages sent from these addresses are accepted. Nobody else can't change your forwarding settings.JA_FWD_FROM_MUST_MATCH = ".*(acc1@a.com|acc2@b.com)"Hm, that's not a bullet proof, because someone may in theory forge the From address. You propably should also set this variable to point to accounts where the mail can be legally forwarded to. Then, even if the imposter forges the From address; he can't get the email forwarded anywhere else than to the valid locations.
JA_FWD_TO_MUST_MATCH = $JA_FWD_FROM_MUST_MATCHConsider also setting JA_FWD_PASSWORD_CASE to Procmail flag D which causes your control word "forward-on" and password to be case sensitive.
Diagnostics
If you don't receive confirmation message, then your control message was ill formed or you're not in the JA_FWD_FROM_MUST_MATCH list. There is no notification sent on failure, so that no attacker can draw conclusions.Required settings
PMSRC must point to source direcry of procmail code. This subroutine will include
- pm-javar.rc
Installation
You should preset all necessary variables prior adding the includerc command to your .procmailrc. Here is one simple setup#JA_FWD_SENDMAIL = "tee $HOME/test.mail" # Uncomment if testing JA_FWD_COPY = no # no copies stored while forwarding JA_FWD_PASSWORD_CASE= "D" # case sensitive JA_FWD_PASSWORD = "MyMagicString" JA_FWD_FROM = $FROM # This is already known. INCLUDERC = $PMSRC/pm-jafwd.rcComments from the author
Please realise that when you set the forwarding from a remote site, be very carefull when you type in the forward address or your mail ends up to somebody else's mailbox. Also I recommend that you keep JA_FWD_COPY to yes so that your local account always keep the copy of forwarded message.A step furher would conventionally encrypt(1)'ing your forwarded messages. This way even your top secret messaes would be mostly safe even if they end up to someone else's mailbox.
File layout
tinybm.el/&tags and tinytab.el for the 4 tab text placement. See also unix what(1) and GNUS RCS ident(1).
File id
Copyright (C) 1998 Jari aalto jari.aalto@poboxes.com
Contactid: jari.aalto@poboxes.com
Created: 1998-06
Keywords: procmail subroutine list detect
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
This subroutine tries to detect and derive the mailing list name as it appears in some of the know methods that ezlm, smarlist, listserv, majordomo etc normally use. After this subroutine has been applied to message the variable LIST contains the mailing list name. Subroutine adaptively finds new new mailing lists from the messages.Quick start
If you just want to jump in and use this module and you see that some list isn't trapped, please set
- JA_LIST_HEADER_REGEXP to match the From: field site regexp.
If you want to make some list more unique, like if name "Alert" was detected as list name, please set
- JA_LIST_MAKE_UNIQUE to match the list name, like "Alert" and the lista name will be converted to HOST-LIST format.
Sendmail plus method for list subscription
If you can use sendmail PLUS addressing capabillities, you may not be interested in this module, because you have an alternative way to handle mailing list messages. Let's suppose you want to subscribe to procmail maling list and want to save all messages to folser list.procmail, then you'd subribe with address:login+list.procmail@site.comThe extra information after "+" is available to your procmail scripts via $ARG pseudo variable when procmail is the LDA. If you fortunate to have new sendmail, you usually subscribe to mailing lists with regular email address:
login@site.comHow do you detect the arriving mailing list messages? Traditionally, you would add a piece of recipe to .procmailrc to catch each list, but that's manual work every time. When you use this subroutine, you no longer need to write separate mailing list recipes to your .procmailrc every time you subscribe to a new mailing list. The detection of a new list happens in this subroutine for you.
What you need to know before using this module
There is lot of heuristics going on in this modules and one thing that you must do, if you're a member of tech support or if you get cron messages from your server. The rule is:If TO domain is same as FROM/SENDER/REPLY-TO domain then it is considered a mailing list message.This causes certain messages landing to category LIST automatically. This module can't possibly know that the following is not from mailing list, because it doesn't know "what is mailing list", only "how it probably looks like it". This is definitedly categorized as maling list message, because From and even Reply-to has same domain foo.bar.net as in To.
To: support@foo.bar.net From: messagepad@foo.bar.net Reply-to: support@foo.bar.net Subject: Vmail See message to EricYou must prevent checking messages like this by surrounding the RC with if statement:
# Do not check these messagesnoList = "From.*(foo.bar.net|support.my.com)":0 *$ ! $noList { INCLLUDERC = $RC_LIST }Ask for help
If you find maling lists that this subroutine does not detect, but which could have been detected by looking the headers in standard way, please send a email to maintainer. There may be cases where it is impossible to detect the mailing list and in those cases you just has to carve a new entry to your procmailrc.When you keep your procmail log running, you may see message
*** potential list ***Which is an indication that some new recipe could be added to to this subroutine to detect that mailing list. If the message you received WAS from a mailing list, please send all the headers to the maintainer so that support can be added.
You can search for mailing list that interests you at:
http://www.lsoft.com/lists/listref.html http://www.netmeg.net/faq/internet/mail/mailing-lists/Code note: Errors-To
Bill Houle sent me interesting headers which caused me to add more heuristical approach that I would have originally wanted. From these headers there really is impossible to derive the original list name. So, I tossed my own and derived the name by combining Reply-To's LOGIN with Errors-To fields first server nameReply-To: news@doodle.foo.net Errors-To: bounced@doodle.foo.netThe list name formed was "news-doodle". So, If you happen to see an odd name like this which doesn't remind your original list name, it may be due to poor headers that have no clue about the real name. No problem, check below how you would convert this name to better mailbox name.
Required settings
PMSRC must point to source direcry of procmail code. This subroutine will include pm-javar.rc from there.
- pm-javar.rc is needed and must reside along $PMSRC
Variable JA_LIST_KILL_POSTFIX
If grabbed LIST match this regexp at the end of list name, then the postfix match will be removed. It is traditional that many lists name themself as list1-info, list2-beta, list3-l and you would prefer more names (for mbox) list1, list2 and list3. The default value will ditch "-(info|beta|l)".Variable JA_LIST_KILL_PREFIX
Just like the postfix variable. If this string is matched at the beginning of the LIST, it is removed.Variable JA_LIST_HEADER_REGEXP
This is optional variable, which you can set to match regexp of the mailing list domain address if it slipped through the tests in this module. There are some lists that send messages that don't carry enough information in headers to determine their list status. If you narrow the group by setting JA_LIST_HEADER_REGEXP, then for example lists like these, that identify themselves only through two headers, can be found:Reply-To: dispatch-faq@cnet.com From: CNET Digital Dispatch dispatch@cnet.comFor that list you would set
JA_LIST_HEADER_REGEXP = "(@cnet\.com)"Don't worry. all the other list detection recipes has already been tried, so this is last test that are carried out and variable JA_LIST_HEADER_REGEXP helps eliminating possible mishist
You don't need set this variable to include all mailing list domains. Only to those ones that were not trapped. The default value for this is:
"(amazon\.com|bookpool\.com)"Variable JA_LIST_MAKE_UNIQUE
If you're subscribed to many mailing lists, that simply tell that they are news or newsletter, it will be impossible to differiantiate A news from B news. This varaible holds regular expression that, if matched, prepend the first hostname to the beginning of listname, thus making the list unique:news@some.com --> some-news news@here.com --> here-newsThe default value matches lists that are contain word news, but you may need to set this to more matches.
Variable JA_LIST_CONVERSION
Many times the grabbed LIST name is not what you would like to use for your mailbox name. You want to make the name perhaps more shorter, more descriptive or categorize the messages according to hierarchy. Let's say that you have subscribed to following mailing lists:LIST LIST name Description of mailing list (as grabbed) you wantjde java.jde Java Development Env java java.prog Java programming FLAMENCO flamenco Flamenco music tango-l tango Argentine Tango dancing tm-en-help tm-en Emacs TM mime package mailing list w3-beta w3 Emacs WWW mailing listFirst, remember that the variable JA_LIST_KILL_POSTFIX is applied, so the actual LIST appear as follows:
jde, java, FLAMENCO, tango, tm-en, w3Ok, Now we apply the conversion table by defining it as follows, where the grabbed LIST is first, then comes space(s), new name and terminating colon. Repeat this for each list you want to convert.
LIST CONVERSION,LIST CONVERSION,This gives us table below: Notice that antries tango-l, w3-beta were not included, because the JA_LIST_KILL_POSTFIX already got rid of the posfixes. Also note how the uppercase match FLAMENCO is converted to more suitable lowercase mailbox name. After you have set up this variable you can start saving messages to folders.
JA_LIST_CONVERSION = "\ jde java.jde,\ java java.prog,\ FLAMENCO flamenco,\ "The list conversion is done with pure procmail means, so it is very fast. It also means that the conversion is limited to FROM-STRING TO-STRING syntax. No wildcards or regular expressions are allowed.
If you consider using an external process, like sed or perl to convert the grabbed list name to something else (when JA_LIST_CONVERSION method was not enough); think again. For each incoming mailing list message you launch external process. I get 700 messages from various mailing lists a day so you can imagine how much load any external process would cause. Just use the grabbed mailing list name and JA_LIST_CONVERSION table if you care about system load.
If you have many mailing lists that use uppercase names, it may be tedious to add each mailing list name to JA_LIST_CONVERSION. Possible alternative is to add conversion recipe: tr is most efficient here to convert name to lowercase. Again; think twice, extra process could be avoided if you use JA_LIST_CONVERSION.
:0 * ! LIST ?? ^^^^ { :0 D # still uppercase list name? * LIST ?? [A-Z] { LIST = `echo $LIST | tr A-Z a-z` }:0 : list.$LIST }List name is not always the same
One important thing to keep in mind is that when mailing list manager sends out list messages, the headers may some times change. This means that the list name grabbed by previous calls changes, although the list in practise has not changed at all.This is unfortunate, but it sometimes happens. Let's see an example. I was previously receiving messages from Cygwin mailing list named gnu-win32
To: <gnu-win32@cygnus.com>, "Foo Bar" foo@example.comHowever, one say that same list was grabbed under name "cygwin", due to new header
Mailing-List: contact cygwin-help@sourceware.cygnus.com; run by ezmlmNow I had two list names that both shuld be going to the same mailbox. No worries, just use the translate table to convert the list name to mailbox name:
JA_LIST_CONVERSION = "\ gnu-win32 cygwin32,\ cygwin cygwin32,\ "Example: basic installation
Here is recipe to save all your mailing list to separate folders. If you subsribe to new lists or unsubsribe to lists, you don't need to change anything.RC_LIST = $PMSRC/pm-jalist.rc # name the subroutine...# Handle all mailing lists with one subroutine and recipe # following itINCLUDERC = $RC_LIST:0 # if list name was grabbed * LIST ?? [a-z] { dummy = "Saving mailing list: $LIST":0 : list.$LIST }
File id
Copyright (C) 1998 Jari Aalto
$Maintainer: Jari Aalto jari.aalto@poboxes.com $
$Created: 1998-01 $
$Keywords: procmail recipe $
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Documentation
The original father of the decoding scheme used here was presented by Peter Galbraith galbraith@mixing.qc.dfo.ca in procmail mailing list somewhere at the end of 1997.This includerc supposes that the header has MIME header Content-Type: text/plain and performs qp or base64 decoding on the whole message. Note, that if you receive messages that have many mime attachements, then this recipe is not suitable for it.
Procmail is not designed to handle mime attachements and this recipe only applies to whole body.
Required settings
PMSRC must point to source direcry of procmail code. This subroutine will include
- pm-javar.rc
Call arguments (variables to set before calling)
(none)Return values
(none)Examples
Instead of testing the existense of text/plain in the body, you can force decoding by settings JA_MIME_DECODE_REGEXP to ".*".RC_MIME_DECODE = $PMSRC/pm-jamime-decode.rc:0 * condition { JA_MIME_DECODE_REGEXP = ".*" }INCLUDERC = $RC_MIME_DECODE # call subroutine.
File id
Copyright (C) 1997-98 Jari Aalto
Contactid: jari.aalto@poboxes.com
Created: 1997-12
Keywords: procmail MIME killer recipe
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
Microsoft Explorer has a bad habbit of including 7k application/ms-tnef attachement to the end of message. The sample message is described in next section. If you don't want to see that additional ms-tnef attachement, then plug in this module and you no longer see MS Explorer cruft.This recipe works like this: If email's structure is
--boundary message-text (maybe quoted-printable) --boundary some-unwanted-mime-attachement --boundarythen the attachement is killed from the body. The message-text part is also decoded if it was quoted printable. This leaves clean text; with no MIME anywhere. MIME headers have been modified as needed (due to conversion from multipart and possibly quoted printable to plain text):
messageBut if email's structure was anything else, like if there were 3 mime sections:
--boundary message-text (maybe quoted-printable) --boundary some-attachement --boundary some-unwanted-mime-attachement --boundarythen the "unwanted" part is emptyed by replacing area with one empty line. The message structure stays the same, but the killed "unwanted" part is labelled as text/plain so that your MUA will decode the MIME message correctly.
Applications for other mime attachements
All thse are covered when plug in the module.
- Lotus Notes sends similar extra attachement and you can use this same recipe to kill it. See example section.
- Microsoft Express sends a copy of message in html format in the attachement, you can kill that too, see example section.
- Netscape's Mozilla sends a copy of message in html. See example. Also it sends vcards
- Openmail sends 10-20 base64 attachements WINMAIL.DAT.
Example of lotus notes attachement
Subject: message From: foo@bar.com X-Lotus-FromDomain: XXX COMPANIES Mime-Version: 1.0 Boundary="0__=cieg4oHxUNf2h3evyOXIsHTGDpFfaZilTDCFhpZSgsw" Content-Type: multipart/mixed; Boundary="0__=cieg4oHxUNf2h3evyOXIsHTGDpFfaZilTDCFhpZSgsw--0__=cieg4oHxUNf2h3evyOXIsHTGDpFfaZilTDCFhpZSgsw Content-type: application/octet-stream; name="PIC10898.PCX" Content-transfer-encoding: base64eJ8+IjsQAQaQCAAEAAAAAAABAAEAAQeQBgAIAAAA5AQAAAAAAADoAAEIgAcA b3NvZnQgTWFpbC5Ob3RlADEIAQ2ABAACAAAAAgACAAEEkAYAyAEAAAEAAAAQ<AND-THE-REST-OF-BASE64>--0__=cieg4oHxUNf2h3evyOXIsHTGDpFfaZilTDCFhpZSgsw--Example of MS Explorer's ms-tnef message
Subject: message From: foo@bar.com MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="---- =_NextPart_000_01BD04D4.A5AC6B00" Lines: 158------ =_NextPart_000_01BD04D4.A5AC6B00 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable<MESSAGE ITSELF IS HERE>------ =_NextPart_000_01BD04D4.A5AC6B00 Content-Type: application/ms-tnef Content-Transfer-Encoding: base64eJ8+IjsQAQaQCAAEAAAAAAABAAEAAQeQBgAIAAAA5AQAAAAAAADoAAEIgAcA b3NvZnQgTWFpbC5Ob3RlADEIAQ2ABAACAAAAAgACAAEEkAYAyAEAAAEAAAAQ<AND-THE-REST-OF-BASE64>------ =_NextPart_000_01BD04D4.A5AC6B00--Example of MS Express's html message
MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_003A_01BD16E2.C97E27B0" X-Mailer: Microsoft Outlook Express 4.72.2106.4 X-MimeOLE: Produced By Microsoft MimeOLE V4.72.2106.4This is a multi-part message in MIME format.------=_NextPart_000_003A_01BD16E2.C97E27B0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable<ACTUAL TEXT>------=_NextPart_000_003A_01BD16E2.C97E27B0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable<SAME IN HTML> ------=_NextPart_000_003A_01BD16E2.C97E27B0--Example of Netscape's html attachement
X-Mailer: Mozilla 4.04 [en] (X11; U; Linux 2.0.33 i686) MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="------------69D9D579CF587DC8BB26C49C"--------------69D9D579CF587DC8BB26C49C Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit<ACTUAL TEXT>--------------69D9D579CF587DC8BB26C49C Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit<SAME IN HTML> --------------69D9D579CF587DC8BB26C49C--Example of Netscape's vcard attachement.
Content-Type: text/x-vcard; charset=us-ascii; name="vcard.vcf" Content-Transfer-Encoding: 7bit Content-Description: Card for Laird Nelson Content-Disposition: attachment; filename="vcard.vcf"begin: vcard fn: Laird Nelson n: Nelson;Laird org: Perot Systems Corporation email;internet: ljnelson@unix.amherst.edu title: Software Engineer tel;work: (617) 303-5059 tel;fax: (617) 303-5293 tel;home: (978) 741-3126 note;quoted-printable:Information is for reference only;=0D=0A= please do not abuse it. x-mozilla-cpt: ;0 x-mozilla-html: TRUE version: 2.1 end: vcardRequired settings
PMSRC must point to source direcry of procmail code. This subroutine will include
- pm-javar.rc
- pm-jamime.rc
Call arguments (variables to set before calling)
First of all, this is primarily a framework recipe to kill any kind of attachement. If you do not set JA_MIME_TYPE before calling this recipe, recipe will try to determine the right value by itself. See if the automatic detection works for you or preset the JA_MIME_TYPE beforehand if it fails (I'd like to hear about the failure and add the conditions as default)
- JA_MIME_TYPE is an case sensitive AWK REGEXP. Always use lowercase letters in this regexp because the line is lowercased before match is made. This regexp determines if the kill recipe is applied to the message or not. Value is set for MS explorer, MS express, Netscape and Lotus Notes etc. messages by default.
- JA_MIME_KILL_RE, additional REGEXP to kill lines from the message. Value is case sensitive awk regexp and by default matches Lotus notes tag: name="XXX.PCX".
It may be possible that some messages are malformed and that they do not contain preper "boundary" definition string in the header. I have seen messages that have text/html attachements, but no proper Mime headers. For those cases there is additional variable that will kill all text up till matching line regardless of message content.
- JA_MIME_KILL2_RE = "text/html|application/ms-tnef" Update this to match attchements you receive. Set variable to "" if you don't want to change the body of non-compliant MIME message.
That is your last resort if the standard MIME detection failed. (there must have been some problem in the sender's MUA that composed message)
Possible conflict with your awk
If you see an error message in the logfile saying that awk failed:procmail: Executing "awk, ... procmail: Error while writing to "awk" procmail: Rescue of unfiltered data succeededit means that the system's standard awk doesn't support the variable passing syntax: do following test
% awk '{print VAR; exit}' VAR="value" /etc/passwdAnd it should print "value". If not, then see if you have nawk or gawk is your system, which should understand the variable passing syntax. In HP 9 and 10 the standard awk will do fine. The only change you need is
AWK = "nawk" # if that works better than standard "awk"Somewhere at the top of your .procmailrc.
Warnings
You should know that the variable JA_MIME_KILL_RE is used to wipe any lines that match that regexp. This is due to MIME structure where continuing header lines exist in the body:------=_NextPart_000_003A_01BD16E2.C97E27B0 Content-Type: text/plain; charset="iso-8859-1" << kill this line tooIf you want to be absolutely sure that anything valuable won't be accidentally killed (like a code line in programming language scripts), you should set this variable to nonsense value that newer matches:
JA_MIME_KILL_RE = "match_it_never_I_hope"Usage example: Customizing the attachement killing
Suppose you receive new application/ms type attachement that the default settings doen't cover. This is a new mime type and you have to instruct this module to kill it. You do this:myCustomMimeType = "application/ms" # must be all lowercase:0 *$ $myCustomMimeType { PM_JA_MIME_TYPE = $myCustomMimeType }INCLUDERC = $PMSRC/pm-jamime-kill.rcIf you recive any other new types, just add similar recipes above INCLUDERC call.
Usage example
For default mime types mentined earlier, you just plug-in this module:INCLUDERC = $PMSRC/pm-jamime-kill.rcIf you want to do all by yourself, you can add all this to your ~/.procmailrc. This demonstrates how the correct type is detected
# ..................................................... # 1) Uncomment following line if your standard "awk" is broken# AWK = "gawk"# ..................................................... # 2) Next set correct value for attachement killing# Kill Lotus notes .pcx attachements [already included]:0 * ^X-Lotus-FromDomain: { JA_MIME_TYPE = "application/octet-stream" }# Kill all html attachements:0 B * ^Content-Type:.*text/html { JA_MIME_TYPE = "text/html" }# ..................................................... # 3) And now handle the message # by default kill MS Explorer's ms-tnef attachement.INCLUDERC = $PMSRC/pm-jamime-kill.rc
File id
Copyright (C) 1998-1999 Jari Aalto
Maintainer: Jari Aalto jari.aalto@poboxes.com
Created: 1998-01
Keywords: procmail recipe
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Documentation
This module saves one simple file attachement (MIME) from he message. The message must define following MIME headers. If "filename=" does not exists, then the message will not be handled.Mime-Version: <version> Content-Type: <type> Content-Disposition: attachment; filename="file.txt"Procmail is not very suitable for saving MIME attachements and you should not think that this the right tool for you. If you receive anything more than 1 attachement, this recipe does nothing, because that's out of our league and you need some more heavy weight mime tools. Eg. Perl CPAN has MIME libraries.
Note: When the attachement is in the body, it is simply written to a disk and the place is replaced with message
Extracted to file:/users/jaalto/junk/file.txt.1998-03-30The existing mime headers that surrounded the attachement are untouched, so don't try to press your Mail Agents MIME buttons at that point any more. 8There is no such file in that spot if you set JA_MIME_SAVE_DEL to yes).
Required settings
PMSRC must point to source direcry of procmail code. This subroutine will include
- pm-javar.rc
Call arguments (variables to set before calling)
- JA_MIME_SAVE_DIR, point this to directory where you want to store attachements.
- JA_MIME_SAVE_DECODE, set this to "yes", if you want that attachement is decoded before written to disk. This usually opens quoted printable or base64 encoding.
- JA_MIME_SAVE_DEL, set this to "yes", if you want to remove the attachement from the body of the message after it has been filed. Be vary carefull if you use this option. If you keep backup cache of incoming mail, then you might try "yes".
- JA_MIME_SAVE_OVERWRITE, set this to "yes" if it's okay to overwrite to an existing filename found from attachement. If you get periodic attachements always with same name, then you would want to set this to yes.
Core dump note
Because procmail uses LINEBUF when filtering messages, a core dump may happen if the attachement beeing filtered is bigger than the LINEBUF. The current setting accepts 524K attachements, buf if you expect to get bigger than that, you want to increase JA_MIME_SAVE_LINEBUF.Possible conflict with your awk
Awk is used because it is much more system load friendly than perl. If you see an error message in the logfile saying that awk failed:procmail: Executing "awk, ... procmail: Error while writing to "awk" procmail: Rescue of unfiltered data succeededit means that the system's standard awk doesn't support the variable passing syntax: do following test
% awk '{print VAR; exit}' VAR="value" /etc/passwdAnd it should print "value". If not, then see if you have nawk or gawk is your system, which should understand the variable passing syntax. In HP 9 and 10 the standard awk will do fine. The only change you need is
AWK = "nawk" # if that works better than standard "awk"Somewhere at the top of your .procmailrc
Return values (none)
File id
Copyright (C) 1997-1999 Jari Aalto
Maintainer: Jari Aalto jari.aalto@poboxes.com
Created: 1997-12
Keywords: procmail MIME subroutine
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Documentation
This includerc reads MIME boundary string from the message if it exists. The boudary string is typically found from Content-Type header.Mime-Version: 1.0 Content-Type: multipart/mixed; boundary=9i9nmIyA2yEADZbWIn addition it will define few other mime variables. See the returned values. You use these variables later in your MIME message processing.
Mime Notes
1998-07-28 Brett Glass brett@lariat.org reported in PM-L that there was security exploit in long attachement filenames: http://www.xray.mpe.mpg.de/mailing-lists/procmail/1998-07/msg00248.htmlAnd here is the url to the matter:
http://www.sjmercury.com/business/microsoft/docs/security0728.htm
When you use this module to detect mime messages, you can check the filename length with recipe:
# Recipe after calling $RC_MIME, this module,re = ".........." # regexp with 10 matches too_long = "$re$re$re$re" # allow 40 characters maximum:0 *$ $SUPREME^0 MIME_H_ATTACHEMENT ?? $re *$ $SUPREME^0 MIME_B_ATTACHEMENT ?? $re { dummy = "** Dangerously long mime attachement filename" dummy = "** $MIME_H_ATTACHEMENT $MIME_B_ATTACHEMENT":0 : /var/spool/mail/MimeDanger }Required settings
PMSRC must point to source directory of procmail code. This subroutine will include
- pm-javar.rc
Call arguments (variables to set before calling)
(none)Return values
- Variable MIME is set to "yes" or "no" if messages has mime version string
- MIME_VER contains the mime version string from the header.
- MIME_TYPE contains the Content-Type from the header.
- MIME_CTE contains Content-Transfer-Encoding from the header.
- MIME_H_QP is "yes" if Content-Transfer-Encoding: quoted-printable is in the header.
- MIME_B_QP is "yes" if Content-Transfer-Encoding: quoted-printable is found from the body.
- MIME_BOUNDARY contains the boundary string, which is used to differentiate mime sections in the body.
- MIME_BOUNDARY_COUNT is the number of boundary strings found from the body. The value is 3 if there is two mime sections, and 4 if 3 etc. MIME_BOUNDARY_COUNT -1 = count of sections.
- MIME_H_ATTACHEMENT, contains the filename if there was attachement filename in the header. Content-Disposition: attachment; filename="..."
- MIME_B_ATTACHEMENT. body file attachement. Note however that this is the match of first string in the body. There may be several attachements. MIME_B_ATTACHEMENT_FILE_COUNT tells you how many filenames are in the body.
Usage example
INCLUDERC = $PMSRC/pm-jamime-tag.rc
File id
Copyright (C) 1998 Jari Aalto
Contactid: jari.aalto@poboxes.com
Created: 1998-01
Keywords: procmail netmind recipe
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
http://minder.netmind.com/...Netmind, or The URL-minder is a free, automatic Web-surfing robot that keeps track of changes to Web pages that are important to you. When the URL-minder detects changes in any of the Web pages you have registered, it sends you e-mail. an efective way to test if the address is known to internet. You could usethis information to see if some automated reply to a address can be sent.
In another words, if your're interested in some URL; say an FAQ page and any updates to them, you can tell Netmind to monitor the page changes for you and it send a message back every time page changes.
This recipe "pretty formats" the announcement sent by Netmind by stripping the message to bare minimum. You usually aren't interested in 4k message which includes "Note from our sponsors", "Try the free online demo" etc. The things saved from the announcement message are:
- The changed url, which is moved to subject
- Cancellation url pointer
- url to the lists of your monitored urls
- your id number
[Note]
Please let Netmind send you one "pure" message first so that you have a huch what it originally looks like. Then plug int his module and see how the original message is reduced.
[Thank you]
The Doctor What docwhat@holtje-christian-isdn.mis.tandem.com 1998-03-12 send me a patch, where a)body message is more informative b) Url is now included in the body for auto-click browsers c) mime headers were removed.
Required settings
PMSRC must point to source direcry of procmail code. This subroutine will include
- pm-javar.rc
- If you se variable JA_NETMIND_SUBJECT to "yes", then the changed url http pointer is put to subject line.
Usage example
INCLUDERC = $PMSRC/pm-janetmind.rc # reformat the message:0: * netmind # drop to folder url.mbox
File id
Copyright (C) 1997-98 Jari Aalto
Contactid: jari.aalto@poboxes.com
Created: 1997-11
Keywords: procmail nslookup subroutine
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
This subroutine runs nslookup on given INPUT address. This may be an efective way to test if the address is known to internet. You could use this information to determine if some automated reply to a address can be sent. The know truth is that you can't validate whole email addressto_someone@foo.combut you can validate "foo.com"; that's the closest you get.
[Warning: If you don't use cache feature...]
Do not however use this module to regularly check all incoming from addresses with this subroutine for possible bogus UBE addresses, because calling nslookup
- may be slow, building to connection and querying the results may take several seconds. (some times, usually it's quote fast)
- consumes quite a lot resources.
You can however check some messages that are likely UBE to verify your doubts.
Required settings
PMSRC must point to source direcry of procmail code. This subroutine will include
- pm-javar.rc
- pm-jaaddr.rc
Call arguments (variables to set before calling)
- INPUT, the address (only strict domain part) which is checked. Eg. "this.domain.com". See examples for more. If this string contains "@" character, then additional subroutine pm-jaaddr.rc is called to extraxt the domain name INPUT = "John Doe foo@site.com" --> INPUT = "site.com"
- JA_NSLOOKUP_CACHE, filename. If exists, cache is used and updated.
- JA_NSLOOKUP_FORCE, if "yes", then cache is not used but a forced nslookup is performed.
- JA_NSLOOKUP_OPT, is currently empty, but you could see if you you want to use "-querytype=MX". However this option may give you response: "No mail exchanger (MX) records available", which is flagged as nslookup failure.
- JA_NSLOOKUP_SERVER, optional, the server to user for nslookup
If the cache file can be read:
- Each entry has format "address.com ns-error". The error indication is added to the line if the nslookup failed when address was checked. Otherwise line contains "address.com".
- The cache is always checked first. If there is no entry matching the current address, only then is nslookup called and new entry added to cache.
Return values
- Variable ERROR will be set to "yes" if nslookup failed or to "no" if nslookup succeeded. It can also contain "maybe" if nslookup returned "No address (A) records available"
- ERROR_MATCH contains one line lookup failure reason.
Following conditions trigger "maybe" and no "ns-error" is written into the cache.
- "No address (A) records available for xxx"
Usage example
If you are going to check some header field, like From:, please explode the content with pm-jaaddr.rc first. Suppose you have string:"From: foo@ingrid.sps.mot.com (Yoshiaki foo)"You have to derive the address from string and pass the site name: Read From: field and address from it.
PMSRC = $HOME/pm RC_NSLOOKUP = $PMSRC/pm-janslookup.rc # name the subroutine:0 * MAYBE_UBE ?? yes * ^From:\/.* { INPUT = $MATCH INCLUDERC = $RC_NSLOOKUP # to nslookup:0 * ERROR ?? yes { # Hmm, nslookup failed, can't send anything back to this # address } }Second example, check if the address is reachable before sending reply
INPUT = `$FORMAIL -rt -x To:` INCLUDERC = $RC_NSLOOKUP:0 * ERROR ?? no { # okay, at least site address seems to be reachable }
File id
Copyright (C) 1998 Jari Aalto jari.aalto@poboxes.com
Created: 1998-06
Maintainer: Jari Aalto jari.aalto@poboxes.com
Keywords: procmail recipe extract original message
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Documentation
This subroutine digs embedded message from the body and replaces current message with it. Copy the message to folder before calling this subroutine if you need original.NOTE: This is simple tool and the sole purpose is to derive simple embedded messages. Write full fledged perl script if you want better extracting features. The used AWK inside this procmail recipe will fail to find 30% of the cases, mostly duo to non-standard way of including the message. The recognized formats are as follows. Anything that differs from these are ignored or incorrectly parsed.
- Message is embedded left flushed "as is". With full headers or Minimum of From: Subject Received
- The embedded message is quoted with > with optional one space.
Where you would use this module
If you're subsribed to mailing lists that regularly sent copies of original message to the list, like forwarding spam to SPAM-L mailing list at http://bounce.to/dmuth, then you'd like to extract the original embedded message which you can then feed to your UBE filter to test if the shield holds.spam-l-request@peach.ease.lsoft.com subscribe SPAM-L <First name> <Last name>This recipe takes simplistic approach and tries it's best to extract embedded message. Idea for this recipe comes from Era Eriksson's posting "recipe to turn list postings back into original spam" 1998-06-25 in Procmail mailing list.
- Body must contain headers
- Remove all > quotations.
- extract everything to the end of message. (There are no means to get rid of the attached signatures that ot forwarding poster or list server may have attached.
How the message is extracted
When this recipe ends, the current message has been modified so that it is the original message. Like if you would receive:HEADER-1 # The poster body-1 # his comments HEADER-2 # The original embedded message body-2 body-1 # And poster's signature or mailing list footerThe message now looks like
HEADER-2 body-2 body-1And you can save this as original message or feed it to your UBE filter and test if it detects it.
Code note: procmail or awk core dump
For some reason procmil kept dumpig core I write the code in more nicer format like below, but if I made it compact, then it didn't dymp core. Go figure. I'm not pleased that I had to sacrafice clarity, but there was no other way.[The good style] [The forced compact style] if () if () { statement } { statement }I have no explanation why this happens, the same AWK code would work just fine most of the cases and then came this message x and caused dumping the code, if I feed some other messsage, I dind't get core dump. Total mystery to me. Don't let the log message fool you, this had nothing to do regexp "^[> ]*From:.*[a-zA-Z]". If I deletd one line from AWK script, it worked ok, if I added it back the core dump happened with that message x
procmail: Assigning "pfx=[> ]*" procmail: No match on "^[> ]*From:.*[a-zA-Z]" Segmentation fault (core dumped)Required settings
PMSRC must point to source direcry of procmail includerc code. This subroutine needs module(s):
- pm-javar.rc
Call arguments (variables to set before calling)
(none)Usage Example
Let's assume that you want to feed all forwarded UBE that is posted to spam-l maling list to your filter and see if it needs improving by checking the logs later. The forwarded UBE to the list is labelled "SPAM:" in the subject line.$RC_LIST = $PMSRC/pm-jalist.rc # mailing list detector $RC_ORIG = $PMSRC/pm-jaorig.rc # extract original $RC_UBE = $PMSRC/pm-jaube.rc # UBE filter...INCLUDERC = $RC_LIST # defines variable `LIST':0 * ! LIST ^^^^ { :0 # spam-l mailing list * LIST ?? spam * Subject: +SPAM: { INCLUDERC = $RC_ORIG # Change it to UBE message# Ok, next feed it to filetr, set some vars first # Log = Short log; What filters were applied to message # mbx = If message was tarpped, save it hereJA_UBE_LOG = "$PMSRC/pm-ube.log" JA_UBE_MBOX = "junk.ube.ok.mbox"INCLUDERC = $RC_UBE:0 : # If comes here, filter failed junk.ube.nok.mbx }:0 : # save normal list messages list.$LIST }
File id
Copyright (C) 1997-1999 Jari Aalto
Created: 1997-12
Contactid: jari.aalto@poboxes.com
Keywords: procmail ping auto-ack recipe
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
When I'm on remote site and I don't seem to get throught with telnet or even with Unix ping(1), I want to know if the at least the mailserver is up. I can send a ping message and the auto responder will reply immediately.Sometimes, when you send a message to a person, it would be nice, if you could test that the destination address is valid, before sending a message to a black hole. If the receiver had ping service running; like this, then you would know that you spelled the the right address. (after wondering two weeks; why you don't get response). Nowadays finger(1) command seems to be blocked many times.
This recipe answers to simple ping message like this:
To: you@site.com Subject: pingRecipe sends a short message back to the sender.
Required settings
PMSRC must point to source direcry of procmail code. This subroutine will include
- pm-javar.rc
- pm-jastore.rc
Call arguments (variables to set before calling)
(see Usage)Usage example
JA_PING_MBOX = $HOME/Mail/spool/ping.spool INCLUDERC = $PMSRC/pm-japing.rc
File id
Copyright (C) 1998-1999 Jari Aalto jari.aalto@poboxes.com
Maintainer: Jari Aalto jari.aalto@poboxes.com
Created: 1998-01
Keywords: procmail pop3 emulation recipe
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
Ahem, that pop3 is just to draw your attention. This module has nothing to do with pop3. The idea may resemble it though. This module listena pop3 requests, and when it gets one, it sends the whole mailbox content as separate forwarded messages to the account from where you sent the request.This is kinda "empty my mailbox in account X and send the messages to account Y"
You might have permanent forwarding on in account X, but if that is your socondary account, you can ask what messages has been arrived there with this recipe.
After you have configured your magic pop3 command, which is your password, simply send a message to account X, and this module initiates emptying the mailbox. Here is an example:
Subject: pOp3-send [mailbox] [kill]
- mailbox, is optional folder name which you want to process. it is $DEFAULT if not given in subject. Use absolute path if you specify one.
- if word kill is found, the mailbox will be meptied after forwarding. If the word is not found, messages are preserved.
Required settings
PMSRC must point to source direcry of procmail code. This subroutine will include
- pm-javar.rc
- pm-japop3.rc # Phew! We include ourself, we are recursive.
Call arguments (variables to set before calling)
- JA_POP3_SUBJECT_CMD is your personal access command string. If this is the first word in the subject line, forwarding starts. This string is case sensitive.
- JA_POP3_TMP is the file where mailbox is moved before starting to forward the messages. Do not put to point to yout $HOME, becaus that may exceed the quota.
- JA_POP3_TO_MUST_MATCH must contain regexp that match the email addresses where the pop3 messages are allowed to send. BE SURE TO DEFINE this. If you have account X,Y,Z where you want to receive pop3 messages, set this regexp to match those site's email addresses.
- JA_POP3_LOGFILE is the log where you can see how the forked procmail processes send each pop3 mail. You may want to set this to different location than your default $LOGFILE.
Return value
STATUS will contain mailbox name if valid pop3 request was received. You may wish to save the pop3 requests to separate folder. See example below.Example usage
You install this same setup for each site where you have account. This is the account X, from where you want to empty the mailboxes.RC_POP3 = $PMSRC/pm-japop3.rc.. somewhere in your .procmailrc ..JA_POP3_SUBJECT_CMD = myPoPcmd INCLUDERC = $RC_POP3# Save all pop3 requests to folder:0 : * STATUS ?? [a-z] mail.pop3-req.mboxIn account Y, from where you send the pop3 requests. Following code stores
The received messages to separate folder
# The MATCH wil lhave the host name from where the messages # were moved:0 : *$ X-Loop-Fwd:.*\.rc +\/$NSPC+ mail.fwd.$MATCH.mbox
File id
Copyright (C) 1997-98 Jari Aalto
$Contactid: jari.aalto@poboxes.com $
$Created: 1997-12 $
$keywords: procmail subroutine random GetLine $
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
Return random line or a line from a file. This subroutine uses shell command awk and possily wc to be as small burden to the system as possible.Required settings
You must have awk that supports VAR=value assignment syntax outside the code block: that is, in the input line. I know no awk that would not have this feature, but at least you know now what it takes.% awk '{print VAR; exit;}' VAR=1 /etc/passwdTry using GNU awk, if your standard awk didn't print 1 in above test. (Put this line to the top of your .procmailrc)
AWK = "gawk"Call arguments (variables to set before calling)
If intend to call this subroutine many times, then please calculate the number of lines beforehand and pass it to this subroutine. If the MAX is not set, then wc is called every time to find your the line count.
- FILE, from what file to select. Make sure this exists; existence is not checked here.
- [MAX] optional, number of lines in the FILE.
Returned value
variable LINEExample usage
# Select random line from a file$RC_RANDF = $PMSRC/pm-jarand.rc $COOKIE = $HOME/txt/cookie.lst...somewhere.. MAX=20 FILE=$COOKIE INCLUDERC=$RC_RANDF# LINE contains randomly read line
File id
Copyright (C) 1998 Jari Aalto
Maintainer: Jari Aalto
Created: 1998-01
Contactid: jari.aalto@poboxes.com
Keywords: procmail FileServer subroutine
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
This subroutine is part of the TPFS or MPFS file server. Check FILE for nonvalid filenames or other access problems.Input
- JA_SRV_F_FILE_CASE_SENSITIVE, flag
- FILE, filename to check. possibly converted to lowercase.
Output
- stat, set to "ok" if filename is acceptable. Otherwise contains brief error reason;
File id
Copyright (C) 1998 Jari Aalto jari.aalto@poboxes.com
Maintainer: Jari Aalto jari.aalto@poboxes.com
Created: 1998-01
Keywords: procmail FileServer subroutine
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
This subroutine is part of the MPFS file server. Handle BOUNCES to mail server messages, eg if delivery failed due to maximum byte limit.552 foo@site.com... Message is too large; 100000 bytes maxInput
(none) This recipe examines headers and body to see if it's daemon bounce.Output
- stat, set to "daemon" if message was handled.
File id
Copyright (C) 1998 Jari Aalto
Maintainer: Jari Aalto
Created: 1998-01
Contactid: jari.aalto@poboxes.com
Keywords: procmail FileServer subroutine
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
This module is part of the MPFS file server. Ssnd error notice: file didn't exist.Input
- FILE, file or command that did ot exist.
Output
- fld, additional field to be added to the saved mbox log message
File id
Copyright (C) 1997-1999 Jari Aalto
Maintainer: Jari Aalto
Created: 1997-09
Contactid: jari.aalto@poboxes.com
Keywords: procmail FileServer subroutine
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
This subroutine is part of the MPFS file server. Compose headers for reply message using formail -rt.Here is dry run example to test this module
% procmail DEFAULT=/dev/null VERBOSE=on LOGABSTRACT=all \ FORMAIL=/opt/local/bin/formail \ JA_SRV_FORMAIL_FROM=me@here \ JA_SRV_CONTENT_TYPE=content-type \ JA_SRV_XLOOP=xloop \ $HOME/pm/pm-jasrv-from.rc \ < $HOME/any-sample.emailInput
- JA_SRV_FORMAIL_FROM, JA_SRV_XLOOP
- JA_SRV_CONTENT_TYPE
Output
(none)
File id
Copyright (C) 1998-1999 Jari Aalto
Maintainer: Jari Aalto
Created: 1998-01
Contactid: jari.aalto@poboxes.com
Keywords: procmail FileServer subroutine
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
This subroutine is part of the TPFS or MPFS file server. Run $CODE and return resutls to to user. Subroutine is meant to be used for informational messages.Input
- code, code to run in shell
- stat, status message for user
File id
Copyright (C) 1998-1999 Jari Aalto
Maintainer: Jari Aalto
Created: 1998-01
Contactid: jari.aalto@poboxes.com
Keywords: procmail FileServer subroutine
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
This subroutine is part of MPFS file server. Send out FILE as multipart MIME message. The message will always be base64 encoded before sending.Input
- JA_SRV_MIME_MULTI_SEND, command to feed the composed and message which will handle sending it as multipart MIME.
- JA_SRV_MULTIPART_THRESHOLD is the hunk size for slitting mail.
- FILE, only filename part. Included in MIME headers.
- file, absolute path to send
File id
Copyright (C) 1998 Jari Aalto
Maintainer: Jari Aalto
Created: 1998-01
Contactid: jari.aalto@poboxes.com
Keywords: procmail FileServer subroutine
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
This subroutine is part of the MPFS file server. Check if file server request is on the JA_SRV_SUBJECT and do case or incasensitive check.To Dry run this module use following skeleton. Substitute keywods as needed to reflect your system setup:
% procmail DEFAULT=/dev/null VERBOSE=on LOGABSTRACT=all \ PMSRC=$HOME/txt JA_SRV_CMD_STRING=send \ JA_SRV_SUBJECT="send newbie_article.rtf noconv" \ txt/pm-jasrv-req.rc < ~/test.mailInput
- JA_SRV_F_CMD_CASE_SENSITIVE; if "yes" then server request is case sensitive.
- JA_SRV_FORMAIL_FROM. the email From field
Output
- stat, set to "ok" if request is accepted
File id
Copyright (C) 1998-1999 Jari Aalto jari.aalto@poboxes.com
Maintainer: Jari Aalto jari.aalto@poboxes.com
Created: 1998-01
Keywords: procmail FileServer subroutine
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
This subroutine is part of the MPFS file server. Send the requested file. You can dry-run test this module with following command: a) make sure that $HOME/test conatins any simple email message b) define FORMAIL if it isnot found from path.% procmail DEFAULT=/dev/null VERBOSE=on LOGABSTRACT=all \ PMSRC=$HOME/txt JA_SRV_LOG=/dev/null \ FORMAIL=/opt/local/bin/formail \ file=$HOME/test FILE=test WORD=WORD JA_SRV_FROM=foo@bar \ SENDMAIL="tee -a $HOME/test.send" txt/pm-jasrv-send.rc < ~/testNote:
The MIME headers here selected previously were:Content-type: application/octet-stream Content-transfer-encoding: x-gzip64But Defining own CTE such as x-gzip64 is strongly discouraged by the MIME RFC's. Most e-mail clients would be at a loss on how to handle these. Many would just bomb out and not even give you the opportunity to save it to a file. a more correct MIME type is this, which is now used:
Content-type: application/x-gzip Content-transfer-encoding: base64Input
o FILE is the filename(chdir to directory is already done) `file' is _absolute_ filename `WORD' is next word from subject line after FILE word. o JA_SRV_CMD_STRING is flag o JA_SRV_F_SUBJ_NOTIFY is flag
Output
- FILE_ERROR is set to "yes" if file is not found.
File id
Copyright (C) 1997-98 Jari Aalto jari.aalto@poboxes.com
Maintainer: Jari Aalto jari.aalto@poboxes.com
Created: 1997-09
Keywords: procmail FileServer
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
You can get newest version by sending email to Maintainer with subject "send <FILENAME>"
Description
This is the MPFS (Mime Procmail File Server) and it can send MIME compliant messages with command"send <ITEM> [WORD1] [WORD2]"Usually only the ITEM arg is used, and the rest of the words are for special uses like password and preventing file encoding. A typical request looks like:
Subject: send help # ask for file named 'help'Overview of features
- MIME types gzip and text/plain are supported.
- .gz .zip etc. files are sent out as base64 attachments
- .gz .tar.gz files that exceed 100K are sent out as MIME multiparts
- requires procmail 3.11+ and MATCH operator \/
- requires mmencode and gzip executables to be present in PATH.
Install: server file directory
You have to create a directory for the server where the files are kept. Usually I don't put the files there, but whenever I want to make a file available, I draw a hard or softlink to the real file.% mkdir $HOME/pm-server# Repeat this as needed for files you want to put available% cd $HOME/pm-server % ln -s $HOME/txt/interesting-file.txt interesting-file.txtYou define the server directory by setting
JA_SRV_FILE_DIR = $HOME/pm-serverThe short server log is written to file pointed by this variable:
JA_SRV_LOG = $HOME/pm-server.logThe incoming "send" requests are stored to mailbox pointed by following variable. The default value is /dev/null, but you may want to set it to ~/Mail/spool/log.srv.spool which can be used read as a newsgroup by Emacs Gnus [In Gnus create newsgroup with `G` m nnml log.srv]
JA_SRV_MSG_MBOXInstall: special files
Tweak this variable to commands you want to allow shell to execute in server's directory. This tells when <ITEM> "ls" means command instead of fileJA_SRV_SH_COMMAND = "^(ls|what)$"That means that request like this:
Subject: send ls # run "ls" command and return resultsBe sure that the commands exist in your system. See man pages for more if you want to know what these commands do. Commands cannot take switches currently for security reasons. Eg. if you want to give access to "ls -la" listing, put a file "ls-la.txt" available in the directory, user can get it with "send ls-la.txt"
ls -- list directory file -- print file type information. what -- prints all @(#) tags from files ident -- print all $ $ tags from filesInstall: file `help'
Users want to get a help file with message "send help" and the help is just a file in your server directory. Be sure to supply it prior to any other files. You can always draw a link to a file if you don't want to name it that way (eg. if you keep several server help files in a RCS tree)# draw symlink to `help'% ln -s $HOME/txt/srv-public-hlp.txt $HOME/server/helpBasic usage in details
The server accepts command in format"send <ITEM> [CMD|PASSWORD]"Where ITEM can be any name as long as it starts with [^ .]. The regexp says: Anything goes as long as FILE does not start with space or period. This gives you quite a much freedom to construct filenames. if you want to hand out file:
.procmailrcYou can't. Instead make a link to point to plain "procmailrc" without the leading period. There is also additional checks against possible security threat "../" like below; user can't request such file.
../../../gotcha or dir/../../gotchaThe filename cannot contain special characters like [*?<>{}()].
Advanced usage
[conversions]If some of your files are big, it makes sense to send them in compressed base64 format; which in MIME world is called content-type gzip. You can set a regexp to enforce encoding for your big files before they are sent to user. The following setting will send all text files in compressed format to user.
JA_SRV_XGZIP_REGEXP = "\.txt"When the message is composed a header is inserted into the message telling how the message is to be decoded, in case user doesn't have decent MUA that can handle the MIME type:
X-comment: To decode, cat msg| mmencode -u| gzip -d > test.txt[noconv and gz]
The WORD1 parameter after the FILE is optional and user can override base64 encoding and request plain file if he uses word "noconv".
Subject: send <FILE> [noconv|gzip]However, there are files where noconv must not be obeyed, like the compressed packages that you have put available in .zip, .gz, tar.gz or .tgz (GNU tar) format. Following variable controls
when file is always sent as base64:JA_SRV_BASE64_ALWAYSIf the WORD1 is "gz" or "gzip", then the gzip is explicitly requested, This may be desirable, because some of the text files in the server directory may be big and some accounts don't accept big messages. A typical bounce looks like:
552 foo@site.com... Message is too large; 100000 bytes max 554 foo@site.com Service unavailableThese kind of file server bounce messages are handled in separate module which notifies the user that his account didn't accept the sent file.
[case sensitivity]
By default the request word ("send") and ITEM (filename) are not case sensitive, unless you set these flags:
JA_SRV_F_CMD_CASE_SENSITIVE = "yes" JA_SRV_F_FILE_CASE_SENSITIVE = "yes"If values are "no", then these are identical commands:
Subject: Send Help Subject SEND HELPMultipart mime messages
If you want to deliver big files, you better be sure not to send them as a big file. That blocks the connection between every host along the path that the big file is transferred. The solution is to use MIME multiparts that can be assembled back in the receiving MUA. (In case you don't have multipart assembler at hand, send message "send mime+.pl" to $Contactid and you will receive Perl script to do it).MIME multiparts are sent out if
- Filename matches JA_SRV_BASE64_ALWAYS, typically tar.gz, zip
- Filesize is bigger than JA_SRV_MULTIPART_THRESHOLD, where default chunk size is 100K.
When a file meets these criteria, it is read to the BODY of message and base64 encoded. This all happens in memory, so watch procmail logs to see if any problems with very big files. (>30Meg). Next, if the base64 conversion succeeded, the composed is handed to
JA_SRV_MIME_MULTI_SENDWhich does the actual delivery and splitting. The default program used is splitmail. Make sure you have it or substitute the program with some equivalent one.
Stopping server
Sometimes you're making rearrangements in you file directory or doing some other maintenance and you are unable to respond to send requests. You can stop the server by settingJA_SRV_IN_USE = "no"And when you want to enable the server again; just comment out the statement or assign yes. [The default is yes]. When this variable is set to no, the server sends a message from following variable as a response to any "send" request.
JA_SRV_IN_USE_NO_MSGUsing password to validate file requests
You should be aware that this file server's implementation is public in nature. Anyone who asks for a file is allowed to get it. But it would be good if you could limit the access to documents with some simple way, like if you set up two file servers (see next chapter) where one is public and the other is interesting only to group of people. You can define a string that must be found in Subject field by setting the following variableJA_SRV_PASSWORD = ".*" # defaultThe default value will match anything in the subject, thus making the server public. But if you set it like this
JA_SRV_PASSWORD ".*123"Then string "123" must be there somewhere in the line, like here
Subject: send <FILE> 123Yes, "123" is actually a CMD definition, but it doesn't matter because there is no CMD 123. Subject now matches password and the server can be accessed. Of course the following is valid too.
Subject: send <FILE> noconv 123If the password was wrong, server won't tell it. The message just lands to your mailbox in that case and you can investigate who tried to access the restricted server.
Changing server's command string (multiple servers)
The default command string is "send", but you can change it and thus create multiple services. Here is one example, where you have set up two file servers where each has its own directory.# The public serverJA_SRV_CMD_STRING = "send" JA_SRV_FILE_DIR = $HOME/server/public INCLUDERC = $HOME/procmail/pm-jasrv.rc# Company server, only interests fellow workers. # Here "xyz-send" is just magic server request string. # Notice case sensitivity settings.JA_SRV_F_CMD_CASE_SENSITIVE = "yes" JA_SRV_CMD_STRING = "xyz-send" JA_SRV_PASSWORD = ".*12qw" JA_SRV_FILE_DIR = $HOME/server/public/xyz-dir INCLUDERC = $HOME/procmail/pm-jasrv.rcNotes from the author
[basic Mime type note]All basic files that you send must be US-ASCII, 7bit. At least that is the default mime type used. See JA_SRV_CONTENT_TYPE. I once received following message back
----- Transcript of session follows ----- 554 foo@bar... Cannot send 8-bit data to 7-bit destination 501 foo@bar... Data format errorbecause in the previous releases, the MIME type headers were not in the message saying that the content really was plain 7bit ascii.
[Sending the file as is]
Note, that the file is included "as is" without any extra start-of-file or end-of-file tags. This is possible, because the file is sent in MIME format.
[Using one line log entry]
It may look very spartan to print a single line log entry. You see messages like above in the file server log. Using one line entry instead of multiline announcements makes it possible to write a small perl tool to parse information from a single line. If you get many file server messages per day, it quicker to look at the single line entries too.
[ja-srv1; sh file; Foo Bar foo@site.com;] [ja-srv1; send xxx-file.txt; Foo Bar foo@site.com;] | Server's request keywords (you may have multiple servers)[todo]
(*) MIME multipart message's mime headers may need some adjustments.
(*) I rely on simple regexp to send out base64 or gzip files. The natural extension would be to use file size threshold: if file is bigger than N bytes, send it out with gzip. And further: if file is more than NN bytes, send it out as multipart MIME.
(*) In fact there is a slight mime type errors: .zip files should be send as application/zip. If you have experience with the mime types, please contact me and help me to sort out proper mime headers.
If you have suggestions or ideas, please drop me am email.
Required settings
PMSRC must point to the source directory of procmail code. This subroutine will include many pm-jasrv-*.rc modules and other files from there.Please test the File Server in your environment before you start using it for every day. For example I had some weird local problem where PATH had /usr/contrib/bin/ where gzip was supposed to be, but in spite of my tries procmail didn't find it along the path. Don't ask why. I now use absolute binary name:
GZIP = /usr/contrib/bin/gzipIn addition, if your messages are not sent to recipient, but you get daemon message:
... Recipient names must be specifiedThat's because you have setting SENDMAIL="sendmail"; which is not enough. It must be
SENDMAIL = "sendmail -oi -t"Usage example
This is my .procmailrc installation. Notice that the file server code is used only if you get "send" request. On the other hand, this double wrapping is not all necessary, you could as well rely on the File server's capability to detect SEND request.PMSRC = $HOME/pm # directory where the procmail rc files are RC_FSRV = $PMSRC/pm-jasrv.rcmySavedLOGFILE = $LOGFILE # record file server actions elsewhere LOGFILE = $PMSRC/pm-jasrv.log# Listen "send" requests. :0 * ^Subject: +send\> { JA_SRV_FILE_DIR = $HOME/fsrv # Where to get the files JA_SRV_LOG = $HOME/fsrv.log # Write log here INCLUDERC = $RC_FSRV # Use file server now }LOGFILE = $mySavedLOGFILE
File id
Copyright (C) 1997-98 Jari Aalto jari.aalto@poboxes.com
Created: 1997-11 $
Contactid: jari.aalto@poboxes.com
Keywords: procmail storing subroutine
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
This subroutine stores the message to file pointed by MBOX. This subroutine is meant to be used with the the other general purpose includerc files. This makes it possible to have a centralized file storage handling for all your rc files.Regular user doesn't get much out of this rc unless he mixes both gz and regular files in his .procmailrc
R e p e a t: This module is basis for general purpose procmail rc plug-ins to strre message to mailbox pointed by some rc configuration variable. Normal user can simply say in his .procmailrc:
:0: mail.privateRequired settings
(none)Call arguments (variables to set before calling)
MBOX must have been set to point to message storage.if MBOX_MH is set to "yes"
- message is delivered to MH mailbox using MBOX_MH_CMD
otherwise
- If MBOX is some.mbox the message is stored as is.
- If MBOX is some.mbox.gz the message is gzipped to folder.
- If MBOX is some-dir/. then deliver as individual files
Example usage
$RC_MBOX = $PMSRC/pm-jastore.rc:0 * condition { MBOX = $HOME/Mail/spool/junk.mbox INCLUDERC = $RC_MBOX }
File id
Copyright (C) 1998 Jari Aalto jari.aalto@poboxes.com
Created: 1998-01
Maintainer: Jari Aalto jari.aalto@poboxes.com
Keywords: procmail SubjectClean recipe
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
There are many different Email programs out there that add their own reply characters to the subject field. The most sad programs come usually from PC platform. Eg. Microsoft has gained a lot of bad reputation due to it's own standards.
- MS Explorer can use localized reply strings, Eg Vs: or vast: seems to be finnish Vastaus.
- MS product Outlook (??) can be configured similarly. I have received swedish Sv: -Svar for Svaring (eng: reply)
- MS mail uses FW: in forwarded mails.
- Intelligent MUAs try to keep count of replies with Re2: or Re[2]
- Japanese MUA Denshin 8 Go V321.1b7 has sent Re^2:
- Some mua uses Re>
- Lotus notes (in French version) uses Ref:
- Some MS product sends UQ:
- XXX uses -reply
- Forwarding schemes: (fwd) [fwd] <fwd> fw: [FWD: [FWD:]]
- Subject references: -subj subj- subj:
There already is a de facto standard where message should contain only single Re: if message has been replied to (no matter how many times). This makes it possible to do efficient message threading by only using Subject and date fields. And grepping same subjects is lot easier than from this horrible mess. Note that all text is on one line, the subject has been broken only for visual reasons:
Subject: re- Re^2: Re[32]: FW: Re: Re(15) Sv: Re[9]: -reply (fwd) [fwd] <fwd> fw: [FWD: [FWD:]] -subj subj: subj: subj- testThis recipe standardizes any subject (like above) that has been replied to, to de facto format below. That is: "Any number of RE will be converted to single RE and any number of FWD will be converted to single FWD."
Subject: Re: test (fwd)About In-Reply-To header
If there is In-Reply-to header in the message, but there is not Re: in the subject line, one is added automatically. Some broken Mailers forget to add the Re: to the Subject line.Variable JA_SUBJECT_SAVE
This is by default yes which makes the original subject saved under header field Old-Subject. If you don't want that extra header generated, set this variable to noVariable JA_SUBJECT_FWD_KILL
This is by default yes, which will kill extra forwarding indication words like (fwd) [fwd] <fwd> <f>. If you set this to no, then all the fowarding words are preserved. The de facto forwad format is:Subject: This subject (fwd)Code note
This subroutine's intention is to make your Subject like more expressive by deleting redundant information. I have taken a simplistic aproach where the Subject consists of list of WORDS whose each attribute can be either ok or delete. There is no attempt made to investigate the structire of the Subject. I'll explain this better with examples:Re: New subject (was Re: Old subject)should be treated syntactically like "New subject" and forgetting anything between parenthesis. This is however not respected and not even tried. The rule applied here is "One Re: is tolerated", so the subject won't change. It doesn't matter where "Re:" is.
But here the subjject is changed. The rule applied is "Delete all unwanted words and then add one Re: to the beginning if OLD content had any Reply indications"
Re: New subject (was Re: Old subject) --> Re: New subject (was Old subject)Now you understand how the Subject changing works.
IMPORTANT notice
Please check that your SHELL variable setting for procmail is sh derivate, /bin/sh or /bin/bash. This module won't work well with other shells.Awk usage note
awk is a small, efective and much smaller than perl for little tasks. See the verbose log and make sure your awk understands VAR="value" passing syntax. Change it to nawk or gawk if they work better than your standard awk.AWK = "nawk" # you may need this, try also gawkExample usage
You need nothing special, just include this recipe before you save message to folder.INCLUDERC = $PMSRC/pm-jasubject.rcCustomizations: Let's say Polish M$Outlook uses ODP: instead of standard re: and you want to handle that too: Then set:
JA_SUBJECT_KILL = "ODP:" JA_SUBJECT_SAVE = "no" INCLUDERC = $PMSRC/pm-jasubject.rcYou ca use JA_SUBJECT_KILL to kill any additional words from the subject line. Eg. if you have good newsreader, you don't need the mailing list prefixes that some mailing lists add to the beginning
Subject: [LIST-xxx] the subject hereto remove that list prefix, you simply match it
JA_SUBJECT_KILL = "(list-xxx|list-yyy)"Inportant: The regexp must be all lowercase, because when match happens, the word has been converted to lowercase.
Debugging
You can dry-run test this module with following command and wathing output. Substitute variables as they are in your system. You feed the content of entire example mail where the Subject thatt needs correction is found.% procmail SHELL=/bin/sh AWK=gawk VERBOSE=on LOGABSTRACT=all \ PMSRC=txt JA_SUBJECT_KILL="(ace-users)" txt/pm-jasubject.rc \ < ~/test.mail
File id
Copyright (C) 1997-98 Jari Aalto
Maintainer: Jari Aalto
Created: 1997-11
$Contactid: jari.aalto@poboxes.com $
$URL: ftp://cs.uta.fi/pub/ssjaaa/ $
$Keywords: procmail subroutine date parsing $
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
You can get newest version by sending email to Contactid with subject "send <FILENAME>"
Description
This includerc parses date from variable INPUT which has string"hh:mm:ss"Example input
"Thu, 13 Nov 1997 11:43:23 +0200"Returned values
hh = 2 digits mm = 2 digits ss = 2 digitsVariable ERROR is set to "yes" if it couldn't recognize the INPUT and couldn't parse all hh, mm, ss variables.
Required settings
PMSRC must point to source direcry of procmail code. This subroutine will include pm-javar.rc from there.Call arguments (variables to set before calling)
INPUT = string-to-parseThe INPUT can be anything as long as it contains NN:NN:NN
Usage example
Get the time of received message. The From_ header will always have the standard time stamp.PMSRC = $HOME/pm RC_DATE_TIME = $PMSRC/pm-jatime.rc:0 c * ^From +\/.* { INPUT = $MATCH # Turn off the logging while executing this partVERBOSE=off INCLUDERC = $RC_DATE_TIME VERBOSE=on:0 * ERROR ?? yes { # Should not ever happen, you have broken From_ } }
File id
Copyright (C) 1997-98 Jari Aalto
Created: 1997-09
Maintainer: Jari Aalto jari.aalto@poboxes.com
Keywords: procmail UBE filter recipe
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Warning
Put all your UBE (aka spam) filters towards the end of your procmailrc. The idea is that valid messages are filed
first (mailing lists, your work and private mail, bounces) and only the uncategorized messages are checked.Overview of features
- Requires procmail 3.11pre7+
- Header based filtering: Minimum headers, Pegasus bulk mail, X-uidl validity check
- Address based filtering: Numeric address, Invalid address (eg. me@myMarketing.global), UBE-like(friend,remove request.)
- Text filtering: no html accepted, common advertising slogans.
- You don't need external files: site block lists, the heuristics nail most of the UBE messages. Just plug in this module and you have UBE shield active.
Remember: this is not 100%, there is always some mishits, so don't just junk messages to /dev/null. Instead set JA_UBE_MBOX to the appropriate mailbox where you want to store the messages.
Description
My life changed when Daniel smith posted his spam.rc, where he had gathered many tips and heuristics to filter UBE email. The filter expressed work of many Procmail usears including [phil] and [dan]. You can get the original file by sending email with subject pm-dsspam.rc to maintainer.I modified the filtering rules a bit, took out some rules that catched false email messages and made the package look a bit more general so that it could be included via INCLUDERC in the standard way. Not all ideas here are mine; many of them are directly from Dan's original filter with slight modifications.
Thanks to Daniel and others, the UBE bomb days are gone, when this filter is active. Couple of UBE email may still lurk into my mailbox, but that's life; no shield is 100% proof.
Logging the events
The default subroutine pm-jaube1.rc does nothing fancy, but writes a short log to JA_UBE_LOG and stores the message into JA_UBE_MBOX, which is sort of a brief biff log, where information about incoming mail and what happened to it is stored. When message arrives; date, from and subject should be written to the log and when message gets filed; you should write the folder and reason. Eg. my biff log looks like this:1997-12-08 Foo bar@x.com send pm-tips.txt [ja-srv1; sent ema-keys.txt; Foo bar@x.com ;] 1997-12-08 work@home.com Extra Holiday $$$$$ [jaube; Marketing-Big-ExitCode; LEGAL, MONEY-MAKING PHENOMENON] 1997-12-09 Denizen logger@california.com [RePol] hiding 1997-12-09 david X dx@a.com Re: Send list to incoming folder 1997-12-09 david X dx@a.com Re: Send list to incoming folder 1997-12-09 OMC manager omcman@foo.fi "Environments updated" [my; work-localenv] 1997-12-09 doodle@gruppe.org Re: Gnus (Emacs Newsreader) FAQ [my; emacs; Re: Gnus (Emacs Newsreader) FAQ ]First, I got a message to my fileserver and it responded. Then came a UBE message that was identified and saved to UBE folder. Next 3 messages vere ones, that were filed to mailing-list folders and there was no [] action displayed for them (left out due to high volume of these messages). Second Last; that was internal work message, Lastly someone mailed me about Emacs and that message was filed to emacs folder.
The basic timestamp recipe in my .procmailrc is as follows. Variable TODAY is $YYYY-$MM-$DD which you get from pm-jadate.rc. The LISTS exclude mailing lists. I don' want to see their activity. Variables FROM SUBJECT are read header fields from the message
BIFF = $HOME/tmp/pm.biff:0 hwic: *$ ! $LISTS |echo "$TODAY $FROM $FSUBJECT" >> $BIFFHere is small perl script to print summary of trapped UBE messages. It gives nice overview which recipes trap most of the UBE messages.
perl5 -ne '/jaube; (\S+)/; $s{$1}++; \ END { $s = (map{$x += $_; $_= $x} values %s)[-1]; \ $i = int $s{$_}/$s *100; \ for (keys %s) { printf "$s{$_} $i $_\n" } \ }' \ pm-hdr.log \ sort -nrHere is sample results of mine during 1998-08-16 - 1998-10-04. There are total of 3248 UBE messages trapped.
count % type ------------------------------------------ 554 17 Marketing-CountBigLetterWords; 457 14 Marketing; 422 12 Marketing-SelectedBigLetterWords; 349 10 AddrBogus-ToFrom; 263 8 ReceivedFrom-Mismatch; 223 6 NoDirectAddress-ToCc; 216 6 HdrForgedPegasus; 164 5 AddrBogus-To; 151 4 MessageId; 121 3 FromReceived-Mismatch; 102 3 BodyHtml; 73 2 Received-IPError; 63 1 Identical-FromTo; 53 1 AddrInvalid; 15 0 From-nslookup; 9 0 HdrReceivedTime; 7 0 HdrX-UIDL; 4 0 Marketing-headers;
About bouncing message back with exitcode 67 or 77
The gneral concensus is, that you should not send bounches, the UBE sender is not there, because the address is usually forged. Do not increase the network traffic. Instead save the messages to folders and periodically send complaints to postmasters that you find in Received headers. If you decide to bounce anyway, be sure to do FROM_DAEMON and mailing list tests before sending anything back. It's not nice to be forced to apologize about boounces to a wrong destination.Required settings
PMSRC must point to source direcry of procmail code. This recipe file will include
- pm-javar.rc
- pm-jaube1.rc (includes pm-jastore.rc)
- pm-janslookup.rc
- pm-jaaddr.rc
Call arguments (variables to set before calling)
- JA_UBE_VALID_ADDR, your email addresses or valid from addresses.
- JA_UBE_RC, inclurerc called when ube was trapped. The default action is to save the message to folder and write a brief log.
- JA_UBE_LOG, if not empty, then a brief message is written to this log file when ube was triggered.
- JA_UBE_HDR, If JA_UBE_LOG is empty, no log is written to separate file, but a new header is added which tells which recipe was triggered.
- JA_UBE_MBOX, where to store the UBE. Add ".gz" extension to file name if you want to save messages in gzip format. If this variable is empty, the message is not stored anywhere, but falls through pm-jaube.rc.
- Various flags: Some of the ube detecting recipes give more false hits than nail real ube. Experiment with yourself and turn on or off the recipes that work for the kind of ube messages you receive.
- JA_UBE_MAX_BIG_WORDS, the maximum count of big letter words in the meswsage that is tolerated. The current count 5 is rather conservative and I suggest you to increase it to prevent trapping too many false hits. Alternatively update JA_UBE_CAPS_OK to include accepted words.
- JA_UBE_APPRENTLY_TO_MAX, how many Apparently-To headers are tolerated. Default is 3.
- JA_UBE_MAX_HTML_TAGS, maximum count of html tags allowed in the body
Return values
If you use your own JA_UBE_RC then you can read these values, when your script is called. They are also set after recipe if JA_UBE_MBOX is empty "".
- ERROR, is set to short ube trigger recipe reason
- ERROR_MATCH, is set to some MATCH that happened while triggering UBE message.
The UBE reason ERROR is also seen in the separate header.
- JA_UBE_HDR, header not added if this is empty ""
Usage example
It is best, that the UBE filterings are put last to your .procmailrc. Handle all legimate messages at the beginning and save them to folders. Only messages that didn't fit to previous categories shgould be feed to to this recipe.# - All legimite messages already handled and saved before this # - Activate the filter only for messages that are not from # daemon and not from valid senders: like from "my" domain # and mailing lists and from soemwhere else. Define your # favourite VALID_FROM regexp.RC_UBE = $PMSRC/pm-jaube.rc VALID_FROM = "(my_domain|my_mailinglist)":0 *$ ! ^From:.*$VALID_FROM *$ ! $from_daemon { JA_UBE_MBOX = "junk.ube" INCLUDERC = $RC_UBE }I have also received UBE messages that fool FROM_DAEMON test, so you could also consider using this condition. The standard daemon error message almost always has sentence "Transcript of session follows" in the body.
* -1^0 ^FROM_DAEMON * ! 2^1 B ?? Transcript of session followsIntead of simple daemon test:
*$ ! $from_daemonFile layout
tinybm.el/&tags and tinytab.el for the 4 tab text placement. See also unix what(1) and GNU RCS ident(1).Bug reports or updates
1998-08-24 Gregory Sutter sent update to his recipe.1998-02-27 bochmann@TUDURZ.urz.tu-dresden.de (Henryk Bochmann) reported that the ReceivedFrom test triggered all htmail messages. Now Fiexed.
File id
Copyright (C) 1997-98 Jari Aalto jari.aalto@poboxes.com
Created: 1997-09 $
Maintainer: Jari Aalto jari.aalto@poboxes.com
Keywords: procmail UBE filter subroutine
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Documentation
This file is part of the "pm-jaube.rc". This subroutine is called when likely UBE message has been triggered.Required settings
PMSRC must point to source direcry of procmail code. This recipe file will include
- pm-jastore.rc
File id
Copyright (C) 1997-98 Jari Aalto
Created: 1997-11
Contactid: jari.aalto@poboxes.com
keywords: procmail vacation framework recipe
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
Framework for all programs that need to reply to messages only once. Usually known as "vacation" feature. If you cahnge th cache file, you can attach this recipe to any messages that you want to deal with only once.Required settings
PMSRC must point to source directory of procmail code. This subroutine will include
- pm-javar.rc
Call arguments (variables to set before calling)
- JA_VAC, To activate vacation, set value to "yes"
- JA_VAC_RC, When new message-id is found, run this includerc
- JA_VAC_ID_CACHE, Remember to clear this file when you start the vacation.
Usage example
To turn on the vacation feature, create ~/.vac file and recipe below activates vacation. If the vacation is not active, then cache file is removed. (automatic cleanup). The VERBOSE is also turned off when you're on vacation; so that your procmail log will not get filled.So when you go to vacation, you 'touch ~/.vac' and update ~/vacation.msg. When you come back, you 'rm ~/.vac'. That's it.
IMPORTANT: If you are subscribed to mailing lists, be sure to file messages from those services first and put the vacation recipe only after the list or bot messages. Also add sufficent "!" conditions in order not to reply to other "bot" service messages.
JA_VAC_ID_CACHE = $HOME/.pm-vac.cache:0 *$ ? $IS_EXIST $HOME/.vac { VERBOSE = off JA_VAC = "yes" JA_VAC_RC = $PMSRC/pm-myvac.rc # my vacation recipe INCLUDERC = $PMSRC/pm-javac.rc # framework } :0 E # else * ? $IS_EXIST $JA_VAC_ID_CACHE { dummy = `$RM -f $JA_VAC_ID_CACHE` }Here is example of pm-myvac.rc recipe
# Change subject:0 fhw * ^Subject: *\/[^ ].* | $FORMAIL -I "Subject: vacation (was: $MATCH)":0 fb # put message to body | $CAT $HOME/.vacation.msg:0 # Send it | $SENDMAIL
File id
Copyright (C) 1997-1999 Jari Aalto
Contactid: jari.aalto@poboxes.com
Created: 1997-12
Keywords: procmail variableDEFS subroutine
This code is free software in terms of GNU Gen. pub. Lic. v2 or later You can get newest version by sending email to maintainer with subject "send <FILENAME>"
Description
This file defines common variables that you can use in the recipe's condition line. Procmail does not know scape sequences like \t or \n and it is therefore much more readable to use variables for common regular expression matches. Pay attention that the line with "*$ ", where "$" expands the variables::0 *$ $s+something+$s+$d+$a+The equivalent without variables (you don't see the tabs and spaces)
:0 # Space + tab * [ ]something[ ][0-9]+[a-z]+In addition all system dependent variables are defined in this module. For example if you have Gnu awk, I stringly suggest that you set:
AWK = "path/to/gawk"Also if your perl binary still runs the perl 4, then please set
PERL = "path/to/perl5"You can define these variables before or after the module, just make sure the binaries reflect your system settings.
Standard variables defined
See pm-tips.txt file for full explanation or look at the source code.SPC WSPC NSPC SPCL # Whitespace, NonWhitespace, Whitespc+linefeed \s \d \D \w \W and \a \A # perl styled variablesSpecial variable from_daemon
In order to boost procmail and to save extra CU cycles, this module defines variable from_daemon that caches the information of ^FROM_DAEMON. You can refer to from_daemon as you would to big brother FROM_DAEMON. This has the advantage that procmail has already computed the result and the variable from_daemon is used as a cache, thus avoiding repeated FROM_DAEMON regexp tests, which are expensive. Variable from_daemon_match contains "" or the result of matched daemon text.*$ $from_daemonor the familiar
*$ ! $from_daemonInstead of using the regexp parsing with
* ^FROM_DAEMONand
* ! ^FROM_DAEMONSpecial variable from_mailer
Works like from_daemon variable but in respect to FROM_MAILER. The matches text is in from_mailer_matchUsage example
Put this as a first line in you includerc. It checks if WSPC is does not include space --> load the variable definitions. If the variable is already defined, the file is not loaded. This is something like #ifdef -- #endif in C language.:0 * ! WSPC ?? ( ) { INCLUDERC = $PMSRC/pm-javar.rc }For your .procmailrc, you can simply put this, because you want to load the variables at startup
INCLUDERC = $PMSRC/pm-javar.rcDefined modules
After this file loads, you can refer to any module with $RC_MODULE. Eg. to call email spit module in your code you would use following. See at the end of this file for all defined modulenames.INCLUDERC = $RC_EMAIL
Copyright (C) Jari Aalto. All rights reserved. This material can be publically distributed and copied with the permission of Author, provided that you mention the Author's name and email address or http reference where to get the original document.
This file has been automatically generated from plain text file
with perl script t2html.pl 1.174
Document author: Jari Aalto
Url: http://www.procmail.org/jari/pm-code.txt
Contact: <jari.aalto@poboxes.com>
Html date: 1999-04-30 12:11