freebsd-ports/mail/dspam/files/UPDATING

# $Tecnik: ports/mail/dspam/files/UPDATING,v 1.10 2006/04/30 10:04:19 itetcu Exp $
# $FreeBSD$
#
# for each PKGNAME or user option change an entry should be added in this file
# each entry 

This file documents some of the problems you may encounter when
upgrading dspam port.  I will try my best to minimize these disruptions,
but sometimes they are unavoidable.

You should get into the habit of checking this file for changes each
time before attempting any port upgrade.

This file concetrate on the FreeBSD / port specific changes and one should read
also the UPGRADING enclosed in the dspam distribution. You can see it by doing
in the port directory:
make extract;  more `find . -type f -maxdepth 2 -name UPGRADING`

###########################################################################
# dspam-3.6.6
#

Removed feature:
- Phased out deprecated Berkeley DB drivers
- Phased out legacy tools (dspam_corpus, dspam_genaliases)
Bugfixes:
- trainPristine preference turned "off" now overrides config turned "on"
- segfault fix for when correcting messages using UIDInSignature with MySQL
- when using logfile, write errors result in segfault

###########################################################################
# dspam-3.6.5
#

New features:
- support for reading from multiple mysql servers; support for separate
read/write servers to be used with mysql_drv; see dspam.conf
- SYSLOG options knob: log to via syslog if "on" or in flat file if "off"
- dspam_stats: -t for displaying a total of all stats included in the
original query

Bugfixes:
- fixed a bug causing --deliver=summary to return no output when used in
dspamc
- fixed a segfault which can occur if TrainingMode is not specified in
dspam.conf
- invalid read/segfault (dspam.c)
- segfault on problems establishing connectivity to clamav (dspam.c)
- segfault on NULL username (mysql_drv.c)
- bug causing writing of flat-file preferences to fail (pref.c)
- fragment file overwritten on retrain (cgi)
- corrects the output of "dspam_admin aggr pref" (tools/dspam_admin.c)
- prevent quarantining of message when delivering summary

WebUI: a lot of fixes and improvements

PostgeSQL: improvements to purge scripts and object creation script

Port changes:
-------------

Add MySQL 5.1 as backend. [1]
Default MySQL version is now 5.0

Optionally depend on LightHTTPD instead of Apache if WITH_CGI=on [2]

Add dspam_debug rc.d options (default off): starts dspam with debug
logging (you need at least WITH_DEBUG).

Fix OPTIONS handling for INDEX (describe); this is a long standing bug
and a *big* pontyhat for me; it affected _only_ dependecies recorded in
INDEX _if_ OPTIONS were not set (like for package building); however,
the package itself was always built right, dependecies recorded OK, etc.

Add SCE-tindy as MASTER_SITE, much faster site that my other server. [3]

Requested by:   Odhiambo WASHINGTON <odhiambo.raburu@wananchi.com> [1]
Submitted by:   "Daniel S. Haischt"
                <me@daniel.stefan.haischt.name> [2] (based on)
My thanks again to Joey Freeland for this machine and the bandwidth [3]


###########################################################################
# dspam-3.6.4
#

BugFixes:
- accuracy fixes (esp. for FP dspam is much better that in 3.6.3 release)
- safeguards for segfault when strange clamav behavior
- for no output when using --classify in client/server mode
- LOG_WARNING if truncating MySQL user/pass + plus othersmall fixes

New features:
- dspam_train: a true training and testing mechanism, useful for building
pretrained databases or training a user with their own corpus. also provides
a test jig for measuring efficiency/accuracy with a corpus over a
configuration; support for training using an index file to define the order
of ham/spam
- DeliveryHost for delivery to different hosts based on domain
- ClassAlias options, useful if classifying things other than spam
- WebUI: undo option for retraining
- WebUI: support for existing storeFragments option to recall message in
history
- WebUI: mass-retraining option
- Documentation: of all user preferences in the README

Changed features:
- dspam-statrs: changed names; now displays TP (true positives),
TN (true negatives), FN (false negatives), and FP (false positives),
OCA (Overall Accuracy)
- dspam_corpus now uses default settings for features and training modes,
instead of its own and now requires --spam or --nonspam arguments
- the trainining buffer (Feature tb=n in dspam.conf) is now offby default
- dspam.conf: HashRec sizes prime numbers for better spread
- no signature on classify
- ignore X-DSPAM-* in tokenizer code

Removed features:
- removed neural networking (NEURAL_NET), which was experimental, needed a
rewrite, had no support and high maintenance


###########################################################################
# dspam-3.6.3
#

To manny things to note here, please see mail/dsapm-devel/UPDATING and read
UPGRADING and CHANGELOG and adjust your dspam.conf accordingly; there are
NEW DIRECTIVES (see dspam.conf.sample) so if you miss this dspam will have
bad accuracy or could (possibly) crash.
- you can now have the database drivers dynamically loaded at run time so
you can select multiple drivers in the OPTIONS screen; see dspam.conf.
- also algorithm selection is done exclusively via dspam.conf
- !!! WARNING !!! MaxMessageSize is also applied to ClamAV virus scanning


###########################################################################
# dspam-3.4.8 (2005-11-17)
#

fix RUN_DEPENDS for WITH_CGI=on by USE_APACHE=1.3+, no need to upgrade if
already installed


###########################################################################
# dspam-3.4.8
#

BUGFIX: File descriptor leaks when connection to delivery host fails
BUGFIX: Decoding NUL characters causes message truncation
BUGFIX: Many files not written to group directory when using managed groups
BUGFIX: Some LDAs/MTAs truncate message when single dots not properly quoted
BUGFIX: 7BIT encoding contains 8BIT characters
BUGFIX: Messages to multiple local recipients may fail to decode completely 
BUGFIX: Delivery of false positives fails in managed groups
BUGFIX: LMTP/SMTP delivery fails on non-250 2xx response code
BUGFIX: Messages missing a terminating boundary fail to receive a signature


###########################################################################
# dspam-3.4.6.20050523.0845
# 

BUGFIX: fix for managed groups where delivery of false positives would fail
due to the managed group not being recognized

BUGFIX: fixed bug where messages lacking a terminating boundary would fail
to receive a signature in the message body


###########################################################################
# dspam-3.4.6.20050512.1145
#

BUGFIX: fixed signature embedding on malformatted boundary bug


###########################################################################
# dspam-3.4.6
#

- BUG FIX: "obscure" [1] bug causing dspam to crash under certain conditions
when the loose signature was provided without the appropriate delimiter
- BUG FIX: fix sqlite3 dependency check [2]
- BUG FIX: don't strip binary if we want GDB suport
- BUG FIX: DOMAIN_SCALE and LARGE_SCALE are incompatible so catch this
before ./cofigure fails [3]

[1] "Obscure" = it took me about 4 hours and a 500 queue processed one by
one by hand to catch and reproduce the problem after 7 days of mail 
processing w/o error
[2] Pointed out by vanilla@
[3] Pointed out by Arvinn L<>kkebakken <arvinn@whitebird.no> for mail/dspam


###########################################################################
# dspam-3.4.5
#

A lot of things have changed and your existing dspam.conf won't probably
work anymore. Please compare it with the dspam.conf.sample located in the
same etc/ directory.

Most notable addition is the SMTP delivery which together with "standard"
inbound LMTP make a very nice content and light (e.g. it's the fastest setup
I've had so far) filter.

Database structure is roughly the same but you may wish to check your
existing database structure with the *.sql corresponding to you database
driver from EXAMPLESDIR (Mysql 4.1.x is now default as it's _much_ faster).
Please read carefully distributed docs like README and UPGRADING.

#############################################################################
# dspam-3.2.8
#

- BUGFIX: fixed TOE autowhitelist


#############################################################################
# dspam-3.2.7
#

- BUGFIX: Signature not written to some email, making them untrainable


#############################################################################
# dspam-3.2.6.20050128.1500
#

- disable ORACLE DB back-end; it's unmaintained and hasn't work on FreeBSD
- (added post-factum) Fix mysql/neural-combination
- (added post-factum) change POSTGRESQL* to POSTGRESQL because of "Split the 
postgresql ports into a server and a client part"


#############################################################################
# dspam-3.2.6
#

- In v3.2.6, the default level of statistical sedation has been changed from
5 to 0 (off). This feature is normally overridden by dspam.conf, but if you
have removed the feature from the default configuration, this may change
DSPAM's behavior. To ensure that nothing changes in your filter's behavior,
make sure the following line is in your dspam.conf:
Feature tb=5

- fix for statisticalSedation ignore


#############################################################################
# dspam-3.2.4
#

- drop WITH_WHITELIST, is not supported by ./configure but by dspam.conf

[20041203.0800] jonz: performance fixes for pgsql_drv
minor performance fixed for pgsql_drv that may have a big effect on some
implementations. you should also consider creating the (unnecessary) index
below to precent the pgsql query builder from getting confused:
CREATE INDEX id_token_data_04 ON dspam_token_data(uid);


#############################################################################
# dspam-3.2.3.20041229.1945
#

- drop QUARANTINE_AGENT as is not supported by ./configure anymore and
set in dspam.conf UntrustedDeliveryAgent and QuarantineAgent to default to
TrustedDeliveryAgent, but leave them commented out
(ports/75548 by Julien Gabel)
- change default DSPAM_HOME_MODE to 0770
(ports/75549 by Julien Gabel <jpeg@thilelli.net>)
- DSPAM_HOME_{OWNER,GROUP} default to DSPAM_{OWNER,GROUP}


#############################################################################
# dspam-3.2.3.20041218.2100_1
#

- fix installation of SQLite docs and implicitly package 
(reported by: pointyhat via kris@)

#############################################################################
# dspam-3.2.3.20041218.2100 (this is really 3.2.3 release)
#

- change of name: SQLITE --> SQLITE2, as the -devel port already have sqlite3
support
- fix deinstall to delete dspam.conf if it hasn't been cutomized, as 
requested by kris@ with "Ports Janitor" hat :)
- correct long-standing copy/paste bug DSPAM_{GROUPD,MODE} where set to
DSPAM_HOME_{GROUP,MODE} 

#############################################################################
# dspam-3.2.3.20041203.1245_1 (entry added post-factum, it was an automated 
#                            update approved by me but not under my control )
#

- chase dependecy for sqlite

#############################################################################
# dspam-3.2.3.20041203.1245
#

- now we support also MYSQL50 and we handle MYSQL cases right based on a 
patch sent by Gea-Suan Lin <gslin@netnews.NCTU.edu.tw>
- add support for exim as suggested by 
Shahbaz Javeed <sjaveed@adderpit.com>; we try to build exim with the same
database back-end if possible (still work to do to automate pg 7.3 case)
- fix PREF_EXT and BDB4 cases as well in our Makefile (actually is 4.1 and 
I'm thinking about dropping support for it since SQLite works a lot better if
you don't want a full sql server; feedback welcome)
- also small vendor bug-fixes (pgp signed mails, pgsql driver)

#############################################################################
#  dspam-3.2.3
#

- important fixes from 3.2.2:
 BUGFIX: DSPAM misreads boundary delimiter in signed Apple Mail messages
 BUGFIX: Signature is not written to multipart blocks with no content type
 BUGFIX: dspam_dump for a particular token fails with MySQL 4.1
 BUGFIX: Bayesian Noise Reduction is never instantiated

#############################################################################
#  dspam-3.2.2.20041117.1515:
#

- dspam configuration is now done via dspam.conf

- please remove any OPTION

- the following OPTIONS should be set now in dspam.conf
OPTION                  .conf name                      def. value
-------------------------------------------------------------------------------
DSPAM_HOME              Home                            /var/dspam
WITH_*_LDA              TrustedDeliveryAgent            mail.local
                        UntrustedDeliveryAgent          /usr/libexec/mail.local d %u
QUARANTINE_AGENT        QuarantineAgent                 #

WITH_SPAM_SUBJ          Preference "spamAction=tag"     #
                        Preference "spamSubject=SPAM"   #
USER_LOGGING            UserLog                         on
SYSTEM_LOGGING          SystemLog                       on
WEBMAIL                 TrainPristine                   off
WITH_OPT_IN             Opt                             out
SAT                     TrackSources spam ham           #
PARSE_TO_HEADERS        ParseToHeaders                  on
BROKEN_MTA              Broken lineStripping            #
BROKEN_ERR_CODES        Broken returnCodes              #
SIGNATURE_HEADERS       Preference "signatureLocation=  message" # headers

- the following OPTIONS should does not exist anymore:
SIGNATURE_ATACH WITH_HOMEDIR_DOT

- the following OPTIONS have their name changed to follow vedor names
TRAD_BAYES              GRAHAM_BAYES
ALT_BAYES               BURTON_BAYES
WITH_SIGNATURE_LIFE     SIGNATURE_LIFE

- new OPTIONS
OPTION                       port default
----------------------------------------------
LOG_DIR                         /var/log/dspam
WITH_USER_HOMEDIR             
DSPAM_HOME_OWNER
DSPAM_HOME_GROUP
DSPAM_HOME_MODE
DSPAM_MODE
DSPAM_OWNER
DSPAM_GROUP

To upgrade you should:

0. Read docs enclosed with teh dpsam distribution, beginnig with UPGRADING and
README

1. Adjust your OPTIONs
( make rmconfig config in dspam port dir, command line, pkg_tools.conf )

2. make VARs.......
in dspam port dir with appropiate OPTIONS and VARs (see changes above)

3. stop you MTA or deactivate dspam from it

5. make VARs........ install

4.a) If using MySQL 4.0.xx or PostgreSQL:
- back-up the database
- use the appropiate migration scripts from files dir of dspam port dir
( for MySQL you should use 2x_to_3x_db.sql then 310_to_320.my.sql)
(if using MySQL 4.1.x or 3.xx you're on your own for now; the same for 
PostgreSQL, but take a look in the UPDATING and 310_to_320.pg.sql)

5. Back-up and them move the files from the old DSPAM_HOME 
(defaulted: /usr/loca/etc/dspam) to the new DSPAM_HOME (default /var/db/dspam) 
or where ever you defined DSPAM_HOME in 2 above.

6. Adjust you new dspam.conf according with your setup OPTIONS and VARs defined
on make / mak install step.

7. Remove the old setup files (you have them in the back-up, right ?)

8. Enable dspam in the MTA setup, but only locally or something and start testing.

If something ain't working is one of:
- permissions problems (hint: with what UID 
is dspam called from your MTA vs. DSPAM_HOME and DSPAM_LOG_DIR)
- something wrong in dspam.conf (hint: you didn't remove old config files 
and they are still used ?)