New RXL plugin

Mar 23 2011

We have created the RXL plugin. The RXL plugin implements RBL (Realtime Blackhole Lists) and RWL (Realtime Whitehole List). These techniques work using the same technology. The difference between then is the score asigned to rules searching in RBL and/or RWL. RBL rules should present a positive score and RWL a negative one.

The technology under the Realtime Lists is basically the DNS (Domain Name System). In order to check if is in the list DNSWL.ORG ( we resolve the domain according with the instructions provided by DNSWL.ORG. The queryes to other lists are the same but replacing the suffix. For instance, to check the server in the SpamHaus ZEN list we use the suffix ( Check this list of RXLs

We are now working in the development of a cache for the queryes. The documentation also should be updated consequently.

You can now call to rxl_check and rxl_check_octect in order to develop your rules. rxl_check(<list_suffix>[, <number_received_heder>]) and rxl_check(<list_suffix>, <octect number>, <octect value>[, <number_received_heder>]). For instance:

header IN_ZEN_SPAMHAUS rxl_check("", 3)

header IN_SBL_SPAMHAUS rxl_check("", 4, 2, 3)
describe IN_SBL_SPAMHAUS according SpamHaus doc is in SBL when result is

header IN_DNSWL rxl_check("", 3)
score IN_DNSWL -3

SPF is now working

Mar 09 2011

SPF is now working. We are now including SPF response caches in order to reduce time. The idea behind cache is avoiding computing the same SPF response for an email 5 times. We had used the same scheme in Bayes plugin.

HTML part parsing is now included in WB4Spam

Mar 07 2011

We had completed HTML part parsing functionality in Wirebrush. Bayes processing has increased speed due to the removal of HTML tags considered as tokens in last versions.

Wirebrush will be presented at IPL

Mar 04 2011

Wirebrush will be presented on March 15 at IPL, Leiría (Portugal). David, Noemi and Moncho will explain to computer science students the main advantages included in Wirebrush to improve speed on Spam Filters.

The parsing of HTML parts is near

Mar 04 2011

Parsing HTML is now possible. Next Monday we get wirebrush parsing HTML parts. The HTML parser is again a finite state machine so it will run fast.

SPF and RelayList plugins comming

Mar 04 2011

The SPF and RelayLists plugins are comming soon. We believe in get it working on next monday.

More improvements on EMLStructureParser

Feb 24 2011

EMLStructureParser is now able to dump text from text/* parts. We are finishing the code. Naïve Bayes will be able to take advantage from this feature.
The extracion of text from an email is only done 1 time. We use a caching scheme to avoid computing it a lot of times.
We also cheked some errors during the parsing to detect malformed rfc2822 files.
And all of this is made using a finite-state machine scheme with a stack when parsing body parts.

Bayes Cache

Feb 22 2011

Imagine a filter containing more than 1 call to check_bayes function provided in bayes plugin (for instance 9 rules similar to those included in Debian or Ubuntu SpamAssassin filters). Every time check_bayes function is called, the probability of the email being spam is computed involving some searching processes on a berkeley database. This situation involves a lot of unneeded computational process. Why compute the same probability 9 times?. In order to solve this problem, we had implemented a small configurable cache. Using a cache able to store 1 element, the problem is avoided. Moreover, the cache can be useful to get classified the same emails delivered to more than one addresses in the same domain (filtered with the same filter).
Now, a filter with 9 bayes rules is executed in less than 35 miliseconds. Really fast. I want to congratulate us :).

Parsing MIME parts

Feb 22 2011

Yesterday, we had completed the development of MIME part parsing. Now we will work on improve tokenizing for Naïve Bayes in order to avoid tokenising of non-text parts.

The memory leaks day

Feb 22 2011

Yesterday we had worked in solving some memory leaks and getting wirebrush more stable. We are now working in solving some final issues about free some dinamic memory.

« Newer posts Older posts »