Stephan Hermann <sh@nwu.de> wrote:

At 02:43 01.10.97 +0200, Geert Jan de Groot wrote:

...

People may be interested to read http://maps.vix.com. It is my understanding

I think, it's not a solution to build a frontier to the ISPs who are housing such spammers. We must stop those spammers with commercial ideas not with technical solutions such as filtering out IPs with as-path access lists.

Um, please read how the list works - it doesn't use as-path access list. It sends /32 routes (normally) for specific hosts which orginate spam, and transiently for specific relays currently being used to propogate spam. Sure it's no defence, but every 3rd party (and, in one instance, a customer - tut tut) who has been on this list and complained about lack of connectivity to my network has since fixed their mail relay not to forward spam (we take the feed). It's very effective at reducing the amount of spam you get (at least for those zones which don't have topologically distant backup MX).

One (technical) idea can be, to install two smtp server:

All this does is stop you relaying. You can do this on one server with the no relay patches on http://www.sendmail.org/ if you can get the IP address stuff to work right, though we use two servers for other reasons. -- Alex Bligh GX Networks (formerly Xara Networks)

...

...

I think, it's not a solution to build a frontier to the ISPs who are housing such spammers. We must stop those spammers with commercial ideas not with technical solutions such as filtering out IPs with as-path access lists.

Um, please read how the list works - it doesn't use as-path access list. It sends /32 routes (normally) for specific hosts which orginate spam, and transiently for specific relays currently being used to propogate spam.

Well...how can I filter hosts out, which are connected dynamically. The most spams I get, are from several IPs which are dial-up customers of (well known) ISPs in USA.

I'm going to describe once again how were dealing with spam. We've installed sendmail (latest) with several patches, namely the no relay patch (unless the host is listed as allowed to use the relay) and also patches for checking the existence of the From and To addresses. We have a daemon on the background scanning the mail log. We accept mail from everywhere, anyplace. As long as its one message at a time or within a reasonable interval. If a spam is detected (several incoming messages from the same domain, or from "weird domains" like cyberpromo.com, 344234.com, etc we just use the packet filtering capabilities of Linux to refuse packets from the incoming IP address doing the spam. About 15 minutes later the daemon cleans up the IP filters in existence and rescans the mail log. This means that an ongoing spam will be continually blocked. It also means that instead of refusing the message, or sending back an error, or whatever, the spammer thinks that the server is out of reach, the network is not working, the server is not responding, etc stopping him from retaliation measures like we had in the past from spammers who got mad with us stopping them with other techniques. Mail from every domain is always accepted, even from well known spammers. They are able to get messages through, as long as they arent patterned. If they are they will be able to send 4 or 5 of them, but them the server will be unreachable. C U! -- Mario Valente

At 14:49 01-10-1997 +0200, IBS / Andre Oppermann wrote:

Can you make this code public (for RIPE ISP's - BSD license or GPL)?

Mario Valente wrote: -snip-

...

We have a daemon on the background scanning the mail log. We accept mail from everywhere, anyplace. As long as its one message at a time -snip-

OK find at the end of this message the shell scripts that we currently use to block spams dynamically. Understand that this is a work in progress, begun a couple of months ago, tailored to our system. As such its not configurable; and its not optimized. It should probably be rewritten in Perl or C, commented, etc It was partly written by me and partly by my cohort Paulo Laureano. We have commented the code just now to help in understanding it. Some variables and files are named in Portuguese :-) Paulo also wrote some of the comments below. Since several people have been asking us for these scripts, we are now thinking of rewriting this, comenting and optimizing. C U! -- Mario Valente & Paulo Laureano (looking over my shoulder) ------------------------------------------------------ This is the main script I run on the cron every 5 minutes... the value of the "tail" numbers of lines to read should be adjusted for your own system since this sets the amount of time a IP address is left with denyed access to port 25 of your machine. The values of 1000 & 2000 used on my server provide suspensions of access for 15/30 minutes depending on the reason that caused the cut (i.e. the script checks the last 5 minutes and cuts access to spammers it found for the next 15/30 minutes). --- cut here (begin /usr/local/bin/lockrelayers) --- #!/bin/sh # script file used for real-time cut of access to port 25 # must be "chmod +s"... # # # Clean up list of commands that will deny access to port 25 # :> /usr/local/bin/cortados.new # # temporary file used to store IP addresses of spamming hosts :> /tmp/spamlist # find relayers and cut access to port 25 on ilegal ones (temporary! # just to break the flow of incomming messages) # # # findspamrelayers addresses using our server as a relay (see below) # /usr/local/bin/findspamrelayers_auto | cut -f3 -d"[" | cut -f1 -d"]" | sort -u | while read endereco do if grep $endereco /usr/local/bin/cortados >/dev/null # Address already in the list of denied addresses then echo "Relay access already cuted..." >/dev/null else # Add address to the list of addresses to deny echo $endereco >>/tmp/spamlist fi done # find domains started with numbers (i.e. ilegal) and # 1- add them to list of known spammers (block'em in the future # based on the domain name)... # 2- deny access to port 25 for the machine that is delivering the # messages (this is temporary just to break the flow of messages) # # # find in the maillog fakedomains (started with a digit) tail -1000 /var/log/maillog | grep "from=" | cut -f2 -d"<" | cut -f2 -d"@" | cut -f1 -d">" | grep ^[0-9] | sort -u | while read fakedomain do if grep $fakedomain /etc/mailspamdomains >/dev/null then # Fake domain already blocked echo $fakedomain is already blocked here >/dev/null else # Add domain to /etc/mailspamdomains, used by sendmail to stop spammers echo Added $fakedomain to domains blacklist echo $fakedomain >>/etc/mailspamdomains fi # Add fake domain to list of addresses to block, since they're currently spamming grep $fakedomain /var/log/maillog | grep "from=" | cut -f3 -d"[" | cut -f1 -d"]" | grep "." >>/tmp/spamlist done # cut access to port 25 (temporary!) on anyone atempting to deliver # messages from our list of known spammers... cat /etc/mailspamdomains | while read spamdomain do # Find IP address of known spammers tail -2000 /var/log/maillog | grep "from=" | grep $spamdomain | cut -f3 -d"[" | cut -f1 -d"]" | grep "." | while read foundip do # Add IP address of known spammers to list of addresses to block echo $foundip >>/tmp/spamlist done done # sort/unique the list of IP's to block so far and add them to the # list about to loose access to the mail port... sort -u /tmp/spamlist | while read endereco do # Create the script with list of commands used to block addresses # denyaccess is a script (see below) # echo /usr/local/bin/denyaccess $endereco >>/usr/local/bin/cortados.new done rm -f /tmp/spamlist cp /usr/local/bin/cortados.new /tmp/spamlist # if we have three or more IP's blocked, lock ALL ip's that we know # relay spam mail to us (have done so in the past...)! This literally # makes esoterica unreachable to loads of people for a while and makes # spam close to impossible by relaying mail thru major ISP's. We only # lock the entire list on the third IP locked to allow space for a # couple of "ilegal" relaying (some new customer not yet known to the # mail postmaster, etc). cat /tmp/spamlist | grep "\." | while read nome do let quantos=$quantos+1 echo $quantos >/dev/null if test $quantos -eq 3 then cat /usr/local/bin/cortados.relay >>/usr/local/bin/cortados.new fi done # cut the access and log it excluding from the log the big list of # IP's cuted because they are know relayers and the list of IP's we # have cuted on a permanent basis... # Run the fixed script (see below) that blocks known spammers /usr/local/bin/cortados >/dev/null # # Put date into log date >>/var/log/maillocked # # Run the dynamic (previously created) script that will block current spammers /usr/local/bin/cortados.new >/dev/null 2>/dev/null # # Use the Linux ip firewall admin command to list the current blocks /sbin/ipfwadm -I -l -n | grep tcp >/tmp/spamlist # Take out of /tmp/spamlist the domains that are always blocked grep denyaccess /usr/local/bin/cortados | cut -f2 -d" " | while read defcut do grep -v $defcut /tmp/spamlist >/tmp/spamlist2 mv /tmp/spamlist2 /tmp/spamlist done # Take out of /tmp/spamlist addresses known as using us as relay # and log the rest of addresses (those discovered in this run of the # script) grep denyaccess /usr/local/bin/cortados.relay | cut -f2 -d" " | while read defcut do grep -v $defcut /tmp/spamlist >/tmp/spamlist2 mv /tmp/spamlist2 /tmp/spamlist done cat /tmp/spamlist >>/var/log/maillocked echo >>/var/log/maillocked ---- cut here (end /usr/local/bin/lockrelayers) ---- The script file "cortados" that follows has a list of addresses permanently blocked from delivering mail to Esoterica ; ---- cut here (begin /usr/local/bin/cortados) ---- /sbin/ipfwadm -I -f # exceptions (addresses that are never cuted down; my relay mail machine) echo 194.130.254.3 >/dev/null echo 195.22.0.33 >/dev/null # The script denyaccess is used (see below) #cyberpromo/savetrees /usr/local/bin/denyaccess 204.137.223.0/24 /usr/local/bin/denyaccess 204.137.222.0/24 /usr/local/bin/denyaccess 204.137.220.0/24 #regulus.net/bulk-e-mail.com/nancynet.com,etc e um ISP para spammers... /usr/local/bin/denyaccess 205.199.4.0/24 #mail-response.com/nancynet.com/nevwest.com/etc,etc,etc /usr/local/bin/denyaccess 205.254.167.0/24 /usr/local/bin/denyaccess 205.254.165.0/24 /usr/local/bin/denyaccess 207.51.48.0/24 #1stfamily.com /usr/local/bin/denyaccess 208.15.229.0/24 #kustom.on.ca /usr/local/bin/denyaccess 204.101.226.0/24 #onlinebiz.net /usr/local/bin/denyaccess 205.164.68.0/24 #netrecruiters.com, uniquepo,com, etc /usr/local/bin/denyaccess 205.198.78.0/24 #asianinvestments.com.au /usr/local/bin/denyaccess 203.32.208.0/24 #spamrelay.grandbikes.com /usr/local/bin/denyaccess 208.219.218.0/24 ---- cut here (end /usr/local/bin/cortados) ---- The file /usr/local/bin/cortados.relay has a list of IP's/pools that in the past were used as relay to deliver junk mail to us. These addresses are ALWAYS blocked on our secondary mail server. This is done because if/when they were denied mail delivery to the primary mail server, the spam would get delivered to the secondary. This script runs if we are being bombed from three or more IP addresses. We cut these down for a couple of minutes also (spammers have a limited number of IP's that they can use for relay, and we cut those in a block whenever we know about them). Since all cuts on the main mail machine are temporary there is no problem on making mistakes... it will delay delivery to the next mail queue processing... only these intervals make mail bombing close to impossible! ---- cut here (begin /usr/local/bin/cortados.relay) ---- /usr/local/bin/denyaccess 12.70.46.0/24 /usr/local/bin/denyaccess 12.70.47.0/24 /usr/local/bin/denyaccess 128.148.157.0/24 /usr/local/bin/denyaccess 128.163.1.0/24 [ ... big list of address pools including machines from uunet and other big ISPs frequently used as relay for spam ... ] /usr/local/bin/denyaccess 208.206.112.0/24 /usr/local/bin/denyaccess 208.206.176.0/24 /usr/local/bin/denyaccess 208.6.192.0/24 /usr/local/bin/denyaccess 209.1.135.0/24 /usr/local/bin/denyaccess 209.30.0.0/24 /usr/local/bin/denyaccess 209.68.1.0/24 /usr/local/bin/denyaccess 209.75.5.0/24 ---- cut here (end /usr/local/bin/cortados.relay) ---- The file /usr/local/bin/cortados.new is empty and has the executable bit active (i.e. it is a script with content filled in real time) and called "at the end" of "lockrelayers". It is filled dynamically with sequence of commands to block addresses. A script used to find relayers by "lockrelayers" is "/usr/local/bin/findspamrelayers"... content follows... ---- cut here (begin /usr/local/bin/cortados.relay) ---- # Find entries in maillog telling of relay use grep relay /var/log/maillog >/tmp/xpto2 tail -600 /tmp/xpto2 >/tmp/xpto # Extracting friendly virtual domains from the list of relayers ... those that # we allow relaying and do mail forwarding # cat /etc/sendmail.cw | grep -v ^# | grep "\." | while read nome do grep -v $nome /tmp/xpto >/tmp/xpto2 mv /tmp/xpto2 /tmp/xpto done # # Extract domains for leased line customers and expanded # addresses from the list of relayers # # cat /etc/legalrelay | grep -v ^# | grep "\." | while read nome do grep -v $nome /tmp/xpto >/tmp/xpto2 mv /tmp/xpto2 /tmp/xpto done #echo "Extracting known spammers (we already filter) from the list..." cat /etc/mailspamdomains | while read nome do grep -v $nome /tmp/xpto >/tmp/xpto2 mv /tmp/xpto2 /tmp/xpto done # Separate maillog relay entries into two lists, to find out # those that are currently relaying (have both a From entry and # a To entry). Those that dont have both, are either local deliveries, # locally originated or are coming from known spammers and were # not delivering them (no To:) # cat /tmp/xpto | grep " from=" >/tmp/froms cat /tmp/xpto | grep " to=" >/tmp/tos # Output on stdout addresses that are in both lists (and so are # currently relaying illegaly). The stdout will be used by other scripts # cat /tmp/tos | if grep " " >/dev/null then cat /tmp/tos | cut -f7 -d" " | while read msgid do grep $msgid /tmp/froms done else cat /tmp/tos | cut -f6 -d" " | while read msgid do grep $msgid /tmp/froms done fi #Cleanup rm -f /tmp/xpto2 rm -f /tmp/xpto rm -f /tmp/tos rm -f /tmp/froms ---- cut here (end /usr/local/bin/findspamrelayers_auto) ---- To examine the logs on my system I run from the comand line the following scrip called "viewspam" (every day to check spamming atempts of the last hours)... it requires the "/etc/mailspamdomains" file to determine what spammers to look for. ---- cut here (begin /usr/local/bin/viewspam) ---- cat /etc/mailspamdomains | while read nome do if grep $nome /var/log/maillog >/dev/null then echo $nome echo "------------------------------------------------" grep $nome /var/log/maillog | if grep " " >/dev/null then grep $nome /var/log/maillog | cut -f7 -d" " | while read msgid do grep " $msgid " /var/log/maillog echo done echo else grep $nome /var/log/maillog | cut -f6 -d" " | while read msgid do grep " $msgid " /var/log/maillog echo done echo fi fi done ---- cut here (end /usr/local/bin/viewspam) ---- The denyaccess script that cuts access (/usr/local/bin/denyaccess) is; ---- cut here (begin /usr/local/bin/denyaccess) ---- # Deny TCP packets coming from source $1 into dest "Our mail server" /sbin/ipfwadm -I -i deny -P tcp -S $1 -D 195.22.0.135 25 >/dev/null 2>/dev/null # Same for UDP /sbin/ipfwadm -I -i deny -P udp -S $1 -D 195.22.0.135 25 >/dev/null 2>/dev/null # Same for ICMP /sbin/ipfwadm -I -i deny -P icmp -S $1 -D 195.22.0.135 >/dev/null 2>/dev/null ---- cut here (end /usr/local/bin/denyaccess) ---- Hufff... now, I have some sendmail related files that are used to deny access based on domain names on "/etc/mailspamdomains" and a list of legal relayers (leased line customers, alias expansion that does not appear in the logs) on "/etc/legalrelayers". My sendmail locks out delivery from/to domains in "/etc/mailspamdomains". I got the domain based lockout scheme for sendmail from "www.sendmail.org"... I also have installed the checking of domains patch from the experimental anti-spam counter-measures for sendmail; this does reverse DNS lookups to check for validity of From and To addresses (also handy to find out your clients misconfigurations). In short the files are; /usr/local/bin/lockrelayers main script to do real time locking running out of crontab /usr/local/bin/findspamrelay_auto used by "lockrelayers" to find out current spammers /usr/local/bin/cortados permanently cuted IP's used by "lockrelayers" /usr/local/bin/cortados.new new IP's to cut; built and used by "lockrelayers" /usr/local/bin/cortados.relay list of machines used in the past to relay to us... used to lock in a block a lot of paths to esoterica and permanently cuted on our relay mail machines... used by "lockrelayers"... /usr/local/bin/viewspam look at log entries related to spam based on /etc/mailspamdomains /usr/local/bin/denyaccess cuts access to port 25 from and address... used by "lockrelayers" /etc/mailspamdomains list of domains to be cuted by sendmail /etc/legalrelay list of domains/users we allow relay to/from... Ouch... this is the first time I actually atempted to explain to someone the anti-spam measures in place here. If something fail to works as it should just drop me a line and I'll add whatever is needed. Basically this systems does not prevent mailspam but makes it impossible to work (i.e. reach more than the firts few addresses) by allowing only a small time windows of uncontrolled access, cuting access to offenders in a three/six time larger time window, and rendering mailspamdomains unusable for more that one time window. Also it detects when a IP address is atempting relay thru our system and shuts it up for a while, it shuts up knows pools of ips (for a couple of minutes only) used for relay if attacks persist, etc. From what I gathered some spammers are going nuts with this; they even forged mail and placed esoterica on the headers out of revenge. The reason is simple; they start spamming us, or using us for relay from some dial-in on aol/whatever, it seems to work in the first few minutes (some messages may even reach their intended destination) and then... esoterica is no longer reachable... they can't even ping us... then, they try another dial-in, get another IP address and... BINGO, working again for a few minutes, but then it stops working also... on the third atempts things repeat themselfs but then it seems that at the fourth atempt not even new IP's get to esoterica... to make them REALLY MAD everything works again a few minutes later; the problem is it would take hours to deliver mass mailings thru this "less that five minutes" windows; worst than that, on the next atempt mail has to be forged again since fake domains are blocked, etc. Spam received/relayed by esoterica has dropped 99% in the last weeks.

At 13:28 02-10-1997 +0100, Pedro Ramalho Carlos wrote:

At 11:29 01-10-1997 +0100, Mario Valente wrote:

...

I'm going to describe once again how were dealing with spam. [...Mario's description of his bulk-email detector/ip-filter...]

just as a side note, Mario, you seem to be on the Vixie's black hole: in http://maps.vix.com/rbl/current.html :-( ... 195.22.0.135

It looks like one of your mail servers

...

dig -x 195.22.0.135 ptr ... ;; ANSWERS: 135.0.22.195.in-addr.arpa. 38932 PTR brilthor.esoterica.pt.

Thanks for the tip Pedro. It seems that we were blocked because of some spam that was relayed through us back in early September. I do not know if this was because our spam filter was not working at the time or if it was because of a small amount of spam that was relayed (I remind you that our system DOES relay some spam through, until a pattern is detected in the mail relaying or until an address is permanently blocked). Anyway, I've requested from MAPS the removal of our address from the RBL and offered to send them the scripts were using for stopping spam. C U! -- Mario Valente

On Wed, 1 Oct 1997, Stephan Hermann wrote:

One (technical) idea can be, to install two smtp server: One, who can only receive mails for known domains and IPs. This SMTP Server use second relay smtp server for delivering the mails to the customers.

This would require a way to distribute which domains belong to which IP number and this alone is a huge technical and administrative problem which would make it hard to be solved. Considering that each person wants their own domain these days to get a "cool" e-mail/webserver address you will have to have a list of more than 10 million domains and some domains use more than one outgoing mailserver so you will probably have to store at least 8 bytes of address info (in IPv4) per domain. If you also want fast lookups in this database you'll have to add space for some huge indexes (50% extra?). But while we are talking about fundamental technical changes to the SMTP standard, why not simply add an digital signature to each mail your SMTP server guarentees to be spamfree, your public key could be sent out via the DNS and it should be signed by a trusted third party (like RIPE). Each SMTP server that don't want to recieve spam, simply filters away all mail without a signature, with an incorrect signature or those that have a signature from an untrusted SMTP server. To get your public key signed by the trusted parties, you have to sign a contract there you guarentee not to send spam through this SMTP server. Those that use your SMTP server as a relay would obviously have to sign such a contract with you. Old SMTP clients (without spam protection) can still recieve all their mail (they'll not list the signature command in their EHLO response) but they will have to relay their mail via some new SMTP server. Most ISPs use sendmail today and if the support was added to sendmail most internet connected sites could relay their mail through their provider if the provider can authenticate their client. I am far from a legal expert but I don't think there are many countries there it is illegal to digitaly sign a message as long as the plaintext is provided with the encrypted data or it is known how to produce the plaintext and verify that the plaintext and the data is the same. Some facist countries prevent their citizens to export implementations of well known cryptoalgorithms (and to use some algorithms because of different patents) so the sendmail patch would probably have to be implemented in a free country where people are not prevented from exporting the patch or the patch could be based on one of the well known crypto programs like PGP or SSLeay, which in turn would have to be gotten from different places depending on the laws of the target language.

For the first smtp server anyone can use qmail (http://www.qmail.org/), which has an anti-spam filter (checking some to:/from:/sender:-addresses, so the first smtp server can be used as a filter for customer spamming and don't function as relay for spammers.

Even though I myself prefer qmail I must say that it can be added to sendmail quite easily (once you've banged your head against the batbook enough...). There are links to these rules from www.sendmail.org. /Sebastian See http://www.hogia.net/keys/sa-pgp.asc for public pgp key.

On Wed, 1 Oct 1997 davidk@ISI.EDU wrote:

Sebastian,

Sebastian Andersson writes:

...

Some facist countries prevent their citizens to export implementations of well known cryptoalgorithms (and to use some algorithms because of different patents) ...

And some ISPs prevent their customers from making the decision to read what they want and having the ability to receive and send anonymous mail which can be considered an important asset in a truely democratic system ...

They are free to choose another ISP. It's much harder to choose another country. Further I don't know any ISP in any reasonably free country that prevents their users from sending mail to/from e-mail anonymiser but there are too many to keep count. Adding the requirement that it should be possible to see who is responsible for sending a message doesn't invade on anyones privacy. If they want to be anonymous, they send their message through a anonymizing server which they thrust and that server would in turn resign the message (and change the headers and such in the normal way).

ISPs have the responsibility to protect against abuse of their infrastructure (relaying, denial of service attacks) but *not* to make decisions on what mail the customer can receive and what not. That is pure censorship which should only be carried out by the customer itself.

Of course not. Adding a signature to the message that can be used to track who is responsible for the usage of my mailserver does not prevent my users from reading anything except bad messages (those messages that wouldn't follow the SMTP standard which would require signed messages) and they can't read all messages today either for the same technical reason (if the message doesn't follow the SMTP standards it might get dropped/bounced).

There exist many tools to assist to make those decisions. For example, the customers can decide for themselves to reject not properly signed incoming messages and only mail from a selected group of people as a reliable but very extreme measure against unsolicited mail.

And if you use qmail you can send out a personal email address to everyone you want to get e-mail from and only autoanswer to all mail that is sent to your "real" e-mail address. A simple script checks that the destination address is valid for the sending address or it will drop the message otherwice. It is also possible to give out timelimiteded emailaddresses that works for any sender (which in turn is nice for newspostings...). BUT: In theory this would be enough but the internet is used by people that has never had a computer until they decided to connect to the internet. They have bad clients which doesn't support signing nor any filtering. They don't know how to upgrade their clients and they don't care. It is working for them. They get a bit dissapointed when they get to download mail for ten minutes only to find spam but most of them don't complain. Some want us ISPs to "fix it". When you tell them to require signatures and that all their personal friends will have to get a new mailclient they will laugh at you for good reasons. /Sebastian See http://www.hogia.net/keys/sa-pgp.asc for public pgp key.