Re: update on netsecdb project
IMHO just having blacklists based on IP addresses is not enough: * rapidly increasing mobile internet (which has dynamic IPs unless one keeps a connection open indefinitely - hardly ever the case) * tendency to reuse one bot for an ever decreasing number of spam messages - so blacklist are and always getting to be less helpful. We did an analysis of a commen DNSBL and found that only 3% of active bots could be found there at the timepoint when they were active. Roughly the same number (6%) can be got when comparing with originating IPs from incoming spam. If spam volume is sinking - and it definitely does at least for me - this has nothing to do with any countermeasures but is probably just a delayed effect from the economic crisis. Let's not delude ourselves here. Actually, our paper on automating botnet tracking was downloaded quite often (we got a mail from Computers & Security / Elsevier that it was among the top 25 downloaded paper in Q4/2009 - whatever that means ;-) so there seems to be a lot of interest in tracking bots with more intelligent techniques. My opinion was and still is that we need to automate detection and tracking techniques and not necessarily rely on old obsolete filtering techniques (although they can be helpful in some cases). But I see the limits of RIPE to make such an approach happen and frankly I don't see any other supranational organization that can pull that off. So here's to hoping the spammers die out from the current crisis and we can switch off all our spamfilters... Best, Alex -- Dr. Alexander K. Seewald Seewald Solutions www.seewald.at Tel. +43(664)1106886 Fax. +43(1)2533033/2764
Hi,
IMHO just having blacklists based on IP addresses is not enough: * rapidly increasing mobile internet (which has dynamic IPs unless one keeps a connection open indefinitely - hardly ever the case) * tendency to reuse one bot for an ever decreasing number of spam messages - so blacklist are and always getting to be less helpful.
most blacklist do not care to block a person, they block IPs. If those IPs are dynamic, its up to the provider how to deal with that.
We did an analysis of a commen DNSBL and found that only 3% of active bots could be found there at the timepoint when they were active. Roughly the same number (6%) can be got when comparing with originating IPs from incoming spam. If spam volume is sinking - and it
Hm, I doubt the result, we block every bot that sends spam to our customers easily. Any dynamic IP is just sending as ONE spam and then never again until a the provider starts to do something. I doubt your results. They are probably based only on open blacklists.
definitely does at least for me - this has nothing to do with any countermeasures but is probably just a delayed effect from the economic crisis. Let's not delude ourselves here.
Actually, our paper on automating botnet tracking was downloaded quite often (we got a mail from Computers & Security / Elsevier that it was among the top 25 downloaded paper in Q4/2009 - whatever that means ;-) so there seems to be a lot of interest in tracking bots with more intelligent techniques. My opinion was and still is that we need to automate detection and tracking techniques and not necessarily
You seem to be too scientific here. Its that easy to track every bot, specially for the access providers, if their own IPs get abused.
rely on old obsolete filtering techniques (although they can be helpful in some cases). But I see the limits of RIPE to make such an approach happen and frankly I don't see any other supranational organization that can pull that off.
So here's to hoping the spammers die out from the current crisis and we can switch off all our spamfilters...
I disagree here. Access provider somehow have to be force to blocked customers with infected PCs. This should be done via the community rather than countries goverments. If will only end in useless methods, if goverments get involved. I still not get how RIPE can accept criminal or even ignorant members. Criminality cant be part of the "free internet". Kind regards, Frank -- PHADE Software - PowerWeb http://www.powerweb.de Inh. Dipl.-Inform. Frank Gadegast mailto:frank@powerweb.de Schinkelstrasse 17 fon: +49 33200 52920 14558 Nuthetal OT Rehbruecke, Germany fax: +49 33200 52921 ====================================================================== Public PGP Key available for frank@powerweb.de
Best, Alex -- Dr. Alexander K. Seewald
Seewald Solutions www.seewald.at Tel. +43(664)1106886 Fax. +43(1)2533033/2764
On Tue, Apr 06, 2010 at 03:57:30PM +0200, Frank Gadegast wrote:
most blacklist do not care to block a person, they block IPs. If those IPs are dynamic, its up to the provider how to deal with that. I'm just noting that almost all of the spam nowadays is sent out from dynamic IP addresses by bots and that these are used to send out a number of spams per bot which tends to decrease (obviously to make detection and blocking harder). These finding are not in dispute in the community AFAIK.
I doubt your results. They are probably based only on open blacklists. We've used SpamHaus XBL which specifically targets bots. I don't see how a non-open blacklist could be used in a scientific paper as nobody would be able to check the results - anything could be claimed. Our claims have been verified by peer review, been published in a prestiguous journal and been quite popular (at least according to download counts, provided these have been correctly counted by Elsevier).
Its that easy to track every bot, specially for the access providers, if their own IPs get abused. Indeed. But since some access providers make money off lots of bot
So here's to hoping the spammers die out from the current crisis and we can switch off all our spamfilters... I disagree here. So you want the spammers to survive? One of the hindrances in my work has been that - because spam filters work extremely well and the costs of FPs are easily overlooked - a lot of companies profit from the status quo: not only spam filter companies, but also free email services, ISPs with traffic-dependent fees (mobile may be upcoming here), anti-virus companies, IT security firms etc.. They are not interested in a permanent solution and indirectly contribute to a
One reason for a seemingly good performance in detecting bots via blacklists could be if you blocked whole network ranges instead of single IPs. This would make it possible to block whole ISPs (mostly those who don't care about bots in their ranges), but also significantly increases the FP rate by blocking legitimate traffic. Not all users from "bad" ISPs are necessarily "bad" themselves. Hidden costs of such as system can be quite high and are costly to analyze. I'm still taking a few hours each month to manually analyze a random sample of incoming spam for false positives but very few companies do. In fact when I worked for a spam filter company for a few months and did the same for a 24h sample, I found out that their actual FP rate was ten times(!) higher than their previously estimated value based on explicit customer feedback. Feel free to read our paper and download our systems, run them on your own data and check our results. traffic, it might be hard to convince them to stop this. If we have to wait till all access providers have software to detect bots, we are likely to wait a long time... prolonging of the current situation. I had some first-hand encounters with this mindset.
I still not get how RIPE can accept criminal or even ignorant members. Criminality cant be part of the "free internet". If it cannot, it can no longer be free. There is a price to pay for freedom and it is exactly that. Also, AFAIK RIPE was never designed for that and has no legal way to enforce their rules even if they wanted to. You can't expect a technical governing body to take the role of world internet criminality police without additional resources. It would be far too much like legislation, judgment and execution in one organization, and that's clearly _not_ my definition of freedom.
Best, Alex -- Dr. Alexander K. Seewald Seewald Solutions www.seewald.at Tel. +43(664)1106886 Fax. +43(1)2533033/2764
On Tuesday 06 April 2010 15.10, Dr. Alexander K. Seewald wrote:
IMHO just having blacklists based on IP addresses is not enough: * rapidly increasing mobile internet (which has dynamic IPs unless one keeps a connection open indefinitely - hardly ever the case) * tendency to reuse one bot for an ever decreasing number of spam messages - so blacklist are and always getting to be less helpful. spamblocklists works often on whole ranges. MIne does ( i block the whole range assigned by ripe/apnic/arin for ISP's that allow spam to flow out of their nets)
But i agree; blocking single ip's does not help.
<snip>
So here's to hoping the spammers die out from the current crisis and we can switch off all our spamfilters... I wouldn't bet on this. As long as law-enforcement does not hunt spammers we will continue to have the spamproblem.
What if spam would be characterized as terrorism, then maybe ISP had to act ...
Best, Alex
-- Peter Håkanson There's never money to do it right, but always money to do it again ... and again ... and again ... and again. ( Det är billigare att göra rätt. Det är dyrt att laga fel. )
participants (3)
-
Dr. Alexander K. Seewald -
Frank Gadegast -
peter h