Month: August 2010

Radio Frequency Exposure (RFE) Calculator

So far in my amateur radio career, I have not been able to offer much that may be of use to other hams. That changes today. A while back, when I was dreaming about where to put my antennas safely, I did a lot of research about radio frequency exposure. I poured over OET Bulletin 65, which details the FCC’s limits on human exposure to RF electromagnetic fields. They have formulas and tables and forms to fill out. It is all wonderful and fine, if you live in the 1960s. Welcome to the 21st Century. We live in a world of computers to do all that number crunching for you. I looked around for any web-based things that would help, but the closest I could find was power density calculator written by W4/VP9KF. This is fine if you want to do it for EVERY band on EVERY transmitter each time you make a change to your station. Plus, it means that I have to transmit all that data to his PHP script, which does the calculations and sends them back. We have this great thing in web browsers called JavaScript, which is more than powerful enough to do the work. I set upon creating a JS-only version of his creation. But it still lacked the memory—I would still need to re-enter for each band for every change. And it wouldn’t let me view multiple bands at once. Bigger calculator!

This is where my offering steps in. My requirements:

  1. Save my data so I don’t have to re-enter everything in every time
  2. Something I can share with others, without saving their data on my server
  3. Let me add, edit, delete at will
  4. Something that can show all my transmitter/antenna/connection information at once

Seems easy enough, right? It was the first two that really got me stuck. I whipped up a little JavaScript ditty that fulfilled number four in very little time at all. Number three was dependent upon the first two and was technically the hardest, but once I had the first two figured out, it was only coding, which I enjoy.

And this is what I came up with: N7OH RFE Calculator. Take it for a spin, share it with your friends. Upon your initial visit, it may not look like much, but if you move over to the “Import/Export” tab, you can press the “Reset to sample data” button and see it in action. Please offer suggestions and comments if you find it to be too difficult to use or see something that might make it better.

As for fulfilling my four requirements, the first two were done once I learned about local storage with HTML 5. This means that your web browser is storing the data. Not as a cookie, but similar. Cookies get sent back to the server with each request. Local storage is meant to be persistent data that a web page can access via JavaScript to be used locally. This means I can save my data on my machine and your data on your machine. I can host the page for everyone, yet not save everyone else’s data on my server. The add/edit/delete requirement was probably the most fun I have had with jQuery to date. And I hardly scratched the surface of what it can do. Lastly, the glory of the Results tab just makes me weak in the knees. Okay, not really, but it is the crown jewel of the whole application. It shows all the stuff you want to know about your radio setup.

Combating SpamBots

The war against spam is ever escalating. Two weeks ago I took my anti-spam tactics to the next level. I want people to be able to post comments to my website without registering. Anonymous comments (or rather unverified authors of comments) should be available if the webmaster sees fit. But I have found that in the past several months that comment spam was getting to be a real problem. I logged in one day and found that there were several hundred spam comments that had gone unnoticed for quite some time. At that time, I did not have any anti-spam measures. I looked around and added a CAPTCHA to the comment form. That stopped most of the spam, but the determined spammers were still getting through.

IP addresses in failed CAPTCHA log Number of failed CAPTCHA responses
212.117.164.8 514
219.252.44.66 250
125.64.96.21 160
69.46.27.117 158
61.145.121.124 138
206.224.254.5 111
212.116.220.224 78
203.77.204.82 78
203.198.126.43 73
211.115.75.169 72
221.214.27.252 69
192.104.18.19 60
218.25.99.135 60
212.116.220.154 54
200.123.147.169 54
95.154.242.207 52
89.233.152.157 50
2700+ other unique hosts <50 hits per host

In the past 2 months, I have logged more than 14,000 failed CAPTCHA attempts. Most the unique hosts have one or two failures, but more than 1,000 unique IP addresses have four or more failures. At some point you have to draw the line and I draw it at four. Or maybe three. One or two failures can easily be done even if a bona fide person is responding. But usually only spambots are dumb enough to get more than three failures.

I can characterize the failures and many of them seem to be of a certain forms: hit twice in rapid succession and then give up for a while. Two hits alone is not usually successful — it usually guesses an empty string or 0 or 1. The problem is if you are using a math CAPTCHA, those can be the right answer. And obviously, if the spambot keeps at it two at a time, it will eventually guess correct and be able to post. I found that the spambot was able to crack several of the CAPTCHAs I offered: ReCAPTCHA, math, word list, word order, etc. Other than ReCAPTCHA, the other ones can be cracked by random entries. I am not sure how they managed to crack ReCAPTCHA. But it was starting to make me angry at all the spam. Finally, in addition to CAPTCHA I resorted to using comment moderation, requiring me to log in and manually approve all comments. I really don’t like this because sometimes I forget. Then the comments get old and people think I don’t care.

I did a little hunting around the Drupal front and found Mollom. This is a nice line of defense against spam. But I read elsewhere that in some cases it wasn’t catching it all. Remember that spambots are in it for the speed and money, so their GET to POST times are very short. I whipped up a little module that checks that. All you super-human typists had better slow down when commenting on my forms. Then I took a page out of Ignacio Segura‘s book and added a honeypot to the comment form to my little module as well. Though you will not see it, (unless you are looking at the html source, reading with a non-CSS compliant browser like lynx, or are a spambot) it is meant to be left empty and will cause a form rejection if it has any text in it.

Then one step more. Because what is escalation if you are not really accelerating? I noticed that once spambots did get in that they usually were ‘advertising’ for companies of ill repute. Offering things like p1Lz and other items to EnH4Nc3 certain parts of one’s body. But in order to get around blacklists for certain words, they intentionally misspell what they are advertising for and also have links to obscurely named domains (which are usually not words either.) I figured any rational thinking human being would spell at least 75% of their words correctly (and that includes things like spambot and acronyms and other non-English shortcuts). So my latest addition to the spam warfare is PHP’s pspell library. So all you spammers out there had better spell it right.

Then as the final blow to spammer (and bad spellers everywhere) I added a “three strikes and you are out” gotcha where if you fail the previous tests more than a given number of times, you will get added to the blacklist. All entries in the blacklist are forbidden to access any part of the website. Permanently. And it seems to work. I have not seen any spam get past the filters in the last two weeks that this has been in effect. Let’s hope this lasts.

I was curious about the actual counts of things, so I whipped up a few SQL queries that gave me the statistics that I wanted. I pushed it all into OOo and came up with this fine chart. There are a couple of things to note:

  • This is about a month of data.
  • The yellow line (number of daily comment spam posts) is on the scale to the right. The other two lines are on the scale to the left.
  • The first day I tried all this stuff out (29 Jul) I didn’t actually have the blacklist implemented, which accounts for no HTTP/403 entries on that day
  • There has been zero comment spam since 29 Jul. It is not for a lack of trying.
  • The blue line shows the number of newly recognized SpamBot IP addresses.
  • The red-orange line shows the number of attempts from previously identified SpamBots that got rejected by the blacklist.
  • I find if quite funny that the HTTP/403 line looks like my server is flipping the bird at the SpamBots. That’s what it is doing…. And no, I did not doctor the data.
  • I see that there seem to be trends or waves of spam. That is fascinating and frightening all at the same time.

Do you do anything to combat spam on your sites? Obviously comment moderation is the only truly perfect filter, but it requires so much work. Especially when I really don’t get that many human comments per day, but loads of spam attempts.

Today ends with Vernon: 15, SpamBots: 0.