Tinkering with anti-spam measures on SEB.

I’ve been flooded with spam user registrations as of late and the WP plugin I had been using to stop them is apparently no longer being updated. So I replaced it with the Stop Spammers plugin from Bryan Hadaway and, hoo boy, does it have a lot of features to it. That means there could be some issues with legit registration attempts by real people.

So, this is just a quick post to say that if you’re a real person who isn’t trying to register an account or leave a comment for spammy purposes and you find that you can’t then drop me an email (over in the sidebar) and let me know and I’ll poke the system with a stick to see if I can’t figure out what needs to be tweaked.

My goal isn’t to stop people from signing up or leaving comments — I’d just turn off registrations and comments if that were the case — but to slow down the bots that are constantly bombarding my site. I’m not even sure what they are registering accounts for because there’s nothing in them to point to someplace else and they never try to leave comments with them and they don’t ever seem to login to those accounts beyond the one time. It seems like a stupid and pointless exercise. They’re not stopping, though, so I’m left with no choice put to put systems in place to cut down on how many get through.

SEB site notes.

Just a quick blurb to mention that I appear to have finally gotten rid of the malware that was randomly inserting Viagra spam into the entries. It took removing every single plugin except for Akismet along with every theme, turning off user registration, and reinstalling WordPress, but I’ve not seen the files show back up since so it appears to have worked.

Now I’m slowly adding plugins back in and watching to see if the spam returns. The podcasts should be playable once more as I put the Blueberry PowerPress plugin back in yesterday, but user registration is still turned off until I test the plugin that helps to cut down on bots making accounts. I had quite a few plugins installed so this will probably take some time and I’ll be evaluating alternatives to many of them while I’m at it.

In the meantime, if the spam starts showing up again be sure to let me know.

Once again I stand on the precipice of a decision.

SEB’s domain name expires on September 9th and I’m debating on whether or not I should renew it. There are a number of factors contributing to this not the least of which is how infrequently I blog these days. It seems my days of writing lengthy posts pontificating on whatever happened to catch my attention are very few and far between compared to the early days of SEB. I don’t know if that’s an indication of having said all I have to say or a growing sense of apathy about the things I used to be passionate about. Most of my entries are throwaways that are barely worth dropping by the site to see. I haven’t even followed through in a decision I made awhile back to practice writing short stories on the blog to develop that skill further.

Further adding to the sense of malaise is the recent escalation of the civil war going on in the Atheist community. For the past year, since at least the events of ElevatorGate, there’s been a raging shitstorm that has come to a head with one faction deciding it’s time to become proto-religious with a new movement they’re calling Atheism+ and they’re demanding people choose which side they’re going to be on lest they decide for them. I’ve seen people I respect in both camps stoop to new lows in the ongoing debate battle over the atheism “movement” and who’s going to control it and, frankly, I’m sick of the whole thing. I’ve deleted all atheism specific blogs from my blogroll (along with many I’m just not reading anymore or that are defunct) and will follow my own fucking path. This isn’t to say that I won’t talk about atheism and religion if I should continue on with SEB, but for the moment I’m too disgusted with the so-called movement to pay them any attention.

The third and final issue that’s prompting this possible abandonment of blogging is the ongoing malware issue SEB has been suffering from. I realize, now, that it was a mistake to move from ExpressionEngine to WordPress. This system is so full of holes and the community is so unhelpful (I had to figure out the WordPress XML import format on my own just to make the transition) that it’s been way more of a hassle than it’s been worth. The templating system is a nightmare, I’ve had to install way more plugins than should be necessary to get it to do what I want, and it seems once the malware gets its hooks into your site getting rid of it is extremely difficult. As you can see, I’ve had to drop to the default template and remove every plugin in my latest attempt to fix things. Things like the SEB Podcasts are now unplayable, most of the anti-comment spam systems are gone (I have reinstalled akismet) which means the moderation queue is filling up fast, and general usability enhancements like being able to edit your comments or the formatting toolbar are gone. All of that and there’s still no guarantee the malware won’t be back. Clearly I’m too much of an idiot for WordPress.

I’ve been at this for ten and a half years now. When I started blogging there were only a few blogs talking about the stuff I cared about. Now there are tons of them and many of them do a much better job of it than I ever have(and are way more popular to boot). I’ve come close to quitting before and I always found a reason to continue on. Now I’m at that point again and I’m doing some major self-reflection to see which decision I should go with.

I suppose I have till September 9th to figure it out.

SEB is now running on WordPress 3.1.

Pic of the WordPress logo.WordPress 3.1 was officially released today and, being the foolhardy fool that I am, I’ve upgraded SEB to it without so much as backing up my database. Mainly because my database is too big for most of the backup options in WP to work. That and the fact that I first tested it on a test installation I have just to be sure it wouldn’t eat the site outright. That said, this is a new version and may have a few quirks and/or bugs. So if you see anything that doesn’t appear to be working properly be sure to let me know.

The one new feature you will probably notice right away, if you are a registered user and are logged into your account, is the new admin bar across the top of the site. I’m not entirely sure what you’ll see as a non-admin, but I believe you’ll have access to your profile, the dashboard, and the ability to quickly submit a new post for possible publication on SEB. Plus it also displays your current Gravatar, if you have one set up (Note: You don’t have to register an account on SEB to make use of a Gravatar).

Beyond that most of the new features/changes are behind the scenes and aren’t immediately obvious even to me. Supposedly there’s a new an easier way to link to pages and posts inside SEB and there’s support for Posts Formats which I’ll probably never understand or make use of.

We’re running on WordPress 3.0 now.

Just a quick note to let you guys know we’re running on the latest release of WordPress. I haven’t had a chance to look over what’s new in this version, though I understand it now supports multiple blogs from just one installation of the script is one of the biggies. I already have all the blogs I’m hosting for friends and family set up as individual installs, though, so I doubt I’ll bother trying to merge them into one Huge Mega Install.

Anyway, it’s a new version and so there may be some issues with plugins or bugs here and there so let me know if you see anything funky.

SEB’s spam filter is being overly aggressive at the moment.

Not sure why, but for the past couple of days several comments left by SEB regulars have ended up in the Akismet spam queue. This happened for a couple days a few months back and then it settled down so I’m hoping it will do the same this time around.

In the meantime if you leave a comment and it gets moderated or tagged as potential spam don’t worry too much about it. I check in on the site throughout the day and I’ll get it approved before too long.

Moving from ExpressionEngine to WordPress in 64 easy steps.

I’ve had a lot of requests for a post on how I moved my ExpressionEngine blogs over to WordPress so here it, finally, is. And, no, it doesn’t really take 64 steps.

The problem with moving from EE to WordPress is that while WP has an impressive selection of importing tools for various platforms built into it, ExpressionEngine isn’t one of the supported platforms. This means we have to do it ourselves and so that’s what I set out to do. You will find in the ExpressionEngine Wiki some information on how to export to the MovableType Format, but this has a couple of limitations that made it less than ideal. For example, I used the SolSpace Tags module in ExpressionEngine and wanted to move all the tags over to WordPress, but the MT format has no facility to do that. A much better  solution would be to export to WordPress’s native WXR format which does support tags, but finding info on how the WXR format is defined (it’s an extension of XML) was more difficult than I thought it would be. It doesn’t appear anyone has sat down and specifically posted what WordPress looks for and accepts in an WXR file.

Eventually I set up a test blog in WP, put in some content that hit all the features I wanted to support, and then did an WXR export to see how the file was set up. I’m far from an expert on the WXR format, but I managed to figure out enough of it to get things to work. My method is similar to the aforementioned MT export templates except that it outputs WXR formatted data instead. You can download the templates by clicking here.

To use them you’ll want to create a new template group called “export” in ExpressionEngine. Then create two RSS templates in that group to hold the templates themselves. I called the first one “export” and the second one “comments.” It doesn’t really matter what you call the first one, but the second one must be named comments so it’ll be embedded properly. If you change the name then be sure to change it in the first template file. Both templates should be set in EE as RSS pages and you should set Allow PHP to “Yes” and “On Input” on the comments template. In the first template on the very first line is the global variable {assign_variable:master_weblog_name=”yourblogshortname”} which you should change to the short name for your blog in EE.

Once you’ve done all of that you’re almost ready to start exporting data. I use the word almost because there’s one thing that I can’t account for ahead of time and that’s the amount of data you are exporting and what RAM limits your server has. There’s also the fact that these templates do not save out to a file. Instead you have to right click on the “view” link in EE’s template listing and select “open in a new tab/window” then click on that new tab and select “show page source” which will open another new window, and then save that new window’s output to an XML file.

Doesn’t matter what you call the files (I named mine SEB1.xml, SEB2.xml, etc.), but if you try to output all of your data at once and you have a crap load of it then your browser will probably crash before you can save the file. On top of all of that, WordPress has a import size limit of 10MBs per file. In the case of SEB, at the time I made the move, I transferred some 6,500+ entries and 75,000+ comments and there was no way in hell it was all going to come out as one great big file so I ended up having to do multiple exports.

Here’s how you do that: In the first template file there’s a line that reads as follows:

{exp:weblog:entries weblog="{master_weblog_name}" dynamic_start="on" limit="9999" offset="0" sort="asc" rdf="off"}

The two key parameters are the limit and offset. The limit sets the total number of entries that’ll be included an in export and the offset tells it at what entry to start at. If you have a very small blog (couple of hundred entries with less than a dozen comments each) then you might be able to get away with a single export using those settings, but if not then here’s where you will have to experiment to find out what you can get away with and it will depend on how many entries you have as well as how many comments on those entries. There is a similar line in the comments template which limits how many comments to include that is set to 1,000 comments. None of the entries on SEB ever hit 1,000 comments so that worked just fine for me, but if you have entries with more than 1,000 comments you may need to edit that template as well. Due to memory limitations on my server I found that I could only export between 100 and 300 entries at a time before EE would abort with an out of memory error. This meant that I ended up exporting some 32 files total to get everything moved over. If I got an out of memory error then I’d go in and change the limit to a smaller number (usually decrementing by 50 each time) until I got a successful export. Once it was successful I’d go back in and change the limit back to 300 and increase the offset by however much the last export put out. Do that as many times as you need to to export all your data.

If that sounds like a pain in the ass, well, it is, but there’s still more that can go wrong. Because you’re generating an XML file your browser can be pretty fucking picky about any weird or garbled characters that might happen to be in your code. SEB was originally on MovableType and in the move from that to ExpressionEngine there were a few entries that ended up with some non-standard characters in them. Whenever an export hit one of those garbled characters it would cause an XML error and I’d have to try and figure out which entry was causing the problem, edit it in EE to fix the garbled characters, and then retry that particular export. I’d say there was about a dozen entries or so that caused me fits, but if you’ve been running on EE all along then this probably won’t be a problem for you.

Now for some good news: WordPress is impressively good at importing those WXR files. If you screw up and forget to change the offset number and end up with a duplicate file (or just one that has a handful of duplicates) WordPress will NOT create duplicate entries in your database. It’ll report those as duplicate entries and reject them. WordPress will also ask you how to handle entries written by people other than the account you are logged in as when doing the importing. You can either reassign those entries to an existing WP user or you can have WP create accounts for those users when doing the import. WordPress is also just fine with importing one file after another to build up your database back to normal.

Some other things to keep in mind with these templates: They make the assumption you’re using the default field names of {body} and {extended} for your blog entries. I also didn’t include {summary} as I never used it myself so you’ll need to add that in if you want it. The code I used for exporting Tags is not included because not everyone uses that module, but I can supply it if you want it. One other thing I should have done and didn’t think of until after I was finished was adding in code to check if the {extended} field existed and if it did to put in the WP code for a Read More link. So for SEB, all the entries where I had a body and extended section became one big posting under WP. If you’re at all comfortable with making EE templates then you should be able to look at these, see how I did things, and tweak them according to your needs.

That’s pretty much the process in a nutshell. Hopefully this isn’t too confusing. If nothing else it should give you a starting point if you want to make the transition yourself. Perhaps someone else will come up with an even better way to do it. Feel free to ask me any questions you have in the comments.