Digitalquill

Over the last couple of months I have been struggling to deal with an increased level of SPAM. I run my own mail server (Debian, Postfix, Courier, ClamAV, SpamAssasin).

About 2 months ago my server was hacked, at that time all I could find in terms of malicious damage was that the root password had been changed and all the other users had been removed from the sudo’ers list.

Since then I have been having great difficulty with the email side of the server, it always seems to have a large number of mails in the main queue, genuine emails are getting spammed (but only on specific addresses), both the volume of SPAM recieved and the amount of SPAM delivered as genuine mail has increased dramatically.

Take Saturday morning for example. I checked my email at around 7:30am and left all mailboxes with no uunread messages, deleting any spam and clearing them up. I then went to one of our properties to do some work and checked my email at around 9:30 and there were 21,000 emails that had been delivered to that mailbox.

My poor HTC Touch could not cope with that volume of emails and while trying to clear those emails it died, since that the phone has not been right, randomly re-booting even thought I have done a hard reset.

It has also come to my attention that emails from some of the feedback and contact forms on my websites are getting spammed and as such the customers for my plugins especially are unable to get support.

There is clearly something wrong with the configuration of SpamAssasin somewhere, but what is the big question. I am tempted to setup a temporary VPS and create a new MX server on that so that I can compare the setup, even move some of the mailboxes across to that, although I would not want that as a permanent solution as I have a high powered dedicated server which I would rather keep as my main server.

It has been said before that the few kill the internet for the rest of us, my server currently handles around 250,000 emails per week, with maybe 1000 of those genuine. The wasted server power and resources costs business a huge amount of money. I am not sure what to do about this, my most immediate action is to try and find out why emails from my feedback and contact forms are being spammed, perhaps I need to add some more headers to the php email.

In my last post I talked about how I setup ClamAV, Squid proxy and Dansguardian on a SheevaPlug. Overall this has been a great success. I have been testing it on my machines before I role it out onto the rest of the network.

I have encountered one problem. When I go to sites such as Google Keyword tool, or I run some of my more complicated scripts that take longer to run it seems to create problems. I am wondering if this is some sort of time out fault that I will be able to change in the setup of Squid or if it is a power issue with the SheevaPlug.

This is not an issue for most normal users, I have tried it on things like YouTube and BBC Iplayer and it works fine with those so I suspect it is something specific to the style of those pages that causes a problem.

I took delivery of my Sheeva Plug Sata Multi today. I bought it yesterday online and it came first thing this morning. Indeed I made a mistake on the order when I bought it with an EU plug so I emailed and then called NewIT who were very helpful and changed over the order without any fuss.

If you remember from my previous post I wanted this device to run a network content filter, Squid and Dansguardian and also to take control of the network I have at home. I therefore, went for the 8GB SD card with Ubuntu 9.04 pre-installed. I will take an image of this before I start to play with it although one can be downloaded from the NewIT website if I need to. I also went for the Sata version so that I can add hard disk space to the device in the future to make it into a NAS drive as well as a network content filter.

First Steps

So my first step was to simply plug the device into my home network, a very simple task of plugging in a network cable and plugging in the power.

I then logged onto my router which currently runs my DHCP and found that a device called Ubuntu had been given an IP address.
Opening up Putty (My preferred SSH client) and ssh’ing to the ip address, I was able to log into the device with the root login and the default password of ‘nosoup4u’

The first impression of the device while navigating around the system via SSH and running the ‘top –d1’ command seemed to be fairly quick, ok the device is not doing anything other than running base Ubuntu but still a good sign at this stage.

I also ran ‘free -m’ which showed 472mb of free memory and df –f to find the free disk space showed:

Filesystem            Size  Used Avail Use% Mounted on
tmpfs                 251M     0  251M   0% /lib/init/rw
varrun                251M   36K  251M   1% /var/run
varlock               251M     0  251M   0% /var/lock
udev                  251M  116K  251M   1% /dev
tmpfs                 251M     0  251M   0% /dev/shm
rootfs                7.4G  443M  6.6G   7% /

All looks in order and I was ready to start playing with the system to see what it could do, but first I wanted to take an image of the system as a backup.

Imaging the SD Disk

I was not sure how I was going to do this so I did a little bit of research for the best method, but I plumped for a simple TAR of the whole system using linux its self:

tar cvpzf backup.tgz --exclude=/backup.tgz /

remembering to exclude the backup file its self otherwise it would get into some sort of loop! The tar of the whole system took only 2 minutes and created a file that was 135 Mb. I then sent this across to my Western Digital MyBookWorld NAS drive which has a hacked Linux operating system on it

scp backup.tgz root@xxx.xxx.xxx.xxx:/shares/internal/backups/

Happy that I had a backup of the system I then went ahead with a system update before trying a setup of Dansguardian.

System update

I ran apt-get update and then apt-get upgrade and upgraded the whole system. It wanted to upgrade the following:

The following packages will be upgraded:
 root@ubuntu:/# apt-get upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
cron dhcp3-client dhcp3-common dpkg file gzip language-pack-en language-pack-en-base libcurl3-gnutls libgnutls26 libkrb53 libldap-2.4-2 libmagic1 libnewt0.52 libpam-modules libpam-runtime libpam0g
  libsasl2-2 libsasl2-modules libsqlite3-0 libssl0.9.8 libvolume-id1 libxcb1 lsb-base lsb-release ntpdate openssl perl perl-base perl-modules sudo tzdata udev wget whiptail
35 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 18.2MB of archives.
After this operation, 1565kB of additional disk space will be used.

This process was again fairly quick taking just 5 minutes most of which was taken downloading the 18mb as Karoo Internet is simply rubbish!
I restarted the device just to check everything was still ok

Shutdown –r now

It took less than a minute to start back up and was back with me.

I then changed the root password ‘passwd’ and I was ready to start my install of dansguardian.

Install of Dansguardian

The setup I wanted was one with Squid for the proxy, ClamAV for virus scanning and then Dansguardian for the content filtering. I followed a tutorial guide on how to forge
http://www.howtoforge.com/squid-proxy-server-on-ubuntu-9.04-server-with-dansguardian-clamav-and-wpad-proxy-auto-detection
but I would skip the auto-detection part as I did not need this and could always add it at a later date.

The first step is to install ClamAV

apt-get install clamav-daemon clamav-freshclam

I then needed to edit the ClamAV conf file so I ran

Vim  /etc/clamav/freshclam.conf

And found that vim was not installed so I installed it

Apt-get install vim

Once installed I could then edit the conf file above. FreshClam is the part of ClamAV that downloads the new virus definitions. The config file contains a line ‘checks 24’ which indicates that FreshClam will check for new definitions 24 times a day or once every hour. I decided that would be adequate o accepted the default values, So my freshclam.conf file looks like this:

# Automatically created by the clamav-freshclam postinst
# Comments will get lost when you reconfigure the clamav-freshclam package

DatabaseOwner clamav
UpdateLogFile /var/log/clamav/freshclam.log
LogVerbose false
LogSyslog false
LogFacility LOG_LOCAL6
LogFileMaxSize 0
LogTime no
Foreground false
Debug false
MaxAttempts 5
DatabaseDirectory /var/lib/clamav/
DNSDatabaseInfo current.cvd.clamav.net
AllowSupplementaryGroups false
PidFile /var/run/clamav/freshclam.pid
ConnectTimeout 30
ReceiveTimeout 30
ScriptedUpdates yes
CompressLocalDatabase no
NotifyClamd /etc/clamav/clamd.conf
# Check for new database 24 times a day
Checks 24
DatabaseMirror db.local.clamav.net
DatabaseMirror database.clamav.net

You then need to restart ClamAV

/etc/init.d/clamav-freshclam restart

It was then time to install Squid, again an easy process with apt

apt-get install squid

Then to make some modifications to the squid conf files:

Vim /etc/squid/squid.conf

Just adding ‘http_port 3128’  to the bottom of the file and then restart squid

/etc/init.d/squid reload

Finally it was the turn of Dansguardian, again a simple task using apt:

apt-get install dansguardian

Again some modifications to the dansguardian conf files

Vim /etc/dansguardian/dansguardian.conf

I simply added my SheevaPlug Server IP address to the ‘filterip =’ line
Uncommented the line:

contentscanner = '/etc/dansguardian/contentscanners/clamav.conf'

to enable the ClamAV virus Scanning and then commented out the line:

#UNCONFIGURED - Please remove this line after configuration

Then restarted dansguardian:

/etc/init.d/dansguardian restart

Testing the install

The system was then ready and I added the proxy settings to my web browser with the address of the SheevaPlug server and then a port of 8080

I then tried Google which was accessed no problem, and then google’d for some adult terms which were blocked.

I do not like the default ‘blocked site’ template that dansguardian uses but I am sure I can change that as it looks like a simple HTML page.
I also need to re-configure the router to stop it handing out ip addresses and limit access to it to the ipaddress of the Sheevaplug and then add DHCP to the sheevaplug but I will carry on with this at a later date.

The SheevaPlug review

I am very pleased with how well this has gone and the performance of the Sheevaplug. I will continue to use this content filtering on my own laptop to see how it goes and test the performance of the Sheevaplug. The first impressions and results are very good indeed.

EDIT:

I should have given a link to NewIT who sell these little SheevaPlug systems they can be found at http://www.newit.co.uk defiantly worth a try.

UK TV Proxy was a service I setup in 2008. It was designed to complement my Super Proxy service. Basically what it does is allow people who are outside the UK to view online TV services in the UK, which are blocked if you are connected to the Internet from any web address other than a UK based one.

Over the last two years it has been ticking over with a few sign-ups. Some whom have stayed to use the service on a monthly basis some of whom who close their accounts after a few months for whatever reason.

Recently however I have seen a rise in the number of sign ups to the service. I have created 10 new servers in the last week. I wonder what the reason for that would be. Its profile has been raising in Google for some time, but I do not think that is the sole reason. The rise in Googles ranking is probably more to do with my buying a .co.uk domain to add to the cheap .info name that I started with.

It has now become something that I am going to dedicate some more time to promoting as it appears worthwhile to invest the time into this project.

If you are interested in the service please go to the UK TV Proxy website.

I have just got the following email from Affiliate Window:

Affiliate Programme Closure: Confetti (2002)

Dear Affiliates

Affiliate Window want to inform you that Confetti will be temporarily closed today, the 17/08/2010.
We apologise for the short notice period but this is only ever done in extenuating circumstances.

Please ensure you have paused all activity for this merchant as affiliate links will be deactivated until further notice.

We will update you as soon as you can re-start your campaign.

Kind regards

Affiliate Window Support

I find this quite worrying, is there something going on that we need to know about. I use the Confetti scheme on several of my sites and giving me this short notice means that I have not way of removing my links. I understand that there maybe extenuating circumstances, but I hope that more information will be forthcoming in the near future. Will this programme be coming back online?

It is not as if I earn a great deal from this affiliate,  however, I would like more information so that I can make a decision if I need to replace them or make do until they are back with us.

Just one hour ago I posted a blog post about work I have done today on the shed. My post talked about the product I have used for this which is Coroline corrugated bitumen roofing.

I use the Google Bot Bling plugin for WordPress which tells you when Google has visited your site and emails you a log of what was indexed. I noticed this email and saw that my latest post had been indexed so I tried a search in google for ‘Coroline corrugated bitumen’ and this site came 5th in the search engine results page (SERPS)

Eva is rapidly growing up (2 years and 4 months) and it will not be long before she is surfing the web. She already asks for websites to be brought up, she loves the night garden website and asks ‘press the night garden button’ every time I am sat with the laptop.

Although I like the idea of NetNanny and similar products, they are all focused around one PC and controlling that one PC or one profile on that PC. As an IT professional I have a number of computers all over the house, A Media Centre, a laptop for both Cheryl and myself and a PC in my office. With the new kitchen extension we will have another LCD TV along with a media centre PC. So the access to computers in the house is rather open.

I want to control the network we have at home rather better anyway. Currently I allow the router to do all the work, it handles the DHCP and the control of the network. I would like to have more control over that and as such a Linux server to control the network would be ideal.

This being so I would then be able to use Squid proxy to filter content on the network, I could then install Dansguardian on top of Squid to filter our Adult content and provide me with more control.

The only problem with this is that I do not like the idea of a server type PC, this both because I do not like the idea of having it running constantly as that could become costly, but also the size and noise of such a machine does not lend its self to the home environment.

I have been looking at low power PC options and have found the Sheeva plug system which is an ARM based mini PC which is the size of a standard power supply.There are a few options but the one that I was looking at was the Sheeva Plug Multi which takes an SD card for the OS. You can optionally buy an SD with an OS pre-installed Debian 6 or Ubuntu 9. Or you can download the CD Card Images from their site.

Sheva Plug

This is ideal as I want to have a system running Ubuntu as my tests with Dansguardian in VirtualBox have all been with Ubuntu and the install went very easily.

The system has a 1.2Ghz ARM processor with 512Mb of Flash Ram and 512 Mb of DDR2 Memory.It has a Gigabit Ethernet port, USB2 port and ESATA type II port. It does not have a VGA or other monitor port, the idea behind this is that you would access it via SSH which suits me. It also has the ability for another SD card to be plugged in or the USB and ESATA ports give the opportunity to add storage to the device effectively giving you a NAS solution.

The power consumption at about 19w is a little higher than some ‘Low power PC’ options, but at £89 I do not think that you can argue that, especially for the size of the device and being able to hide it away in a draw or behind a desk somewhere without it causing trouble with noise.

My only concern about this is that once I setup this system, I will change all the browsers settings of all the PCs on the network so that they run through the Squid proxy rather than directly to the router. I need to figure out how to stop the router accepting traffic from anything else other than the Squid proxy server so that you can not simply take out the proxy settings.

I have a fairly advanced Billion router (I forget the model) but I ams sure that somewhere on its configuration this will be possible.

I will be placing an order for this device and I will post her how my trials go with setting it up as both a home network server and as a content filter or net nanny.

Following my recent post about the ‘3 must have Media Centre Plugins‘ I thought that I would post about the three WordPress plugins that I consider a must have for all WordPress installs no matter what you use WordPress for, be that a Blog or a CMS.

I of course would promote my plugins at this point, however they are targeted at a specific niche on the WordPress users and as such do not fit into the must have for all WordPress installs category.

I also could go on about SPAM filter this, and SPAM filter that, but in all honesty that has been done so many times before and in my opinion most of the big SPAM plugins for WordPress out there work well so pick one and go with it.

My List is one that give practice benefit to you the blogger. They are all available for free on the WordPress Plugin Directory.

Search Meter

WordPress has a great inbuilt search facility, most if not all themes integrate with this to allow users of your WordPress site to search your content. This information is very valuable, especially if you are using WordPress as a CMS or as an ecommerce site, but also if you are a blogger.

To be able to have an insight into what your users are searching fo on your WordPress site gives you an insight into what they are interested in and as such you can tailor your future content to suit that, providing more content based on the search trends on your site.

Combine this with information coming from Google Analytics which will show you what keywords people used to find your site on search engines and you start to have a very powerful body of data at your disposal.

The Search Meter WordPress plugin records the searches made on your WordPress site and provides you with that information along with how many results those searches had on your site.

This allows you to start writing the content that your users want on your WordPress site.

All in one SEO

You can not have a list of WordPress plugins without the ‘All in one SEO plugin. By rights it should always be the number one plugin that you install every time you setup WordPress. In all honesty what should happen is that this plugin is taken on into the main code of WordPress and maintained by the WordPress developers.

The plugin improves the Search engine optimization of WordPress and allows you to control the meta tags created. Meta tags are things like the title of the page, hidden keywords and descriptions that can be picked up by the search engine spiders.

It is a must for all WordPress installs.

Keyword Density

Ok, so I said that I would not promote my WordPress plugins, what i really meant was I would only promote one of them! Seriously though, I do believe that this little tool is essential for everyone.

If you have done any research into getting search engine ranking, you will know that content is king. It does not matter what your site is, the most essential aspect is the content, its quality and its uniqueness.

The keyword density plugin aids you in writing this content to ensure that you get the balance of your chosen keywords in your text. It is a very simple plugin, you tell it to monitor keywords of your choice. If for example you have a blog about Cooking you may want to make sure that your content is rich with keywords such as ‘Cooking, cook, chef, recipe, kitchen, food, meal, baking’

This plugin tells you how you are doing with the balance of those keywords as you type your content. It also makes sure that you do not include your keywords too often as that would bee seen by Google as spamming the search engine.

I have been adding some additional functionality to our media centre over the weekend. I used to run many plugins on Vistas Media centre, but after the upgrade to Windows 7 some of these did not work and others I just have not got round to installing. I thought I would post about the three that I see as being the most beneficial.

1. Remote Potato

Remote Potato has to be my number one must have plugin. Indeed it was this plugin that I was waiting for with the Windows 7 version of Media centre as the one I used to use did not work with Windows 7.  Basically what this plugin does is allow you to access your media centre via a web browser. It allows you to schedule recordings, view your recorded content and even stream it over the web if your Internet connection is good enough.

I understand from the developers website that this plugin has undergone a great deal of development over the last year or so, and is set to have more functionality added, however as it stands it does everything I want.

Imagine you are out and about, at work, out in the pub or even on holiday and you have forgotten to record your favorite program. Simply find a connection (us use my HTC mobile phone) enter the address of your home network (again I use dyndns.org to create a static address) and load up your media centre, browse or search for your program in the listings and click record, and hey presto it will be recorded for you to watch when yo get back home.

Here is a video that the developer has created of the plugin at work:

In my opinion this plugin essentially adds the missing functionality to Windows media centre that I have been looking for.

2. Macrotube

The MacroTube adds the facility to watch Youtube, Dailymotion, and MSN’s Soapbox videos from your media centre. I am a fairly light YouTube user, but having said that there are occasions when I would like to be able to view YouTube on the TV. This plugin adds this functionality to Windows 7 Media centre meaning that you can do it without exiting the media centre and starting a web browser. This also means that I can do it from my Media Centre Remote rather than having to use the mouse and keyboard.

3. Trailer Library

Trailer Library links into FilmTrailers.com to bring trailers of files soon to be released to your Media Centre. We find that this is a great plugin as we are subscribers to LoveFilm and we always struggle to find movies that we want to rent. Having the ability to preview some new releases as and when we want on the media centre gives us ideas to add to our LoveFilm rental list.

I understand that it only currently works for UK and US users, but as I am UK I am happy!

There are many other great plugins out there. These are just some that I use on our Media centre.

I have been asked by many people how I managed to post a blog post on Friday when I was at a Wedding of a friend (Congratulations to Lian and Bobby!) Well the fact of the matter is that although the post was published on Friday it could have been written at any time.I generally use the Scheduled posts facility in WordPress to publish the posts when I want them to be published which is not necessarily when I write them.

Why would I not want to publish when I write them? Well for various reasons, generally I will write a whole bunch of posts in one go and then schedule them to be published over the coming days to ensure that there is a constant feed of content to the live site, rather than a big chunk and then nothing for weeks followed by a big chunk again.

Having a constant stream of regularly posted content is better for Search Engines (SEO) especially for Google. There is a chance that Google would see a mass of published content as being SPAM rather than being genuine content if it did not fit within the pattern of posting for that site.