Let me back track a little and give some background, at home we run a Windows 7 Media Centre, my main desktop and two laptops also run Windows 7 and as such have Media centre capabilities. Until now, I have attached external USB drives to the Media Centre PC in the front room, this solution was fine and has worked well, however, this does mean that the disks are all in the living room and as such a little messy and the shares are managed by the media centre PC its self, thus when the laptops access media the main media centre PC slows down.

We will be adding another Media centre to our network for the Orangery Kitchen Extension, and I have also installed a Satellite with picks up Freesat HD so I will be buying some Satellite receivers for the media centres and soon will be recording HD video, which will compound the problems.

I wanted to move all the storage into a single location, and as I have a ShevaPlug that runs the rest of the network (DHCP, Dansguardian, Squid Proxy and virus scanning) it made sense that this system running Ubuntu would also manage the network shares. A simple job of just plugging in a USB hum to the ShevaPlug, mounting the USB disks in Ubuntu and away we go…

Reality was not so simple, the main problems came around the NTFS file systems on the USB drives. Ubuntu does not support this natively (or my install does not) so I had to install ntfs-3g, no problems, and after a short while I got the disks mounted and up and running. I then started to try and play media from those disks and to copy some media back to them, every thing seemed very, very slow.

I was getting transfer rates of less than 2MB/sec, Okay this was over the network from the media centre to the Shevaplug Samba share, but the network I have setup is all running at 1Gb, all switches are 1Gb and all network deices (apart from wireless ones) are 1Gb, so this speed was very disappointing.

While investigating this, I was logged onto SSH on the Ubuntu Shevaplug. I noticed that SSH seemed to be running very sluggish, I had the bad feeling that the Shevaplug was not powerful enough to do what I wanted, but after doing a ‘top’ command to find out what the thing was actually doing, I found that ntfs-3g was running 90% of the processor and had taken all of the available memory.

A quick search on google revealed that NTFS-3g is notoriously poor and many people recommended keeping clear of it.

This presented a problem of what to do.

I had a spare disk so I set about trying different file system formats and the one I have ended up with is EXT4. I now have transfer speeds of 60MB/sec and more so I am well pleased with that.

Changing the file system obviously meant that I had to re-format the disks, so moving 2Tb of data from one place to another, finding space on different PCs for all the date while I cleared the first fist for formatting, then moving date from disk to disk to format each to EXT4.

I now have it all back up and running, all EXT4 disks, all my Samba shares setup and the media centre is currently re-indexing all the media, and everything looks good.

Related posts:

1. Home Computer Network Finally Finished Recently I have posted about my tribulations with setting up my perfect network at home. A quick re-cap is that we have two Windows Media...
2. Disk Space Monitor Bash Script For Linux Having setup the media storage and samba shares on the network at home, I do loose some control via windows of monitoring the space left...
3. Review Of The Western Digital Elements 2 Tb External Hard Drive Regular readers of this blog will have seen that I have been doing some work on a solution for the storage of all my digital...
4. Mistakes With Network Active Plugins On WordPress Multi-site Several days ago I was working on a network of WordPress sites I have setup, all using WordPress Multi Site, not an especially strange setup,...
5. Google Analytics And Using Segments To Track Keywords As part of a review of my online marketing activity for both my Software Business and for my affiliate business I have been looking back...

If you remember I was discussing the media that we kept, digital video of our Daughter growing up, priceless material and the problems I had of backing up the large amounts of data that this had become. My preferred option is Amazon S3, yes there is a small cost involved, but I currently backup all my digital photographs, web servers, email servers, project files, accounts and more to Amazon S3. I would like to keep all my backups in the same place, so that if the worse comes to the worse I can always recover those priceless memories.

Cloudberry online backup as its name suggests is a Windows application that runs backups. It supports Amazon S3, Dunkel and Walrun backup services. I am of course only interested in the Amazon S3 part, but i do intend to checkout the other two as I have never heard of them. Cloudberry online backup costs $29.99 (about £15 ish) so not costly at all if it makes life a little easier and makes me able to secure all my data.

The install process works exactly as you would expect, a simple wizard takes you through the install process, and the download of the application was fairly small, just a couple of megabytes from memory, so even on my poor Internet connection no problems there.

I installed it on my Windows 7 Media centre, which is where all my media is kept and within 2  minutes I had a backup running, so very simple to use.

When you first open Cloudberry online backup, it shows you a welcome screen, which asks you what you want to do, backup wizard or recovery wizard. After adding in your security credentials for Amazon S3, as described above, creating a backup plan is as simple as following the wizard.  Once you have plans created, you can click on the tabbed interface to manage those backups and check their status. A history panel tells you the details of what has been going on, and on the welcome page it shows the progress of any active backups or when the next one is scheduled to run.

The backup wizard takes you through a series of questions asking you what you want to backup, you simply tick the folders and then choose some more options for if you want to compress the files or not, and in the case of Amazon S3 how many versions of each file do you want to keep. This means that you can recover past versions of changed files if you like. As my Media generally stays the same, I have saved cost on the backup storage and chosen to only keep one copy of each file which should be good enough for my needs, however if I start to use this application for other uses, backup of project files for example I may choose to keep many versions of files.

My First backup plan was for digital media, I chose some videos from our latest holidays, a folder of about 3.3Gb and Apart from the speed, which is my poor broadband (I live in Hull where the only broadband supplier is Karoo who are about 2 or 3 years behind the rest of the country, I get 6Mbs down and 200k up, and at peak times it just grinds to a halt) it seems to be working file, it is still backing up as we speak! It is going to take some time to backup the 100Gb+ that I have!

The system has a built in alerts system, you can choose an email address to have it send alerts to, these alerts can be simply if it encounters a problem, or if you are like me, you will want emails when it completes backups as well so you can monitor the progress.

As I say it has not completed its backup plan yet simply because of the size and the speed of my broadband, however, the files it has backed up to Amazon seem to be fine.

The software also optionally allows you to encrypt your files. I am not sure what type of encryption it uses and would think that this would mean that you have to use the cloudberry software to recover from any disaster, not that this is a problem, it has recovery wizards built into the software so that you can recover files as easily as you back them up. I chose not to encrypt as I like to have access to those files from other Amazon S3 clients such as the Firefox plugin and S3Sync.

A very useful feature is that you can choose to filter what files are backed up, so once you have chosen the folders you want to backup you can further refine that backup by either including or excluding files of a particular type, so in my case I have some log files which I do not want backing up as these just take up space so I exclude log files.

There were a couple of features that I would have liked to see.

Firstly you cant define where to back it up to in your S3 buckets, it just creates folders of the computer name, and then replicates the folder structure on your drive. This is perfectly ok for most scenarios and exactly what you want as a home user, just tick the boxes against what you want backing up and leave the software to it. However it would be nice to have an advanced option so that you can define where the files will be backed up to, so if like me you like to keep your Amazon S3 organized to suit you then the software could follow your choice.

Just to reiterate, the way it works now is fine for 99% of people, I always like that extra little bit of control.

The only other issue I found was that you cant schedule a backup plan that has items from networked drives in it, I run a NAS drive which I currently backup via S3Sync on Linux Command line, however this is difficult to maintain, I would have liked to move all my backups across to Cloudberry, and have the one machine (my media centre) running all the backups to Amazon S3. The software will do the backups but strangely you cant schedule regular backups if you have a networked drive included in the plan.

As a software designer and developer myself I understand the logic here, a network drive may not always exist, and as such it a backup tried to run where the drive was not there this would cause problems, however, perhaps this could be handled with a check to see if the drive was available before the backup started and email the alerts email address if it was not.

I have not yet tried the recovery wizard, but I expect that it is very much like the backup wizard. Recovery will be quicker as the way broadband works you have a far greater download bandwidth than upload bandwidth.

Would I recommend Cloudberry online backup? In a word Yes. Very simple to use, some great features even taking into account that I always want more! If it could do scheduled backups from network drives and also let you choose where in your Amazon Bucket you wanted it to backup to I would not be able to fault it, but as I say for most users these features would not be a necessity.

I have also noticed that on their website they have a coming soon section and they plan to add FTP support, I assume this means that you will be able to backup via FTP. They also have planned Storage cost estimates, which will be good when you select a folder with 100Gb of Data in it like I am about to do, to be able to know how much it will cost you before you start would be perfect.

You can get Cloudberry online backup by clicking here

Related posts:

1. Home Computer Network Finally Finished Recently I have posted about my tribulations with setting up my perfect network at home. A quick re-cap is that we have two Windows Media...
2. Amazon Import Plugin For WordPress Live Demo Now Working I have just put the Live Demo of the Amazon import Plugin for WordPress Live again. I had hope to get this done yesterday, however,...
3. Media Centre And Network Storage Solution I have spent the past week moving 100′s of terabytes of data from disk to disk, trying to find the best solution for the central...

As you will have recently ready I am backing up all my media to Amazon S3. My photographs are stored on my Western Digital MyBook World which I have hacked to open up the Linux shell, and as such i use S3Sync on command line to back up my photographs. I have even written a bash script that does it as it backs up some other stuff as well.

The rest of the media is on an external drive on the Windows 7 media centre, I am looking for a decent app that will sit there and sync the videos from my digicam to Amazon S3. I have tried loads but they all seem to fall short, either they are buggy and do not work or they can not sync and just manage adhoc single transfers.

I need something that is automated that will sync once a day/week and any changes or new videos will be uploaded to Amazon.

If anyone can recommend any applications free or otherwise?

Related posts:

1. Media Centre And Network Storage Solution I have spent the past week moving 100′s of terabytes of data from disk to disk, trying to find the best solution for the central...
2. Home Computer Network Finally Finished Recently I have posted about my tribulations with setting up my perfect network at home. A quick re-cap is that we have two Windows Media...
3. Amazon Import Plugin For WordPress Live Demo Now Working I have just put the Live Demo of the Amazon import Plugin for WordPress Live again. I had hope to get this done yesterday, however,...
4. Disk Space Monitor Bash Script For Linux Having setup the media storage and samba shares on the network at home, I do loose some control via windows of monitoring the space left...

I have blogged before about windows 7 and how I have bought copies for all the machines we own, the media centre included. I have this setup as a home network so that all the media stored on the media centre is accessible via every machine and as such is accessible wherever you are in the house.

Up until Windows 7, the HD media from the Sony HD Camera had to be converted in order to be viewed, now Windows 7 can play these in their native format, which is msts. This is whey I have bought the new drive so that I can archive all our digital video within the media centre system.

The next challenge is to be able to share some of this media to family and friends. Photographs and Videos are a great way of letting family who are spread out around the country and even internationally, but the big question is how.

In the past I have setup Gallery2 on one of my servers and used that as a gallery for photographs, the problem with that is keeping it updated, uploading the pictures and sorting them into the gallery was always a time consuming task and one that I never kept on top of.

I have been experimenting with Youtube for some time now. On one of my other blogs (On the allotment) I upload videos to Youtube in the format they come out of the camera (M2ts) Up to now this has been a very much one off thing every now and then, but I am looking to see if I can use YouTube with a private video stream to upload many more of my videos straight out of the camera.

The first problem I have experienced is that when uploading from the media centre it powers down in the middle of an upload. I am not sure how to stop this without turning off the power management all together, something that I do not want to do.

The next problem I now have is one of backup. Until now I have uploaded all my Digital photos to Amazon S3 currently standing at around 50gb of data. I would like to do a similar thing for the digital videos, however, this will dramatically increase the data that will be stored as there is currently around 100gb of video data. My current thinking is that the cost of backing this up to Amazon S3 will be relatively low and compared to the value of this data to us as a family keeping it safe from fire or other loss can not be measured.

I am also looking at getting a BluRay writer, I have been holding off until now as I do not like to be an early adopter for this sort of technology, back in the days when Writable CD’s first hit the market, I could have filled all the cafes in England with coasters that I burnt!  I have read that they are now fairly stable and the price of the media is coming down all the time.

BluRay would only be a feasible backup method if I could Geolocate the disks somewhere else and can guarantee that those disks do not get damaged.

Related posts:

1. Media Centre And Network Storage Solution I have spent the past week moving 100′s of terabytes of data from disk to disk, trying to find the best solution for the central...
2. Home Computer Network Finally Finished Recently I have posted about my tribulations with setting up my perfect network at home. A quick re-cap is that we have two Windows Media...
3. Review Of The Western Digital Elements 2 Tb External Hard Drive Regular readers of this blog will have seen that I have been doing some work on a solution for the storage of all my digital...
4. Google Analytics And Using Segments To Track Keywords As part of a review of my online marketing activity for both my Software Business and for my affiliate business I have been looking back...

I deliver SPAM to a Spam folder in each users mail boxes, I used to have a script that deleted emails after 7 days, I need to re-write this and get that working again.

There are still some of my sites that are down or out of date. I spent allot of time in the week before the disaster entering all my domains in Domain Name Portfolio and unfortunately did not get a back up of that work so I need to re-enter them all.

I also have some software that I wrote that monitors Google (and others) page rank for all my sites. I also had this as a download on my Digitalquill site, however I must have written this in VIM directly on the production server, which means I do not have a copy of that anywhere. I am hoping that someone who downloaded it will be nice and send me a copy.

I am not sure about all the themes for my blogs, including this one. I had customized themes to be specific to my requirements. I made the fatal mistake of thinking that I did not need to back-up my wordpress folders as I can just download a new copy, but this means of course that both the modified themes and any uploads (media and images) have been lost.

The theme for this blog is not too bad, not sure i like it as much as the old one so I may yet still change it, but the one for Earning From Affiliates is not good enough for what I require, so I will be finding the starting theme that I modified and making my changes to bring it back to what I had.

Other than that, I think we are just about there.

Related posts:

1. Can Viral Backlinks Affect Your Seo We have been looking at Search engine optimisation in the last few posts on this site. I talked about ways of getting back links. It...
2. Spam Bait Posts We have all heard about link bait posts, but how about SPAM bait posts. The recent post I made ‘Are Nofollow Links Worthless?‘ has attracted...
3. Don’t Unsubscribe From Spam Emails In the interests of saving embarrassment, shall we say ‘A friend’ has just clicked on an unsubscribe link in a SPAM email, and now has...