This is the second part in my story of installing a solar photovoltaic system in the Northern California Bay Area.  You can also read Part 1, Solar Panels / Energy.

I figure the best way to organize what happened is to do a timeline from the very beginning:

4/1/2010: Contract with REC Solar signed.  The process is:

• REC does a site evaluation
• REC pulls permits
• I go with REC to Costco to purchase modules & inverter
• Roof Right removes old roof and installs plywood and underlayment
• REC installs the roof mounts for the rails
• Roof Right installs the rest of the roof
• REC installs modules, inverter, and all electrical
• REC coordinates city inspection
• REC submits all the info to PG&E
• PG&E does interconnection
• I turn the system on!

The whole process is expected to take 14 weeks which includes the 3 weeks for the PG&E interconnection.  So, that would be a completion date of mid July.

4/13/2010: REC sends out their crew to do a site evaluation.  This includes getting the roof measurements (dimensions, pitch, direction etc.) and the shade analysis.

They have a cool device that takes a “fishbowl” image of the sky from various points on the roof to show what items will cast a shadow onto the roof (click the image to the right).  We discover that the chimney and two big oak trees will be casting significant shadows onto the roof.   Not much we can do about the chimney other than shift around the modules a little.  The trees will either need to come out or be significantly “topped”.

Note:  In the shade image the engineers highlight in green the areas that will cast a shadow.  Items that will be removed (trees) are not highlighted.  The system then uses the data to calculate how much sun that particular spot on the roof will receive during which hours of the day and months of the year!  Really cool!

5/4/2010: The REC sales manager and I meet up at Costco and make the biggest purchase I’ve ever made at Costco or put on a credit card, $12,800!  I’m now the owner (at least on paper) of 18 solar panels and an inverter.

5/11/2010: REC also sent me the following that shows where the trees need to be trimmed and how much difference having no shade will make in the electricity production.

Based on the calculation, we lose 15% output (most during the Winter) due to the shade from these trees.  You can see how significantly we need to cut back these trees to remove the shade.  Both of the trees are “wild” (meaning nobody planted them there… well, maybe squirrels) and even though they are pushing up against the foundation we’re not sure we want to take them out.

After a lot of back and forth we decide against taking the trees out.  Every person I talked to said that you shouldn’t top oak trees.  We decided it was “better” for the trees to have them topped than 100% removed.    We had a great arborist come out and, even though against his better judgment, he “trimmed” the trees.   While very aggressive, they actually turned out pretty nice.

5/6/2010: Roof Right comes out to redo the part of the roof where the panels will be installed (we’re only getting that section replaced).  They do the tear off, lay the plywood, and the pretty blue underlayment.   I have them install 4 new “eyebrow vents” in preparation for a whole house fan that I eventually will install.  I also have them cut out two 10 inch holes for adding sweet new tubular skylights that I got at Costco for $150 each!   I also had to move the vent for our hot water heater up and to the West to get it as far from the modules as possible so it wouldn’t be in the way or cast a shadow.

5/7/2010: REC sends out their guys to install the roof mounts.  I had previously requested extra tall “standoffs” be used.   I wanted these for 2 reasons:  1)  For clearance above the eyebrow vents.  2)  For extra air movement below the modules (which lose performance as they get hotter).

While they were here, the REC guys took new shade measurement pics and everything looks good… except the big ol’ chimney.   I’m still bothered that the chimney is casting such a significant shadow on a couple of panels in the early morning.   I draw up my own plan for module placement and send it to the engineers.  Basically I moved one panel up to the top row and shifted parts of the array to the West (away from the chimney).  Turns out that one of the big factors for the way they layout the array is symmetry.   I said, “Hey, these are on the back side of the house so I’m not really worried about them being symmetrical.”  The engineers said, “No problem, we’ll go with your layout.”   Guess what… this change was expected to net me an extra couple percent increase in output!

At the end of the day the Roof Right team comes out and completes their roof work and flashes in the roof mounts and even install my tubular skylight flashing for me!  I think it all looks absolutely fantastic!   Click the picture to the right and you’ll see the new roof compared to the crumbly Terrashake / Cal-Shake roofing.

5/11/2010: We were hoping to begin the install today, but REC ran into some problems with TIGO and their supply of the management units that attach to each module.  I’m told there could be up to a 2-3 week delay before we can install the modules.   While I’m a little bummed, I realize that we’re way ahead of the 14 week schedule I was originally expecting.   I start thinking that either the 13 week schedule was a case of “under promise, over deliver” or that somehow things just went way faster than they expected?

5/18/2010: I get some great news about 6 days later that the units are in and we’re ready to go!  The team comes out and begins installing the rails and getting everything ready for the inverter and tie-in to the main electrical service box.

5/19/2010: The team comes back and installs 17 of the 18 modules (the one on the bottom of the stack shattered during travel).  The inverter and most of the electrical is installed!

They look absolutely fantastic and I’m super excited!

Click for larger images

5/21/2010: The team comes back out and installs the last module and finishes a few other items.

This is where things get a bit sticky:  While working to tie into the main service box we found that there were some electrical problems that had the potential to be a huge problem (and expense).   We were looking at having to replace and relocate the main service.  This would also require that the main line coming into the house to be moved and more roof work.  We’re talking potentially $5,000+ more work, permits and months of delays!!!!  Well, long story short, after 2 weeks working with 3 electricians and scouring the world for a few specific old parts and a TON of extra time and work from an amazing guy at REC, we were able to get the problem solved for under $200!  I seriously dodged a bullet!

6/7/2010: The city inspector comes by and meets with the rep from REC and walks the grounds.  I’m terribly nervous and am crossing my fingers that there aren’t any problems.   After only about 15 minutes the inspector is gone and I get the great news:  We Passed!   WOOT!

6/7/2010: All the paperwork is then submitted to PG&E for them to do the interconnection.  Even though my system is 100% ready to go, we need to get their stamp of approval to pull the switch.   PG&E has up to 30 days to do their inspection and the interconnection.  Rumor has it that they are taking up to 3 weeks to get out to do the interconnection.   UGH… it feels like “HURRY UP and wait.”

6/16/2010: It’s 9:50 AM and I’m loading the kids in the car for their very first day of swim lessons.   As we’re about to pull out of the driveway I see a PG&E truck driving up the street.   Can this be it?!?!  I send my wife off with the kids so I can stick around.    The truck parks in front of the house and out comes a guy with a new bi-directional meter in his hand… I swear he looked just like Santa Claus!  Within 15 minutes he had pulled the old meter, installed the new one, and gave me the A-OK to pull the switch.  WOOOT!!  9 Days?   I think this is a new record for PG&E interconnection!

Right after he was gone I turned on the power.  After what seemed like FOREVER the inverter kicked on and I was live!  The new bi-directional meter started at 00000.   I sat there and watched it until I saw it roll back to 99999. 

Summing up:  from contract signing to full interconnection was 76 days (almost 11 weeks) which put us about 1 month ahead of schedule… and that’s with the extra 2-3 weeks of problems related to the electric service issue.

You can click the image below for a high resolution picture of the final installation.  Also I’m including an image of the TIGO interface to show the cool visibility I have into the performance of each solar panel.

We’re using about 20-24 kilowatt hours per day and generating about 25.  At the time of writing this our meter says 99954 and has been as low as 99946.   We didn’t build the system to completely offset our usage, but that’s just what happens during ideal conditions during the summer months.   Our “credit” will carry forward month to month.  As the days get shorter we’ll be producing less than we use, and over the course of the year the credits that we’ve banked up over the summer will be used up.  We don’t expect that we’ll ever produce more in a year than we’ll use unless we totally change our habits… and with our daughters getting older I doubt that’s going to happen.

Hopefully you found this information helpful.   If you’re interested in solar you should contact REC Solar and tell them Rob Ludlow sent you! Even if you’re not in the California San Francisco Bay Area, they may be servicing your area or have partners that do!

Any questions, simply comment below.

VERY long story short: We’re getting a 3.9 kw (DC) solar photovoltaic system installed on our house that should have a “payback” of 8 – 10 years.

Now, all the sordid details:

Why we need / want solar: I posted earlier about how we’ve been looking for ways to save electricity.  Unfortunately we’ve hit the point balance between conservation and significantly changing our lifestyle.   We realized that changing consumption would only get us so far… we needed to begin production!

My Solar Love Affair Begins: I’ve always been fascinated by solar power.  As a kid I loved electronics and did weekly bike rides to Radio Shack to spend all my money on motors, LEDs, etc.  I remember getting my first solar powered calculator and falling in love from that day forward.  I was so enthralled that for my high school “Senior Project” I chose to write about solar energy (thermal and photovoltaics).

Fast forward to 2010: A few entrepreneur friends and I went to a “green” networking event and the place was full of solar companies.  I thought I’d pass by and hear their sales pitch.  One of the more aggressive companies was able to get me to fill out a “free consultation” card… a few days later I had a voicemail for an appointment.  I was honestly dreading the meeting.  I always thought solar energy for the house would be cool, but wasn’t really considering adding residential solar power to our home because:

1. Our roof has about 5 years of life left.  No sense to get solar installed then have to have it removed / reinstalled when the roof needs to be replaced.
2. I heard that the “solar break even” or “solar payback” periods were 12 – 15 years.
3. We’re not sure we are going to stay in the house we’re in.
4. I kept thinking “The solar panels will just get more efficient and cheaper.”

Well, I figured I’d have the guy come by to do his solar song and dance.   During the appointment the most interesting things I leaned:

1. The goal of a solar install (at least in the California Bay Area)  isn’t to remove your need to be “tied to the grid” but to remove you from the top tiers of electricity prices.  For example, if you use 200% of your “baseline” you get charged $0.50 a killowatt hour!!  NOTE:  If you are tied to the grid (and you probably will be unless you want to spend $20k more on a battery system) it is required that the grid be powered for your inverter to function.   That means if the grid goes down at noon then your inverter turns off and your panels, even with full sun, will not produce a single watt of power.  A bummer for sure because if the power company is down for 3 days, you will be too, even with a 4k watt power plant on your roof.
2. Based on the above, if you use a lot of electricity (like we do) then your break even happens much quicker.  We average about $160 a month just in electricity (about 30 kwh/day). I’ve been told you should be at about $80+ a month if you are thinking about solar for mostly monetary reasons.
3. We could just replace the roof section where the solar array would go and leave the rest of the roof for replacement down the road.
4. Rebates and Credits:   There is a 30% Federal Tax Credit of on a solar system install and California State “Go Solar” rebate of $1.10 per watt installed.
5. The CA rebate was VERY CLOSE to dropping down to the lower “step” of $0.65 within a couple weeks, so if we wanted to do solar, now was the time to do it.   Going from $1.10 to $0.65 would be an opportunity cost of potentially $1,755 (3900 x $0.45).

After really thinking about the items above, we realized that we’d probably stick around in the house we’re in and that it financially made sense for us to get solar and to take advantage of the great rebates and credits.

Our final decision process was  as follows:

1.    Does solar make sense financially?  Fortunately this was pretty much already resolved.  I got in touch with a few people that have had solar for up to 3 years and all were very happy with the ROI and the efficiency of their systems / cost savings.
2.    Which panel manufacturer?  The manufacturer of our previous roofing material went bankrupt so we’re kinda sensitive to making sure the panels be covered by a solid manufacturing company.
3.    Which installer?:  I wasn’t about to sign a contract with the first company that came by (and he really was pushing us to do so).  I ended up getting a bunch of referrals and  read a bunch of reviews to narrow the ton of California based solar companies down to a handful.

Solar Company Competition: Below are the companies that did on-site evaluations (in alpha order).

• Akeena Solar
• REC Solar
• SunLightAndPower
• Super Solar
• The Solar Company

Each company had pretty good presentations.  In fact, I ran into a problem I didn’t expect:  Each company had glowing reviews and a really good value proposition.  After an extensive amount of research I had narrowed the list down to three companies: REC Solar, SunLightAndPower, & Super Solar.

SunLightAndPower: A large, well known, reputable company in the SF Bay Area.  They have many installs under their belts and the sales rep was extremely helpful and knowledgeable.  Their price was the highest of the group, but I felt like there was great value in working with a large company.

Super Solar: Owned by a former employee of SunLightAndPower who had been one of their top installers for about 10 years.  They are just a few person company which meant low overhead and a low price.   Unfortunately, I just couldn’t get past the fear of working with a small company.

REC Solar: Actually this company wasn’t even on my original list.  I was literally about to choose (that night) between SunLight&Power and SuperSolar when I was referred to REC Solar from a really smart guy I met online.  I found that REC Solar was the company that was partnered with Costco, so I thought I’d give them a quick chance to throw their hat into the ring.   They are a good sized company with a great reputation. The sales rep was amazing to work with, and the proposal was very impressive and competitive with the others I received.  Oh, and they were the only company on my short list that was willing to guarantee the  CA Rebate at $1.10 even if the application for my array didn’t make the cut before the drop down to $0.65!

In case you can’t tell, I ended up choosing REC Solar

Why REC Solar? Here are some of the highlights of why I want with REC Solar (Renewable Energy Concepts):

• Large company with a ton of installs under their belt
• Partners with Costco
• Very knowledgeable and friendly sales rep and engineering staff
• Guaranteed CA Rebate before the price drop
• Turn key installation.  They do everything including permitting with the city, etc.
• Availability of the TIGO Energy optimizer (up to 20% more energy from the array and a cool online monitoring system)
• Price per watt installed was very competitive with the competition

The system details:

• 3,960 Watts DC (3,295 Watts AC) Solar Electric System that would require about 323 sq/ft.
• Estimated 6,378 kilowatt hours produced per year (we use about 9,500 per year)
• 18 – REC220AE-US solar panels / modules (This REC is “Renewable Energy Corporation”)  + 25 year warranty
• SMA 3000US Inverter
• Tigo system components
• All labor + 10 year warranty

System Grand Total: $23,800

CA Rebate:  $3,440 (estimate based on shading, etc.)
Costco Member Savings $233.60
Costco American Express Savings $116.80
Federal Tax Credit $6,108.00

Net System Cost $13,901.60

Note: This “net cost” doesn’t include the new roof section ($2,400) or the tree trimming ($600).

Our “payback” or “break even” is estimated to be 8 years, and is calculated based on our current usage and a 6% annual utility rate increase.  I personally believe 6% is too aggressive, but this is the % that all the solar companies use.   Obviously the payback time gets longer or shorter if our kilowatt usage changes (3 girl household) and/or if our electricity rates change.

Another way I calculated the financial aspect of the panels is by setting it up like an annuity.

System cost: $13,900
Periods: 300 (25 year life expectancy)
Future value: $0.00 (assuming worst case that the panels aren’t worth anything after 25 years)
PMT: -100  (we expect to save at least $100 per month in electrical bills)

Based on the above, the annualized rate of return on this system is 7.2% using the “RATE” calculation in Excel.  Keep in mind that this is POST TAX MONEY, so when comparing it to another “investment” you’d need to add your tax rate.  So, if I’m in a 30% tax bracket my comparable rate would be more like 9.36%!  Compared to other investments (especially at 9%) I think solar panels also have  much lower financial risk.

When I asked my REC Solar sales rep about which of my friends would and wouldn’t be good candidates for solar he replied:

“Realistically, the cutoff for a decent ROI with PV is going to be around $80 per month.  There are infinite reasons to go solar that are not directly related to immediate payback and ROI (increased value of home, hedge against the rising cost of power, property tax exempt investment, very safe and stable part of an investment portfolio (think CD’s but with a higher ROI),  and an inflation beating investment) but the reality is the ROI would probably be around 4%-7% (not that bad!).

Also, one day the Plug-in hybrid vehicle, or all electric vehicle will be a reality, which will change the PV industry forever as people will see the value in creating their own “gasoline” for their cars with solar.

There’s also the little part about not buying power from fossil fuel burning sources.  Unfortunately at this point, there is not much of a monetary value associated with this (though there should be since the environmental effects do COST money).”

I couldn’t agree more!  In addition to the financial reasons to “go solar” I personally put a lot of weight into:

• The fun and cool factor of having your house powered by the sun
• The warm fuzzies knowing you’re doing your part to reduce your carbon footprint
• The joy of fully meeting up with your childhood love for solar!

The panels should be installed this month.  I’ll post part 2 with all the installation details when the solar juice is flowing.

Have you thought about getting solar installed?  Why or why not?  Post your thoughts below.   Oh, and if you’re thinking about getting solar, give REC a call and tell them Rob Ludlow sent you! 

(For part 2, see: Solar Power Installation California Bay Area – Part 2 )

HomeMade Monitor Wall Mount

A week ago I decided to upgrade to a new 23″ flat panel monitor (since I spend about 80% of my waking hours at the computer).  I was thinking about buying a wall mount, but I discovered two problems:

• Most didn’t extend from the wall as far as I needed (about 18 inches)
• The prices for the ones that extended only about 13″ were $60 +

Here is my setup before I started:

Well, once again my overwhelming impatience and desire to do-it-yourself won over.  I did some internet searching and was surprised there weren’t a lot of homemade monitor wall mounts out there on the net (at least few I could find).   I did stumble upon this one, but I didn’t need fancy articulating arms, etc.  Well, while I was laying in bed about to go to sleep I came up with a few ideas (as often happens).

My first idea was to have a two 3″ pieces of 1/2 inch pipe, one for the wall and one mounted to the back of the monitor.  Then I’d bend a 2 ft. piece of rebar into a z shape which would fit into the pieces of pipe.  At the last minute I opted for just going with a bent piece of pipe by itself which would fit into brackets.   Here are the supplies I used:

• 2″ – piece of 1/2 inch electrical conduit
• 4 – 1/2 inch brackets
• 4 – M4 .70 pitch metric screws (about 1.25 inches long)
• 8 washers
• 8 wood screws
• 1/2 pieces of scrap wood

The first thing I needed to do was to find a way to attach the brackets to the monitor.   I created a diagram based on the 100 mm square holes for the VESA mounts.  To the right is the image I created.  I simply printed the sheet at 5″ x 5″ exactly (including the 1″ white border) and taped it to my board.

From there I pre-punched the holes with an awl and drilled all 4 holes with a 3/16 bit.  (I also countersunk the holes on both sides)

This provided a perfect backing for the VESA mount on the back of the monitor where the M4 screws would fit.

The metal Arm:
This was one of the more complex parts of the project.  Fortunately I had a 1/2 ” bender on hand, but I discovered that the bend radius was so long that I was using up much of my pipe.  I ended up doing a mix of using the bender and the vice to get the angles I needed (You’ll see in the pics below).

Working with the metal arm to get it just how I wanted was a bit of work.  Make sure you get 100% level bends or you’ll end up with my problem… when the monitor is rotated it is a bit off center and not level.

Next, the brackets:

The 1/2 inch brackets are meant to hold the conduit very snugly, but I needed at least a tiny bit of play so I could insert & remove the whole thing.   I was able to solve this problem by bending out the brackets ever so slightly and then putting a 1/16″ washer between the bracket and the wood.

I attached one bracket to the wood that would go on the back of the monitor and inserted my metal arm and then attached the other bracket and made sure it had just the right amount of play.

Next I found a stud in my wall (what a pain that turned out to be) and used 3″ screws to mount the other board.   I propped up my monitor to a height I wanted and then put the metal arm in place and fastened the other two 1/2″ brackets with screws and washers just as before.

It took a few adjustments to get my pan and tilt just right, but I’m pretty happy with the results:

(click for larger version)

Here is the monitor swung completely out of the way… I’ve never had so much desk space!! 

It isn’t pretty (as most of my “get it done and make sure it works projects end up) but it works really well!

I’ll keep testing it and make notes of future adjustments I’d like to make including painting everything black.

Well, I’m also one of those guys that has about 15 windows open at any given time… browsers, spreadsheets, Dreamweaver, Outlook, Explorer, etc.   I find that I often need to quickly get access to my desktop to edit, move, or upload a file.    Hitting the minimize button on each window isn’t an option, and the “Windows Key + D” command just isn’t comfortable to me.   So, having the ability to hit a single button where my finger is already hanging out is just super convenient!

Up until a year ago my software allowed me to easily set this middle button, via a pull down option, to one of many things.  Minimize All was there and so mapping that function to the button was just super simple… well, that changed and so this is why I’m writing this post!

A while ago I got a new wireless keyboard mouse combo (a Logitech Cordless Desktop LX500).  I was super happy with it… but wait, what’s this:

NO OPTION FOR MINIMIZE ALL?    Oh my gosh you’ve got to be kidding me!  

Well, I wasn’t going to trash my keyboard for the lack of minimize all, so I resigned myself to simply using the “Minimize” option which just minimizes one at a time.

I can hear you geeky types now, “Just assign the keyboard command “WindowsKey + D” as a “Keystroke Assignment”!  Yeah, well, I tried that about 20 times and the software doesn’t recognize the WindowsKey!

Well, today my combo went looking on the net for an upgrade to the software during a periodic update check and found one.   I downloaded and installed with my fingers crossed hoping for an answer to my prayers… but alas, I was left with the same boring options as before.   But this time, there was a difference… I was determined to find an answer, so here’s what I did:

First Step:
Find or create the application that minimizes all called “Show Desktop” (instructions on how to create a minimize all shortcut here).

Second Step:
My keyboard allows me to map to a program of file, but of course my mouse does not.   I read the instructions on how to create a keyboard shortcut (bottom of page) to a program.

Third Step:
After some trial and error I realized I had to create a regular desktop shortcut to the “Show Desktop” application (copy it and paste as shortcut) to my desktop.  I then clicked “properties” on this shortcut and assigned ctrl+alt+9 to that shortcut.

Final Step:
Finally, into the Logitech software I went and assigned my Middle Button to the new key command, which triggers a shortcut, that triggers the “Show Desktop” command that actually minimizes all the windows…. PHEW!

If you know of a better way to do this (other then “Hey stupid, get a different keyboard”) then please let me know.  

The privacy of our visitors to Nifty-Stuff.com is important to us.

At Nifty-Stuff.com, we recognize that privacy of your personal information is important. Here is information on what types of personal information we receive and collect when you use and visit Nifty-Stuff.com, and how we safeguard your information. We never sell your personal information to third parties.

Log Files
As with most other websites, we collect and use the data contained in log files. The information in the log files include your IP (internet protocol) address, your ISP (internet service provider, such as AOL or Comcast, Verizon, etc.), the browser you used to visit our site (such as Internet Explorer or Firefox), the time you visited our site and which pages you visited throughout our site.

Cookies and Web Beacons
We do use cookies to store information, such as your personal preferences when you visit our site. This could include the ability to login to some of our features, such as forums.

We also use third party advertisements on Nifty-Stuff.com to support our site. Some of these advertisers may use technology such as cookies and web beacons when they advertise on our site, which will also send these advertisers (such as Google through the Google AdSense program) information including your IP address, your ISP , the browser you used to visit our site, and in some cases, whether you have Flash installed. This is generally used for geotargeting purposes (showing New York real estate ads to someone in New York, for example) or showing certain ads based on specific sites visited (such as showing cooking ads to someone who frequents cooking sites).

DoubleClick DART cookies
We also may use DART cookies for ad serving through Google’s DoubleClick, which places a cookie on your computer when you are browsing the web and visit a site using DoubleClick advertising (including some Google AdSense advertisements). This cookie is used to serve ads specific to you and your interests (”interest based targeting”). The ads served will be targeted based on your previous browsing history (For example, if you have been viewing sites about visiting Las Vegas, you may see Las Vegas hotel advertisements when viewing a non-related site, such as on a site about hockey). DART uses “non personally identifiable information”. It does NOT track personal information about you, such as your name, email address, physical address, telephone number, social security numbers, bank account numbers or credit card numbers. You can opt-out of this ad serving on all sites using this advertising by visiting http://www.google.com/privacy_ads.html

You can choose to disable or selectively turn off our cookies or third-party cookies in your browser settings, or by managing preferences in programs such as Norton Internet Security. However, this can affect how you are able to interact with our site as well as other websites. This could include the inability to login to services or programs, such as logging into forums or accounts.

Deleting cookies does not mean you are permanently opted out of any advertising program. Unless you have settings that disallow cookies, the next time you visit a site running the advertisements, a new cookie will be added.

We hope you enjoy your visit at Nifty-Stuff.com

I needed a process that would help me sleep peacefully at night.  As mentioned twice before on this site I’m pretty diligent about computer backups.  In my opinion it is exponentially important to backup the constantly changing information on my sites.  Just a single day’s loss of data could be a huge problem.  For example, on one of my sites, http://www.backyardchickens.com/ there are about 20 new members signing up per day and members are posting about 1,200 new posts per day.  Imagine loosing a week or a month’s worth of members and posts!!!

My Database Backup Process & Software:

1. After trying a few different MySQL backup scripts I found a gem hidden within a blog.  Ameir’s MySQL Backup to FTP and Email script works flawlessly and does everything I need.
2. I have the script setup to run every morning around 3:00 AM EST to do a full backup of about 7 MySQL Databases.
3. The last 3 days worth of backups are kept on one of my secure folders on the server.
4. Each day’s backup is also sent to a completely different web host and server in a different state.
5. Each day I synchronize (using SyncBackSE) my desktop computer with this second web host server and download each day’s backup files.
6. Every month I burn all the backups to a DVD and put them into a fire safe.

 If you’ve been counting that is 3-5 copies of my databases that are backed up in different locations throughout the US.  If you count the daily backups my hosting company provides that bumps the number up to about 7 copies of each day’s DB backups!

Database Backup – Only Half The Battle: 

Well, the process above sounds very impressive, but about a year ago I made an almost fatal mistake.  Learn from my experience and know that your backups are only as reliable as the last time you attempted to restore one of the backups.  

Prior to using Ameir’s MySQL Backup to FTP and Email script I was using an application that had been working fine for me for the previous 2 years.  I hadn’t needed a DB restore for about 10 months when I had someone come in and hack one of my forums.   I attempted to restore my most recent DB backup file and got errors.   I tried to restore one that was 2 days old… same errors.   After trying 2 more backup files I gave up and called my server admin to have them restore their backup… phew!   After this fire drill I discovered that my backup script has stopped working about 4 months prior!!!   I don’t know if it was a change in my hosting environment, the size of the DB’s or some other odd problem, but that’s when I abandoned that application for something new.   Since then I’ve been using Ameir’s script without fail.   BUT, the moral of this story:

1. Backup soon and backup often
2. Frequently try to restore your DB backups to ensure all your hard work and preparation is paying off.

UPDATE 7/27/08:  The other day a member of my community deleted a bunch of their content and said they would try to re-submit it.  They said it would take them a while to get all the pics and text back up.   I said, “Don’t worry about it.” 

In about 15 minutes I had restored a backup from 2 days earlier into a copy of my community system, exported the data the member had deleted, and then imported it into the production site.  Thanks goodness for daily backups!

UPDATE 8/23/08: Regarding backups: For the “static” content of my sites I will occasionally (read very rarely) do a “Full Backup” from within each cPanel hosting account (at least for the top 10 accounts). For the most part I rely on my hosting company for the more frequent backups of my static content…. so a I had a question to ensure I’d sleep a little bit more soundly at night:

What is the most simple way to do occasional (i.e., quarterly) do full backups of all my sites? I’ve read people mentioning using the backup systems in WHM and some talk about using rsync, etc.  These sound good for frequent backups, but I feel I only need to do this relatively infrequently (again, I rely on my daily database backups and the backups my hosting company performs).

After some discussion with one of the NOC engineers, here is what worked REALLY simply for me (using cPanel):

1)  I have a secondary hosting plan at another hosting company (i.e., good backups = all eggs NOT= one basket).

2)  I logged into root cPanel / WHM and edited the backup settings under “Configure Backup”.  I selected all the options to only backup accounts and their respective databases. 

3)  I loaded the FTP info for the other host

4)  I then manually selected my most important accounts to be backed up

5)  I logged into ssh as root and ran the command:  /scripts/cpbackup

6)  For the system to completely tar, backup, and FTP 26 accounts and a total of 3.1 gb of compressed data took about 1 hour in the background.   I didn’t see any obvious changes to server load (I did this at about 10:00 pm PST).

7)  I downloaded all the files to my desktop via my DSL line:  Transferred 26 files totaling 3,146,615 KB in 6 hours 30 minutes 55 seconds (134.2 KB/s).

I don’t expect to do this very often… quarterly at most.  I have a lot of faith in my hosting company and their backups but I still like to cover my bases.

UPDATE 2/2/2009
It has been over a year since I posted this thread so I thought I’d do an update, especially as my BYC forum reaches 22,222 members on 2/2.   From the time of the original post my DB for this forum alone has doubled and then doubled again!   Two of the search tables were taking up at least half of that space so I asked Ameir to write a unique version of his backup script that would backup all the tables except the search tables (which can be recreated if they are dumped).    This cut my backup sizes in half, which has made a huge difference in my ability to do frequent backups and store copies in multiple locations.  This process of daily backup stored in multiple locations really helps me sleep better at night.

1. Install a 400 GB SATA drive I got from the Staples ”Black Friday” sale ($59 after rebate)
2. Partition the drive into 2 partitions, one NTFS and one FAT32 (seems easier to work with Ghost using a FAT32 drive).
3. Installed Windows XP onto the NTFS partition of the new drive.
4. Create a full Ghost Image of the brand new XP install AND another Image of the original hard drive from DELL with the Vista OS (just in case I ever want to give Vista a try).
5. Wipe Vista off the original hard drive giving me another 320 GB of storage.

Well, my plan came to a screeching halt at step 4 when my Version of Ghost (which I’ve been using since 1999) kept freezing at the intro screen.  I couldn’t figure out if it was a problem with the SATA drive or something else with the new system… bottom line:  Ghost wouldn’t work. 

What to do, what to do…  I had the choice to try a new version of Ghost or a different solution all together.   I’ve been speaking with my brother and reading some articles on the new versions of Ghost and I wasn’t too impressed.  To the Internet I went and searched for the term “alternatives to ghost” and found a GREAT resource: http://ghost.radified.com/ghost_alternatives.htm.  This is where I found the $19 program “Image For Dos” that offered a free trial.   Well, I loaded it up and it worked like a charm!  I was able to fully backup AND restore my whole hard drive partition data.

Being all about simplicity I decided to try to find a way to automate my backup process.  Here’s what I did:

1. Created a bootable CD using a modified Autoexec.bat file that would automatically run the batch file below on my FAT32 drive (c:/)
2. Used the Batch File Sample for Maintaining a Rotating Backup designed by “Image For Dos” –  This runs the backup script automatically and rotates the backups through the letters A, B, C then back to A.  I edited the script so that it will go through the whole alphabet before it overwrites A again.

So, all I have to do is pop the bootable CD into my computer, reboot, go get some ice cream and by the time I’m back there is a shinny new backup image file waiting for me.

In my original backup procedures article I mentioned that I try to do monthly Images of my hard drive and then use an application to do automated weekly backups of all my important documents and email.   I’ve moved from using “backer” as my synchronizing application to a really great application called SyncBackSE.  I use SyncBackSE to automatically do the following:

Backup all my important files and email to another hard drive / partition

1. Synchronize all the images (and we have a lot) between my computer and my wife’s computer.  This allows both of us to edit the pictures and provides a backup of all our important stuff in case one PC / hard drive fails.
2. Download the daily database files from my websites to my local computer
3. Upload about five VERY important documents (financial info, passwords, etc.) to a secure server that I can access from any PC.
4. Synchronize all my files, pictures, backups to an external hard drive.  This is great because only edited files are changed or added.

Well, I’m pretty happy about how simple and clean my new backup process is.  If you have any comments or suggestions on the applications or the process I’m using please comment below.

Here’s how it plays out about every 6 months:

1) I get tired of creating pages in Dreamweaver and then shooting everything over to the server.
2) I start researching CMS’s (back in the day when I first did this the big ones were Xoops and PhpNuke)
3) I find and install one I think I like
4) I decide it is more hassle than it’s worth (keeping it updated, hacker safe, etc.)

Well, over the past year I’ve been becoming more and more drawn to wards CMS’s for different reasons than pure functionality / tools for me, the publisher. Most of the new reasons have to do with 3 relatively new points of consideration:

1) Blog based systems (see WordPress): SE’s LOVE blogs
2) RSS: It would be great to have readers subscribe to new posts on my site. The old school way of doing this was using a newsletter to tell people there are new articles… what a pain.
3) Comments: I’d love to enable my visitors to add comments to the stuff posted.

Found myself asking the following:

“Do I finally take the big plunge and go all out CMS? Convert all my pages and copy all the data into individual blog posts?

If yes, then which system do I use? Here are my two main contenders: Drupal & WordPress (I know, WP is not a traditional CMS, but you get the point).

For the most part I’m 85% converted to using WordPress. It’s clean, user friendly, has a great community with tons of plugins, SE’s love it, etc. My main problem. I just don’t like the chronologically sorted post structure of Blogs. I’m just not totally sold that a blog format matches my hobby site. This could just be legacy thinking.”

Well, after a few months of going back and forth I decided the pros outweigh the cons.  I paid someone to help me move every page, one at a time into WordPress.  There are still quite a few bugs and kinks to work out, but my main questions / issues were resolved. 

The biggest issue: keeping existing PageRank and back links to the pages without having to create 100 redirects.  WordPress allowed me to use “Custom Permalinks” to have all my posts look like php pages.  Naming the post the same as the file name resulted in a perfect (90% of the time) match up between the old file url and the new blog post url.

Background: We’ve had the Series 2 Tivo made by Toshiba, the SDH-400 (the one with free basic service) for the past year. We’ve been very happy with it. Tivo has converted us to the world of DVR and we realize now we’ll never go back to non-recorded TV. The only problems were: (1) We needed a second DVR unit for my parents and (2) We didn’t want to shell out the extra $13 a month for the rest of the features like “Season Pass” or “Wishlist”.

We opted to upgrade our 2 Dish Network receivers for a dual tuner DVR receiver, the 625 model. We were already paying an extra $5 a month for the second receiver, so upgrading to the 625 meant simply adding an extra $5 a month for what turned out to be two DVR units with full functionality (not the watered down “Basic” service we were getting with our Tivo).

A few instant things we’ve noticed over our first week:

Pros to Dish Network’s DVR over the Tivo:

• Overall, cheaper for more features.

• Better / seamless integration since the receiver and DVR are made to go together.

• Better quality recordings (we were getting about 80 hours of low quality on the Tivo and about 100 hours of good quality on our Dish Network DVR).

Cons to Dish Network’s DVR compared to Tivo:
(Disclaimer: A lot of these are nit picky items that we probably just need to adapt to)

• It seems to take way more button presses to get the same tasks done.

• After watching / deleting a recorded show on Dish you are brought back to the top of the list of recorded shows. With Tivo you were put back in the same position on the list as you were before.
• When you leave a recorded show for live TV and then go back to the recorded show it doesn’t seem to remember your position (where you left off) and you have to FF to find your spot.
• Speaking of FF, one of Tivo’s coolest functions we miss the most is this: During FF when you hit play it automatically skips back about 5 seconds since 99.99% of the population (and technology) won’t react fast enough once you reach the spot you want.
• Tivo was always recording 30 mins worth of any show no matter what (until you changed the channel). So far it seems that no matter what we do, when we are watching something live on Dish, then go to recorded shows, and then go back to “live tv” there isn’t any history of the live show that was playing during the time we were watching recorded shows. With Tivo we could go back to live TV and watch the last 30 mins of whatever channel was on.
• We haven’t found a way to search for a show (and all episodes) without going through the process of setting up a recording.
• When pushing record on an item in the guide the default is to record all instead of just the single occurrence.

As I mentioned, it has only been a week, so we’re still learning all the ins and outs. I’m especially interested in learning all the secrets and shortcuts (tips and tricks) of using this unit.

Update 10/4/07: I received the following email that I really enjoyed and wante to share:

“I just read your article and I can’t tell you how much I agree, at least with the cons. My wife and I have had TiVo for 2 years and just recently switched to Dish network and the DVR. After 3 months, we can’t stand it. We are considering paying the early contract cancellation fee to get rid of Dish Network and go back to DirecTV and TiVo.

As for your Pros, here is our opinion.

We think the recordings are of less quality than the TiVo, at least it seems that way. I don’t remember a quality issue with TiVo.

The extra 20 hours hasn’t made a difference because we never used all of the 80 hours.

As for more features, I don’t really care for any of the additional features, as they simply clutter the interface and make it more difficult to use.

I develop software for a living and am often tasked with designing user interfaces. I can tell you from years of experience that the user interface for the Dish Network DVR is horrible. I would never let an application out the door with that confusing of an interface. The TiVo interface was much more ‘user friendly’.

To sum up, we will be switching back to TiVo as soon as our Dish Network contract expired and perhaps even sooner.

Feel free to post this to your web site if you like”

I’ve had a bit of experience with a plethora of web hosting companies over the years and thought I’d post some of my thoughts and experiences over the years.

Speaking of “a plethora of web hosting companies” here are the ones I’ve dealt with over the years (that I can remember) in order of when I had my very first family website with e-access.net:

e-access.net – had our family website here for about 3 years starting around 1998. Their site hasn’t changed since.
tripod.com
geocities.com
hostonce.com – BAD COMPANY. Lots of downtime, terrible support.
freeservers.com
netfirms.com
cfm-resources.com – a bit of downtime
ehostingbiz.com – pretty good product and support
worldwebhosters.com – bad support and product
host4students.com – very short stay
prowebspace.com – started off really well, but couldn’t handle growth and had lots of unexplained downtime and data loss
dathorn.com – Very happy with them, but outgrew their services after a bit over a year

I’ve now moved to a server that provides me with way more resources, features, and access than I’ve ever had before!

Update – 1/11/09:

Boy, I haven’t touched this page in a while!   I’ve been at my current datacenter for almost 5 years and couldn’t be happier.   I’m paying a premium for top of the line hardware, management, and trunk internet connections, but it is worth it to have the security and scalability, especially as BYC grows.

One thing that has been frustrating over the years, especially since taking on BYC, is dealing with some people that don’t have server management experience that say, “Why not use one of those unlimited hosting plans for $5.00 a month!?”    It is really hard to convey to some people that these cheap plans are usually provided by companies that are overselling their servers. 

I learned the hard way that these cheap plans (50 GB disk space and 500 GB transfer) are perfect for a family, hobby, small business site, but try loading a CPU / Memory intensive database driven website and you’ll quickly get a SUSPENDED notice from the host and wake up to 300 emails from clients and site visitors (I know from experience).   You, like me, will say, “But I didn’t go over my disk space or bandwidth!!!???”.   The host will either not answer you or will say that you need to get a dedicated server (I’ve received that email a few times too).

There is a light at the end of this tunnel!  Every once in a while I run across members of my communities with server management and/or large website hosting experience and find that they totally get it.  They empathise with me and know that as the site’s usage grows linearly that the complexity of running the services and keeping them up, along with other challenges grow exponentially.