Downtime today - diyAudio
Go Back   Home > Forums > Site > Forum Problems

Forum Problems If there is a forum related problem please leave a message here so an Admin will see it.

Please consider donating to help us continue to serve you.

Ads on/off / Custom Title / More PMs / More album space / Advanced printing & mass image saving
Reply
 
Thread Tools Search this Thread
Old 30th June 2011, 07:08 AM   #1
Jason is offline Jason  Australia
:)
diyAudio Administrator
 
Jason's Avatar
 
Join Date: Oct 2000
Location: Melbourne
Blog Entries: 1
Default Downtime today

My sincere apologies for the 9 hours of downtime today. When I "improved" the backups a few months ago I switched the write location to the main partition instead of the backup partition and today during the "PST/EST dawn backups" the main drive filled up which linux really doesn't like happening, and it killed the webserver. I was just going to sleep at that time and didn't get notification until I woke up, hence the 8 hour delay in my response.

For the record, our server is monitored and managed by our host, and they did detect the problem immediately, logged in and attempted to resolve it, but weren't familiar enough with the configuration to be able to resolve the problem. We're working with them to ensure that in the future they have enough information on our (rather hardy/secure/locked down) configuration to be able to fix this kind of problem without additional assistance from myself in the future.

As this is the kind of model we're shooting for now (if stuff breaks it gets fixed immediately by the server management company), I guess this was a good real world test, and in this case it failed and again my sincere apologies for the interruption to your service. We'll do what it takes to ensure this can't happen again in the future.

Jason

Last edited by Jason; 30th June 2011 at 07:11 AM.
  Reply With Quote
Old 30th June 2011, 07:55 AM   #2
diyAudio Member
 
Join Date: Feb 2008
Location: Jackson,michigan
Thanks Jason for letting us know what happened which is more than what other services would do!

Ironicaly I was just on and then went off to do a virus scan and then came right back and then nothing ,so ,I thought that my scanner had messed up something as it has happend to me before.

But by using one of my backup computers I figured that it must have been something with your server,Shooo, WoW I'm glad that you are back online !

jer
  Reply With Quote
Old 30th June 2011, 07:58 AM   #3
gpapag is offline gpapag  Greece
diyAudio Member
 
gpapag's Avatar
 
Join Date: Nov 2002
Location: Athens-Greece
I was a bit worried as to what happened.
I am illiterate on such stuff (what you wrote sound Greek to me ), but I don’t think it is a bad idea to deliberately program for some regular downtimes in the future, to give us a chance to do some diyaudio real work off-line.

Jason, I am glad that You are back and catering for this wonderful site.
I hope You are doing well.

Regards
George

PS. Against the “common wisdom” here, a few hours sleep each day is a normal activity.
__________________
["Second Law is a bitch." - SY] ["The Road To Heaven:Specify the performance & accept the design. The Road To Hell:Specify the design & accept the performance"-Bruno Putzeys]
  Reply With Quote
Old 30th June 2011, 12:08 PM   #4
Jason is offline Jason  Australia
:)
diyAudio Administrator
 
Jason's Avatar
 
Join Date: Oct 2000
Location: Melbourne
Blog Entries: 1
I just sorted out a few more problems with the revised backup routines which just caused an extended maintenance period, things should be back to normal now.
  Reply With Quote
Old 30th June 2011, 01:45 PM   #5
ptempel is offline ptempel  United States
diyAudio Member
 
Join Date: Jul 2008
+1 on geraldfryjr's comment. Good to be informed but its ok to have a glitch once in a while. Unlike audio and electronics, I have some experience with being a Unix admin (only over 12 years). Let me know if I can help with anything. Getting the operations team on board will require some documentation. Maybe they have a wiki site for that? That's what I used for the last three jobs to document proceedures and changes. If for nothing else than to remind me of "how it was done."
  Reply With Quote
Old 30th June 2011, 01:52 PM   #6
diyAudio Member
 
Join Date: May 2011
Location: Portland Oregon
Glad we're back......wish I had a dollar for every time I tried to log on yesterday......I could get a few projects done!
  Reply With Quote
Old 30th June 2011, 01:57 PM   #7
49 - for the 18th time - again!
diyAudio Member
 
c2cthomas's Avatar
 
Join Date: Oct 2006
Location: Near "Music City" (Nashville Tennessee)
The down time proved to me that I'm waaaaaaay to addicted to diyAudio!!! Man - I wuz Jones'n for my diyAudio!!!! Maybe I should get a life......

Thanks Jason for getting things put right and letting us know what is going on.
__________________
DIY audio can be expensive – but getting to see things go up in smoke - that's priceless!!!! ..... "whatever - call it brainfart of Mighty ZM"
  Reply With Quote
Old 30th June 2011, 03:41 PM   #8
diyAudio Member
 
Join Date: Feb 2008
Location: Jackson,michigan
I second that c2cthomas I am on here more helping others than getting my own projects done!
But thats okay it is what I enjoy doing.
Thanks DIYAudio for everything that you stand for!


jer
  Reply With Quote
Old 1st July 2011, 05:04 AM   #9
Jason is offline Jason  Australia
:)
diyAudio Administrator
 
Jason's Avatar
 
Join Date: Oct 2000
Location: Melbourne
Blog Entries: 1
Thanks ptempel for the offer.

Our host now has all the information and our special requirements in case of an emergency, backups are now back on their own partition (rolleyes), I'm getting texts and phone calls if disk space ever gets unplentiful, or the site becomes unresponsive, and our senior admins on here have a direct line and verbal passwords to our host in case of extended drama. It's been a long time since we last had any downtime, hopefully it will be another year or two until the next problem
  Reply With Quote
Old 1st July 2011, 09:58 AM   #10
Previously: Kuei Yang Wang
 
Join Date: Nov 2002
Hi Jason,

Quote:
Originally Posted by Jason View Post
My sincere apologies for the 9 hours of downtime today.
Don't worry about it. It's not like DIYA is "business critical".

It's just a "downtime" thing for those who post here.

Occasional breaks are probably a good thing.

You should probably schedule 24 Hour outages every two weeks or so...

Appreciate the hard work to keep everything running and getting things back up to scratch when Murphy strikes...

Ciao T

PS, I remember once nearly 2 Weeks outage on a business critical Mini system, it was the Payroll...
  Reply With Quote

Reply


Hide this!Advertise here!
Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Downtime Jason Site Announcements 5 9th April 2010 05:11 AM
Scheduled Downtime this week AudioFreak Site Announcements 1 30th August 2009 09:37 PM
Scheduled Downtime AudioFreak Site Announcements 12 4th May 2009 06:17 PM
Forum downtime Jason Site Announcements 0 14th May 2003 09:14 AM


New To Site? Need Help?

All times are GMT. The time now is 01:48 PM.


vBulletin Optimisation provided by vB Optimise (Pro) - vBulletin Mods & Addons Copyright © 2014 DragonByte Technologies Ltd.
Copyright ©1999-2014 diyAudio

Content Relevant URLs by vBSEO 3.3.2