Unexpected outage - diyAudio
Go Back   Home > Forums > Site > Forum Problems

Forum Problems If there is a forum related problem please leave a message here so an Admin will see it.

Please consider donating to help us continue to serve you.

Ads on/off / Custom Title / More PMs / More album space / Advanced printing & mass image saving
Reply
 
Thread Tools Search this Thread
Old 28th May 2013, 09:58 AM   #1
Jason is offline Jason  Australia
:)
diyAudio Administrator
 
Jason's Avatar
 
Join Date: Oct 2000
Location: Melbourne
Blog Entries: 1
Default Unexpected outage

About 4 hours ago, while I was sleeping, our regular web server shut down / was replaced by the generic apache web server. I'm currently trying to work out what happened.
  Reply With Quote
Old 28th May 2013, 10:03 AM   #2
Gyuri is offline Gyuri  Hungary
diyAudio Member
 
Gyuri's Avatar
 
Join Date: Nov 2003
Location: Budapest
sucks
__________________
Those who would have deserved the life they are all dead already. "Oh, my dead dears!" - Captain Cat
  Reply With Quote
Old 28th May 2013, 10:05 AM   #3
Jason is offline Jason  Australia
:)
diyAudio Administrator
 
Jason's Avatar
 
Join Date: Oct 2000
Location: Melbourne
Blog Entries: 1
Ok looks like "while you sleep" caretakers of our managed server detected a problem, logged in 4 hours ago, and then restarted the wrong apache.

I'm very sorry for the inconvenience.

I'll certainly be making sure that doesn't happen again.
  Reply With Quote
Old 28th May 2013, 10:25 AM   #4
diyAudio Member
 
zeonrider's Avatar
 
Join Date: Mar 2004
Location: Belgrade RS
Quote:
Originally Posted by Jason View Post
About 4 hours ago, while I was sleeping, our regular web server shut down / was replaced by the generic apache web server. I'm currently trying to work out what happened.
Gremlins

Regards zeoN_Rider
  Reply With Quote
Old 28th May 2013, 03:33 PM   #5
Jason is offline Jason  Australia
:)
diyAudio Administrator
 
Jason's Avatar
 
Join Date: Oct 2000
Location: Melbourne
Blog Entries: 1
The hosting company's monitoring service detected that port 80 was not responding on the server. I'm currently trying to find out if that was the case or a false alarm. Things appeared to be going swimmingly up until a point and then everything dropped to zero. It could well have been a teething problem in the new setup. I'm going to be investigating that and monitoring things closely as time goes on.

When that happened, one of their techs logged in to save the day, but didn't follow their standard operating procedures which included instructions on how to properly restart the web server (this was documented 2 years ago when we last had an issue). A sequence of other non SOP events unfolded from there, resulting in the site being down for 4 hours. The host has taken full responsibility for their failure to following SOP and promises this won't happen again in the future. I'll be taking proactive steps to "future proof" things so even if SOP is not followed in the future, it will be hard to bork it up again.

Sorry for the downtime.

Other than this very consequential "glitch", the server is coasting along at record low load averages and serving up pages faster than ever. Hopefully it stays that way, I'll be baby sitting things in the meantime...
  Reply With Quote
Old 28th May 2013, 04:17 PM   #6
Zen Mod is offline Zen Mod  Serbia
diyAudio Member
 
Zen Mod's Avatar
 
Join Date: Jan 2003
Location: ancient Batsch , behind Iron Curtain
tnx Jason

we are grateful for your good work
__________________
my Papa is smarter than your Nelson !
clean thread; Cook Book;PSM LS Cook Book;Baby DiyA ;Mighty ZM's Bloggg;Papatreasure;Papa... by Mighty ZM
  Reply With Quote
Old 28th May 2013, 06:31 PM   #7
Jason is offline Jason  Australia
:)
diyAudio Administrator
 
Jason's Avatar
 
Join Date: Oct 2000
Location: Melbourne
Blog Entries: 1
Case closed.

Various things have been implemented to make this (more) idiot proof next time round. Here's hoping!
  Reply With Quote
Old 29th May 2013, 12:12 AM   #8
zany is offline zany  Australia
diyAudio Member
 
Join Date: Oct 2010
It's a real problem when 'idiots' are managing server farms... :-(
  Reply With Quote
Old 29th May 2013, 12:28 AM   #9
Sprags is offline Sprags  United States
diyAudio Member
 
Join Date: Mar 2013
I think if the computers used tried and true 75 year old tube technology there probably wouldn't have been an issue...but noooo...those computer geeks think transistors are more reliable.
  Reply With Quote
Old 29th May 2013, 02:15 AM   #10
Johno is offline Johno  Australia
diyAudio Member
 
Johno's Avatar
 
Join Date: Oct 2010
Location: Up in the alps - motorcycle heaven
I live on a farm and we use electric fencing to keep the cattle on the correct side. About 10kV but very low joules.
Works really well, one individual tests the envelope and gets a zap and the rest of the herd learns from the squeal and respects all fencing for the next few weeks.

I have thought about introducing similar "learning" tools at work but the health and safety people tell me there are a number of issues and will not let me. Wonder if the issues apply to contractors too?
  Reply With Quote

Reply


Hide this!Advertise here!
Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Unexpected resonance Alex M Multi-Way 32 17th June 2011 08:56 AM
Unexpected results from a budget stereo!! somebizarredude Everything Else 10 9th May 2011 05:39 PM
Forward SMPS - unexpected behaviour vectorplane Power Supplies 19 18th December 2005 02:28 PM
NE US power outage: Da5id4Vz Everything Else 3 16th August 2003 06:37 PM
Outage AudioFreak Forum Problems 3 5th February 2003 09:48 PM


New To Site? Need Help?

All times are GMT. The time now is 05:24 PM.


vBulletin Optimisation provided by vB Optimise (Pro) - vBulletin Mods & Addons Copyright © 2014 DragonByte Technologies Ltd.
Copyright 1999-2014 diyAudio

Content Relevant URLs by vBSEO 3.3.2