Issue: Pages have been timing out
Analysis: Services and the server itself have not been restarted for a long time.
Action Taken: Services restarted. Full server reboot performed.
Outlook: Still looking choppy. Situation being monitored.
Analysis: Services and the server itself have not been restarted for a long time.
Action Taken: Services restarted. Full server reboot performed.
Outlook: Still looking choppy. Situation being monitored.
Last edited:
Issue: Pages have been timing out
Analysis: The majority of web traffic is coming from bots aggressively spidering the site.
Action Taken: Blocks have been put in place to block bots hogging server resources.
Outlook: Page timeouts still evident. Situation being monitored.
Analysis: The majority of web traffic is coming from bots aggressively spidering the site.
Action Taken: Blocks have been put in place to block bots hogging server resources.
Outlook: Page timeouts still evident. Situation being monitored.
Last edited:
Issue: Site down (database unavailable)
Analysis: Database crashed
Action Taken: Database repaired, dumped, reimported, and rechecked. This caused 1 hour of downtime.
Outlook: Page timeouts still evident. Situation being monitored.
Analysis: Database crashed
Action Taken: Database repaired, dumped, reimported, and rechecked. This caused 1 hour of downtime.
Outlook: Page timeouts still evident. Situation being monitored.
Last edited:
Issue: Pages still timing out
Analysis: Tracing of database queries showed that searches were creating traffic jams for the database
Action Taken: Search has been disabled for the next 24 hours while we monitor the situation.
In the meantime please use Google for your searches using the search format:
site:diyaudio.com beer
Outlook: At the time of writing this post, server behaviour is back to normal, with average response time matching historical averages of 85-120ms.
We will wait 24 hours and monitor the situation before making a decision on how to restore search without it impacting the regular use of the site.
Analysis: Tracing of database queries showed that searches were creating traffic jams for the database
Action Taken: Search has been disabled for the next 24 hours while we monitor the situation.
In the meantime please use Google for your searches using the search format:
site:diyaudio.com beer
Outlook: At the time of writing this post, server behaviour is back to normal, with average response time matching historical averages of 85-120ms.
We will wait 24 hours and monitor the situation before making a decision on how to restore search without it impacting the regular use of the site.
Last edited:
Issue: Pages timing out
Analysis: Combination of vBulletin and plugins with MyISAM table type causing the issue
Action Taken: Table for thread upgraded from MyISAM to InnoDB, search re-enabled, search disabled for non-logged in members
Outlook: Excellent
Analysis: Combination of vBulletin and plugins with MyISAM table type causing the issue
Action Taken: Table for thread upgraded from MyISAM to InnoDB, search re-enabled, search disabled for non-logged in members
Outlook: Excellent
Issue: Infrequent 1 minute delays after making a post
Analysis: Post table causing locks
Action Taken: Posts table upgraded from MyISAM (table level locks) to InnoDB (row level locks)
Outlook: Situation being monitored, bad news is not expected
Analysis: Post table causing locks
Action Taken: Posts table upgraded from MyISAM (table level locks) to InnoDB (row level locks)
Outlook: Situation being monitored, bad news is not expected
Unexpected downtime just now of ~10 minutes, unrelated to these issues but part of ongoing optimisations, now resolved.
Downtime just now of ~5 minutes while the user table was upgraded to InnoDB, to prevent another class of locks occurring. This was not announced in advance due to the short duration of the downtime. Apologies for the inconvenience.
- Status
- Not open for further replies.