Over the years, I've accumulated way more hosting accounts than I can keep track of, including four dedicated servers.
It's easy to lose track of things as we move onto new projects and the old ones just hum along.
I had a catastrophe this morning caused by poorly configured server. If you're on a shared host, this likely cannot happen as the hosting provider will keep an eye on things. But with dedicated servers, you're given more than enough rope to hang yourself.
On one site, I set up some extra error logging to record the SQL queries that run, along with their timings, so I could find the poorly performing queries. This was back in March or so. When I was finished, I forgot to turn off the logging.
There's a file size limit of 2GB in Linux (at least the version I'm running), and my site's error log reached that limit this morning.
Apache isn't terribly informative about this, it just terminates the connection. I found the culprit by looking in the apache error_log file. I found 1,678 of them to be exact (ouch).
[Tue Aug 29 00:45:14 2006] [notice] child pid 10249 exit signal File size limit exceeded (25)
It took me a quite a few nervous minutes to find just WHICH file. I renamed the error log file, restarted apache to let it recreate the new log files and everything was good again.
Adding to my long list of things to keep in mind as I run my servers:
344: make sure log files are rotating
345: keep an eye on growing files with this shell command:
find /* -size +999999k | xargs ls -lah
That command will show you any files that are 1GB or larger.
This little lesson cost me a few hundred bucks in PPC costs and lost sales. It's free for you.
RSS
Over the years, I've accumulated way more hosting accounts than I can keep track of, including four dedicated servers.
It's easy to lose track of things as we move onto new projects and the old ones just hum along.
I had a catastrophe this morning caused by poorly configured server. If you're on a shared host, this likely cannot happen as the hosting provider will keep an eye on things. But with dedicated servers, you're given more than enough rope to hang yourself.
On one site, I set up some extra error logging to record the SQL queries that run, along with their timings, so I could find the poorly performing queries. This was back in March or so. When I was finished, I forgot to turn off the logging.
There's a file size limit of 2GB in Linux (at least the version I'm running), and my site's error log reached that limit this morning.
Apache isn't terribly informative about this, it just terminates the connection. I found the culprit by looking in the apache error_log file. I found 1,678 of them to be exact (ouch).
[Tue Aug 29 00:45:14 2006] [notice] child pid 10249 exit signal File size limit exceeded (25)It took me a quite a few nervous minutes to find just WHICH file. I renamed the error log file, restarted apache to let it recreate the new log files and everything was good again.
Adding to my long list of things to keep in mind as I run my servers:
344: make sure log files are rotating
345: keep an eye on growing files with this shell command:
find /* -size +999999k | xargs ls -lahThat command will show you any files that are 1GB or larger.
This little lesson cost me a few hundred bucks in PPC costs and lost sales. It's free for you.