Internetworkers: http://www.ibiblio.org/internetworkers/

["Tony Spencer" <tony AT tonyspencer.com>]

I lease a server running RedHat Linux.  Recently it has started disappearing
for no apparent reason.  When I say disappearing I mean that there is no
response on port 80, I can't FTP to it, I can't SSH2, and I can't even ping
it.  I have to request my ISP to reboot it.  It always comes back up with no
problem after a reboot.  I'm not a sysadmin and  I've usually managed fine
but this one is boggling me.  Can anyone give me any more pointers for what
to look for?  I'm wondering if the firewall is closing everything for some
reason.

Here is the situation:
RedHat v3.1.0
2 GB of RAM
Running two websites.  Load rarely tops 0.2 and usually is 0 to 0.1 I ran
chkrootkit as well as Rootkit Hunter and found no problems.
Telnet is off.  
Runs APF firewall with the following ports open:
TCP_CPORTS="
21,22,25,26,53,80,110,143,443,2082,2083,2086,2087,2095,2096,19638" 
UDP_CPORTS="53"

Here are logs just before and after the disappearing act.  I think the
proftpd stuff is just a monitor on the box checking whether or not FTP is
working.  I don't understand why the apache logs actually go backwards in
time at one point (14:38:37 to 13:35:21).

-----------------------------------------/var/log/messages------------------
----------------
Dec  1 11:48:47 server1 proftpd[3892]: server1.mydomain.com
(localhost[127.0.0.1]) - FTP session closed. 
Dec  1 11:57:07 server1 proftpd[4500]: server1.mydomain.com
(localhost[127.0.0.1]) - FTP no transfer timeout, disconnected Dec  1
11:57:07 server1 proftpd[4500]: server1.mydomain.com
(localhost[127.0.0.1]) - FTP session closed. 
Dec  1 12:05:28 server1 proftpd[5466]: server1.mydomain.com
(localhost[127.0.0.1]) - FTP no transfer timeout, disconnected Dec  1
12:05:28 server1 proftpd[5466]: server1.mydomain.com
(localhost[127.0.0.1]) - FTP session closed. 
Dec  1 12:13:48 server1 proftpd[6336]: server1.mydomain.com
(localhost[127.0.0.1]) - FTP no transfer timeout, disconnected Dec  1
12:13:48 server1 proftpd[6336]: server1.mydomain.com
(localhost[127.0.0.1]) - FTP session closed. 
Dec  1 12:22:08 server1 proftpd[7163]: server1.mydomain.com
(localhost[127.0.0.1]) - FTP no transfer timeout, disconnected Dec  1
12:22:08 server1 proftpd[7163]: server1.mydomain.com
(localhost[127.0.0.1]) - FTP session closed. 
Dec  1 12:30:28 server1 proftpd[8289]: server1.mydomain.com
(localhost[127.0.0.1]) - FTP no transfer timeout, disconnected Dec  1
12:30:28 server1 proftpd[8289]: server1.mydomain.com
(localhost[127.0.0.1]) - FTP session closed. 
Dec  1 12:38:48 server1 proftpd[8787]: server1.mydomain.com
(localhost[127.0.0.1]) - FTP no transfer timeout, disconnected Dec  1
12:38:48 server1 proftpd[8787]: server1.mydomain.com
(localhost[127.0.0.1]) - FTP session closed. 
Dec  1 12:47:08 server1 proftpd[9280]: server1.mydomain.com
(localhost[127.0.0.1]) - FTP no transfer timeout, disconnected Dec  1
12:47:09 server1 proftpd[9280]: server1.mydomain.com
(localhost[127.0.0.1]) - FTP session closed. 
Dec  1 12:55:29 server1 proftpd[9713]: server1.mydomain.com
(localhost[127.0.0.1]) - FTP no transfer timeout, disconnected Dec  1
12:55:29 server1 proftpd[9713]: server1.mydomain.com
(localhost[127.0.0.1]) - FTP session closed. 
Dec  1 13:03:49 server1 proftpd[10167]: server1.mydomain.com
(localhost[127.0.0.1]) - FTP no transfer timeout, disconnected Dec  1
13:03:49 server1 proftpd[10167]: server1.mydomain.com
(localhost[127.0.0.1]) - FTP session closed. 
Dec  1 13:12:09 server1 proftpd[10635]: server1.mydomain.com
(localhost[127.0.0.1]) - FTP no transfer timeout, disconnected Dec  1
13:12:09 server1 proftpd[10635]: server1.mydomain.com
(localhost[127.0.0.1]) - FTP session closed. 
Dec  1 13:15:29 server1 proftpd[11067]: server1.mydomain.com
(localhost[127.0.0.1]) - FTP session opened. 
Dec  1 13:15:29 server1 proftpd[11067]: server1.mydomain.com
(localhost[127.0.0.1]) - FTP session closed. 
Dec  1 13:28:49 server1 proftpd[11484]: server1.mydomain.com
(localhost[127.0.0.1]) - FTP no transfer timeout, disconnected Dec  1
13:28:50 server1 proftpd[11484]: server1.mydomain.com
(localhost[127.0.0.1]) - FTP session closed. 
Dec  1 14:38:13 server1 syslogd 1.4.1: restart.

--------------------------------Apache error_log---------------------------
[Wed Dec  1 13:20:14 2004] [error] [client 65.33.122.230] File does not
exist: /home/myuser/public_html/images/trans.gif
[Wed Dec  1 13:20:18 2004] [error] [client 66.243.50.46] File does not
exist: /home/myuser/public_html/images/trans.gif
[Wed Dec  1 13:23:02 2004] [error] [client 66.206.96.196] File does not
exist: /home/myuser/public_html/images/trans.gif
[Wed Dec  1 13:24:24 2004] [error] [client 209.30.109.245] File does not
exist: /home/myuser/public_html/images/trans.gif
[Wed Dec  1 13:25:04 2004] [error] [client 24.119.238.68] File does not
exist: /home/myuser/public_html/images/trans.gif
[Wed Dec  1 13:26:26 2004] [error] [client 66.139.27.254] File does not
exist: /home/myuser/public_html/images/trans.gif
[Wed Dec  1 14:38:37 2004] [warn] pid file /usr/local/apache/logs/httpd.pid
overwritten -- Unclean shutdown of previous Apache run?
[Wed Dec  1 14:38:37 2004] [crit] (98)Address already in use: make_sock:
could not bind to port 443
[Wed Dec  1 14:38:37 2004] [notice] Apache/1.3.29 (Unix) Resin/2.1.12
mod_auth_passthrough/1.8 mod_log_bytes/1.2 mod_bwlimited/1.4 P
HP/4.3.3 FrontPage/5.0.2.2634 mod_ssl/2.8.16 OpenSSL/0.9.7a configured --
resuming normal operations [Wed Dec  1 14:38:37 2004] [notice] suEXEC
mechanism enabled (wrapper:
/usr/local/apache/bin/suexec)
[Wed Dec  1 14:38:37 2004] [notice] Accept mutex: sysvsem (Default: sysvsem)
[Wed Dec  1 13:35:21 2004] [notice] SIGUSR1 received.  Doing graceful
restart [Wed Dec  1 13:35:21 2004] [notice] Apache/1.3.29 (Unix)
Resin/2.1.12
mod_auth_passthrough/1.8 mod_log_bytes/1.2 mod_bwlimited/1.4 P
HP/4.3.3 FrontPage/5.0.2.2634 mod_ssl/2.8.16 OpenSSL/0.9.7a configured --
resuming normal operations [Wed Dec  1 13:35:21 2004] [notice] suEXEC
mechanism enabled (wrapper:
/usr/local/apache/bin/suexec)
[Wed Dec  1 13:35:21 2004] [notice] Accept mutex: sysvsem (Default: sysvsem)
[Wed Dec  1 13:35:21 2004] [error] [client 12.219.125.81] File does not
exist: /home/myuser/public_html/free-stuff/free-download-ant