[Wapt] Arrêt aléatoire du serveur wapt

[Frederic Garesche]

Re, 

Les services fonctionnent : 

systemctl status waptserver.service 
● waptserver.service - WAPT Server startup script 
Loaded: loaded (/usr/lib/systemd/system/waptserver.service; enabled; vendor preset: enabled) 
Active: active (running) since Tue 2019-10-22 12:28:47 CEST; 1 day 21h ago 
Main PID: 1527 (python) 
Tasks: 1 (limit: 36864) 
CGroup: /system.slice/waptserver.service 
└─1527 /opt/wapt/bin/python /opt/wapt/waptserver/server.py 

oct. 24 09:31:01 wapt2.insa-rennes.fr python[1527]: 2019-10-24 09:31:01,977 WARNING SocketIO connection refused for uuid 4C4C4544-004C-521 
oct. 24 09:33:40 wapt2.insa-rennes.fr python[1527]: 2019-10-24 09:33:40,975 WARNING SocketIO connection refused for uuid 4C4C4544-004E-591 
oct. 24 09:34:01 wapt2.insa-rennes.fr python[1527]: 2019-10-24 09:34:01,811 WARNING SocketIO connection refused for uuid 4C4C4544-0035-471 
oct. 24 09:36:01 wapt2.insa-rennes.fr python[1527]: 2019-10-24 09:36:01,806 WARNING SocketIO connection refused for uuid 4C4C4544-0036-471 
oct. 24 09:37:00 wapt2.insa-rennes.fr python[1527]: 2019-10-24 09:37:00,858 WARNING SocketIO connection refused for uuid 4C4C4544-004E-571 
oct. 24 09:39:11 wapt2.insa-rennes.fr python[1527]: 2019-10-24 09:39:11,128 WARNING SocketIO connection refused for uuid 4C4C4544-0054-321 
oct. 24 09:42:50 wapt2.insa-rennes.fr python[1527]: 2019-10-24 09:42:50,226 WARNING SocketIO connection refused for uuid 4C4C4544-0050-331 
oct. 24 09:42:56 wapt2.insa-rennes.fr python[1527]: 2019-10-24 09:42:56,099 WARNING SocketIO connection refused for uuid 4C4C4544-0050-321 
oct. 24 09:48:01 wapt2.insa-rennes.fr python[1527]: 2019-10-24 09:48:01,713 WARNING SocketIO connection refused for uuid 4C4C4544-0050-321 
oct. 24 09:54:42 wapt2.insa-rennes.fr python[1527]: 2019-10-24 09:54:42,395 WARNING SocketIO connection refused for uuid 773F4CFF-233A-11E 

systemctl status nginx.service 
● nginx.service - A high performance web server and a reverse proxy server 
Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled) 
Drop-In: /etc/systemd/system/nginx.service.d 
└─nginx_worker_files_limit.conf 
Active: active (running) since Tue 2019-10-22 12:28:45 CEST; 1 day 21h ago 
Docs: man:nginx(8) 
Process: 1335 ExecStart=/usr/sbin/nginx -g daemon on; master_process on; (code=exited, status=0/SUCCESS) 
Process: 1293 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=0/SUCCESS) 
Main PID: 1340 (nginx) 
Tasks: 17 (limit: 36864) 
CGroup: /system.slice/nginx.service 
├─1340 nginx: master process /usr/sbin/nginx -g daemon on; master_process on; 
├─1341 nginx: worker process 
├─1342 nginx: worker process 
├─1343 nginx: worker process 
├─1346 nginx: worker process 
├─1347 nginx: worker process 
├─1348 nginx: worker process 
├─1349 nginx: worker process 
├─1350 nginx: worker process 
├─1351 nginx: worker process 
├─1352 nginx: worker process 
├─1354 nginx: worker process 
├─1355 nginx: worker process 
├─1356 nginx: worker process 
├─1357 nginx: worker process 
├─1358 nginx: worker process 
└─1359 nginx: worker process 

oct. 22 12:28:45 wapt2.insa-rennes.fr systemd[1]: Starting A high performance web server and a reverse proxy server... 
oct. 22 12:28:45 wapt2.insa-rennes.fr nginx[1293]: nginx: [warn] "ssl_stapling" ignored, issuer certificate not found 
oct. 22 12:28:45 wapt2.insa-rennes.fr nginx[1335]: nginx: [warn] "ssl_stapling" ignored, issuer certificate not found 
oct. 22 12:28:45 wapt2.insa-rennes.fr systemd[1]: Started A high performance web server and a reverse proxy server. 

systemctl status postgresql 
● postgresql.service - PostgreSQL RDBMS 
Loaded: loaded (/lib/systemd/system/postgresql.service; enabled; vendor preset: enabled) 
Active: active (exited) since Tue 2019-10-22 12:28:47 CEST; 1 day 21h ago 
Process: 1525 ExecStart=/bin/true (code=exited, status=0/SUCCESS) 
Main PID: 1525 (code=exited, status=0/SUCCESS) 
Tasks: 0 (limit: 36864) 
CGroup: /system.slice/postgresql.service 

oct. 22 12:28:47 wapt2.insa-rennes.fr systemd[1]: Starting PostgreSQL RDBMS... 
oct. 22 12:28:47 wapt2.insa-rennes.fr systemd[1]: Started PostgreSQL RDBMS. 

Le log fonctionne : 

Oct 24 09:47:00 wapt2 systemd[32266]: Startup finished in 16ms. 
Oct 24 09:47:00 wapt2 systemd[1]: Started User Manager for UID 0. 
Oct 24 09:48:01 wapt2 python[1527]: 2019-10-24 09:48:01,713 WARNING SocketIO connection refused for uuid 4C4C4544-0050-3210-8058-B5C04F354B32, sid 549dcba1b526467aa16d4d396f942344: SocketIO connection not authorized, invalid token: 400 Bad Request: The browser (or proxy) sent a request that this server could not understand., instance 
Oct 24 09:54:42 wapt2 python[1527]: 2019-10-24 09:54:42,395 WARNING SocketIO connection refused for uuid 773F4CFF-233A-11E4-8522-31D4970180FF, sid 83d5ea41409745cd9174de763e41a397: SocketIO connection not authorized, invalid token: 400 Bad Request: The browser (or proxy) sent a request that this server could not understand., instance 
Oct 24 09:55:01 wapt2 CRON[44747]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) 
Oct 24 10:03:51 wapt2 systemd[1]: Started Run anacron jobs. 
Oct 24 10:03:51 wapt2 anacron[44794]: Anacron 2.3 started on 2019-10-24 
Oct 24 10:03:51 wapt2 anacron[44794]: Normal exit (0 jobs run) 
Oct 24 10:03:51 wapt2 systemd[1]: anacron.timer: Adding 1.768916s random time. 
Oct 24 10:05:01 wapt2 CRON[45080]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) 

Que faire ? 

Frédéric GARESCHÉ 
Équipe Assistance et Exploitation 
Direction du Système d'Information (D.S.I) 
Correspondant Département EII et laboratoire IETR 
Tél. : +33 (0)2 23 2 3 82 81 
20 avenue des Buttes de Coësmes 
CS 70839 - 35 708 RENNES Cedex 7 

[ https://www.insa-rennes.fr/ ] 


De: "Frederic Garesche" <frederic.garesche at insa-rennes.fr> 
À: "Hubert TOUVET" <htouvet at tranquil.it> 
Cc: "wapt" <wapt at lists.tranquil.it> 
Envoyé: Jeudi 24 Octobre 2019 09:54:43 
Objet: Arrêt aléatoire du serveur wapt 

Bonjour, 

Ça y est, le serveur est de nouveau tombé. 

Voici la fin du syslog : 

... 
Oct 24 09:35:01 wapt2 CRON[32194]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) 
Oct 24 09:36:01 wapt2 python[1527]: 2019-10-24 09:36:01,806 WARNING SocketIO connection refused for uuid 4C4C4544-0036-4710-804D-C7C04F425832, sid d7dd626b8b1a4d00a164d4b8f130cd4b: SocketIO connection not authorized, invalid token: 400 Bad Request: The browser (or proxy) sent a request that this server could not understand., instance 
Oct 24 09:37:00 wapt2 python[1527]: 2019-10-24 09:37:00,858 WARNING SocketIO connection refused for uuid 4C4C4544-004E-5710-8053-B5C04F354B32, sid 7fef72636b294b999db9bc9dd045ab4d: SocketIO connection not authorized, invalid token: 400 Bad Request: The browser (or proxy) sent a request that this server could not understand., instance 
Oct 24 09:39:11 wapt2 python[1527]: 2019-10-24 09:39:11,128 WARNING SocketIO connection refused for uuid 4C4C4544-0054-3210-804D-CAC04F465931, sid a9ee20f9518645f596057db6c332d6a1: SocketIO connection not authorized, invalid token: 400 Bad Request: The browser (or proxy) sent a request that this server could not understand., instance 
Oct 24 09:42:50 wapt2 python[1527]: 2019-10-24 09:42:50,226 WARNING SocketIO connection refused for uuid 4C4C4544-0050-3310-8057-B5C04F354B32, sid 1ed4b0741b224f869961b1d3e88c8702: SocketIO connection not authorized, invalid token: 400 Bad Request: The browser (or proxy) sent a request that this server could not understand., instance 
Oct 24 09:42:56 wapt2 python[1527]: 2019-10-24 09:42:56,099 WARNING SocketIO connection refused for uuid 4C4C4544-0050-3210-8053-B5C04F354B32, sid bc178e93e17f4062b4fcc8fa77628242: SocketIO connection not authorized, invalid token: 400 Bad Request: The browser (or proxy) sent a request that this server could not understand., instance 
Oct 24 09:45:01 wapt2 CRON[32251]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) 
Oct 24 09:47:00 wapt2 systemd[1]: Created slice User Slice of root. 
Oct 24 09:47:00 wapt2 systemd[1]: Starting User Manager for UID 0... 
Oct 24 09:47:00 wapt2 systemd[1]: Started Session 341 of user root. 
Oct 24 09:47:00 wapt2 systemd[1]: Started Session 342 of user root. 
Oct 24 09:47:00 wapt2 systemd[1]: Started Session 344 of user root. 
Oct 24 09:47:00 wapt2 systemd[32266]: Listening on GnuPG cryptographic agent and passphrase cache (restricted). 
Oct 24 09:47:00 wapt2 systemd[32266]: Reached target Paths. 
Oct 24 09:47:00 wapt2 systemd[32266]: Listening on GnuPG cryptographic agent (access for web browsers). 
Oct 24 09:47:00 wapt2 systemd[32266]: Listening on GnuPG cryptographic agent and passphrase cache. 
Oct 24 09:47:00 wapt2 systemd[32266]: Listening on GnuPG cryptographic agent (ssh-agent emulation). 
Oct 24 09:47:00 wapt2 systemd[32266]: Reached target Timers. 
Oct 24 09:47:00 wapt2 systemd[32266]: Listening on GnuPG network certificate management daemon. 
Oct 24 09:47:00 wapt2 systemd[32266]: Starting D-Bus User Message Bus Socket. 
Oct 24 09:47:00 wapt2 systemd[32266]: Listening on D-Bus User Message Bus Socket. 
Oct 24 09:47:00 wapt2 systemd[32266]: Reached target Sockets. 
Oct 24 09:47:00 wapt2 systemd[32266]: Reached target Basic System. 
Oct 24 09:47:00 wapt2 systemd[32266]: Reached target Default. 
Oct 24 09:47:00 wapt2 systemd[32266]: Startup finished in 16ms. 
Oct 24 09:47:00 wapt2 systemd[1]: Started User Manager for UID 0. 

La commande pids=$(pgrep -f "server.py"); for f in /proc/$pids/fdinfo/*; do cat $f | grep flags | grep -q -v 400[0,1,2] && ls -al /proc/$pids/fd/$(basename $f); done; donne : 
lr-x------ 1 wapt www-data 64 oct. 22 12:28 /proc/1527/fd/0 -> /dev/null 
lrwx------ 1 wapt www-data 64 oct. 22 12:28 /proc/1527/fd/1 -> socket:[15786] 
lr-x------ 1 wapt www-data 64 oct. 22 12:28 /proc/1527/fd/15 -> /dev/urandom 
lrwx------ 1 wapt www-data 64 oct. 22 12:28 /proc/1527/fd/2 -> socket:[15786] 
lrwx------ 1 wapt www-data 64 oct. 22 12:28 /proc/1527/fd/3 -> anon_inode:[eventpoll] 
lr-x------ 1 wapt www-data 64 oct. 22 12:28 /proc/1527/fd/8 -> /dev/urandom 

Cordialement, 


Frédéric GARESCHÉ 
Équipe Assistance et Exploitation 
Direction du Système d'Information (D.S.I) 
Correspondant Département EII et laboratoire IETR 
Tél. : +33 (0)2 23 2 3 82 81 
20 avenue des Buttes de Coësmes 
CS 70839 - 35 708 RENNES Cedex 7 

[ https://www.insa-rennes.fr/ ] 


De: "Frederic Garesche" <frederic.garesche at insa-rennes.fr> 
À: "Hubert TOUVET" <htouvet at tranquil.it> 
Cc: "wapt" <wapt at lists.tranquil.it> 
Envoyé: Mardi 22 Octobre 2019 15:53:08 
Objet: Re: [Wapt] Arrêt aléatoire du serveur wapt 

Ok, la commande fonctionne. 
pids=$(pgrep -f "server.py"); for f in /proc/$pids/fdinfo/*; do cat $f | grep flags | grep -q -v 400[0,1,2] && ls -al /proc/$pids/fd/$(basename $f); done; 
lr-x------ 1 wapt www-data 64 oct. 22 12:28 /proc/1527/fd/0 -> /dev/null 
lrwx------ 1 wapt www-data 64 oct. 22 12:28 /proc/1527/fd/1 -> socket:[15786] 
lr-x------ 1 wapt www-data 64 oct. 22 12:28 /proc/1527/fd/15 -> /dev/urandom 
lrwx------ 1 wapt www-data 64 oct. 22 12:28 /proc/1527/fd/2 -> socket:[15786] 
lrwx------ 1 wapt www-data 64 oct. 22 12:28 /proc/1527/fd/3 -> anon_inode:[eventpoll] 
lr-x------ 1 wapt www-data 64 oct. 22 12:28 /proc/1527/fd/8 -> /dev/urandom 

Je n'ai plus qu'à attendre une panne... 

Frédéric GARESCHÉ 
Équipe Assistance et Exploitation 
Direction du Système d'Information (D.S.I) 
Correspondant Département EII et laboratoire IETR 
Tél. : +33 (0)2 23 2 3 82 81 
20 avenue des Buttes de Coësmes 
CS 70839 - 35 708 RENNES Cedex 7 

[ https://www.insa-rennes.fr/ ] 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.tranquil.it/pipermail/wapt/attachments/20191024/88e2d593/attachment.html>