2017-06-01 12:53:50,095 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 12:53:50,096 INFO repeater task os150bb_beamplots starting 24.3 seconds late 2017-06-01 12:53:51,050 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 12:53:51,050 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-01 12:53:51,050 INFO repeater task os150nb_beamplots starting 25.3 seconds late 2017-06-01 12:53:52,005 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 12:53:52,005 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-01 12:53:52,006 INFO repeater task os150nb_beamplots starting 26.2 seconds late 2017-06-01 12:53:52,965 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 12:53:52,965 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-01 12:53:52,965 INFO repeater task posmv_hcorrstats starting 27.2 seconds late 2017-06-01 12:53:53,812 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 12:53:53,813 INFO repeater task posmv_plotposmv starting 28.0 seconds late 2017-06-01 12:53:54,418 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 12:53:59,424 INFO repeater task os150bb_lastensq starting 23.4 seconds late 2017-06-01 12:54:01,117 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 12:54:01,117 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 12:53:26 2017-06-01 12:54:06,125 INFO repeater task os150bb_10pings starting 0.3 seconds late 2017-06-01 12:54:06,220 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 12:54:06,221 INFO repeater task os150nb_10pings starting 0.4 seconds late 2017-06-01 12:54:06,313 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 12:58:26,869 INFO repeater task posmv_hcorrstats starting 1.1 seconds late 2017-06-01 12:58:27,732 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 12:58:27,732 INFO repeater task posmv_plotposmv starting 2.0 seconds late 2017-06-01 12:58:28,340 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 12:58:38,355 INFO repeater task os150bb_lastensq starting 2.4 seconds late 2017-06-01 12:58:40,150 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 12:58:40,150 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 12:58:26 2017-06-01 12:58:40,151 INFO repeater task os150nb_lastensq starting 4.2 seconds late 2017-06-01 12:58:41,282 INFO repeater task os150nb_lastensq ended after 1.1 seconds 2017-06-01 12:58:41,282 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 12:58:26 2017-06-01 12:59:26,374 INFO repeater task os150bb_10pings starting 0.6 seconds late 2017-06-01 12:59:26,508 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 12:59:26,509 INFO repeater task os150nb_10pings starting 0.7 seconds late 2017-06-01 12:59:26,602 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 13:03:27,124 INFO repeater task posmv_hcorrstats starting 1.3 seconds late 2017-06-01 13:03:28,111 INFO repeater task posmv_hcorrstats ended after 1.0 seconds 2017-06-01 13:03:28,112 INFO repeater task posmv_plotposmv starting 2.3 seconds late 2017-06-01 13:03:28,731 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 13:03:38,751 INFO repeater task os150bb_lastensq starting 2.8 seconds late 2017-06-01 13:03:40,451 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 13:03:40,451 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 13:03:26 2017-06-01 13:03:40,452 INFO repeater task os150nb_lastensq starting 4.5 seconds late 2017-06-01 13:03:41,622 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 13:03:41,622 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 13:03:26 2017-06-01 13:04:46,798 INFO repeater task os150bb_10pings starting 1.0 seconds late 2017-06-01 13:04:46,959 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 13:04:46,960 INFO repeater task os150nb_10pings starting 1.2 seconds late 2017-06-01 13:04:47,052 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 13:08:27,383 INFO repeater task os150bb_quick starting 1.6 seconds late 2017-06-01 13:08:30,591 INFO repeater task os150bb_quick ended after 3.2 seconds 2017-06-01 13:08:30,591 INFO repeater task os150bb_3dayplots starting 4.8 seconds late 2017-06-01 13:08:36,970 INFO repeater task os150bb_3dayplots ended after 6.4 seconds 2017-06-01 13:08:36,970 INFO repeater task os150bb_tsstats starting 11.2 seconds late 2017-06-01 13:08:37,531 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-01 13:08:37,531 INFO repeater task os150nb_lastensq starting 1.5 seconds late 2017-06-01 13:08:38,710 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 13:08:38,710 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 13:08:26 2017-06-01 13:08:38,711 INFO repeater task os150nb_quick starting 12.9 seconds late 2017-06-01 13:08:41,757 INFO repeater task os150nb_quick ended after 3.0 seconds 2017-06-01 13:08:41,757 INFO repeater task os150nb_3dayplots starting 16.0 seconds late 2017-06-01 13:08:48,140 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 13:08:48,140 INFO repeater task os150nb_3dayplots ended after 6.4 seconds 2017-06-01 13:08:48,140 INFO repeater task os150nb_tsstats starting 22.4 seconds late 2017-06-01 13:08:48,685 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2017-06-01 13:08:48,685 INFO repeater task os150bb_beamplots starting 22.9 seconds late 2017-06-01 13:08:49,627 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 13:08:49,627 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 13:08:49,628 INFO repeater task os150bb_beamplots starting 23.8 seconds late 2017-06-01 13:08:50,554 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 13:08:50,554 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 13:08:50,554 INFO repeater task os150nb_beamplots starting 24.8 seconds late 2017-06-01 13:08:51,472 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 13:08:51,472 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 13:08:51,473 INFO repeater task os150nb_beamplots starting 25.7 seconds late 2017-06-01 13:08:52,420 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 13:08:52,420 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 13:08:52,421 INFO repeater task posmv_hcorrstats starting 26.6 seconds late 2017-06-01 13:08:53,279 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 13:08:53,279 INFO repeater task posmv_plotposmv starting 27.5 seconds late 2017-06-01 13:08:53,915 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 13:08:58,919 INFO repeater task os150bb_lastensq starting 22.9 seconds late 2017-06-01 13:09:00,569 INFO repeater task os150bb_lastensq ended after 1.6 seconds 2017-06-01 13:09:00,569 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 13:08:26 2017-06-01 13:10:10,710 INFO repeater task os150bb_10pings starting 4.9 seconds late 2017-06-01 13:10:10,862 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 13:10:10,863 INFO repeater task os150nb_10pings starting 5.1 seconds late 2017-06-01 13:10:10,955 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 13:13:26,339 INFO repeater task posmv_hcorrstats starting 0.6 seconds late 2017-06-01 13:13:27,226 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 13:13:27,226 INFO repeater task posmv_plotposmv starting 1.4 seconds late 2017-06-01 13:13:27,857 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 13:13:37,868 INFO repeater task os150bb_lastensq starting 1.9 seconds late 2017-06-01 13:13:39,592 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 13:13:39,592 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 13:13:26 2017-06-01 13:13:39,593 INFO repeater task os150nb_lastensq starting 3.6 seconds late 2017-06-01 13:13:40,745 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 13:13:40,745 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 13:13:26 2017-06-01 13:15:25,952 INFO repeater task os150bb_10pings starting 0.2 seconds late 2017-06-01 13:15:26,087 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 13:15:26,087 INFO repeater task os150nb_10pings starting 0.3 seconds late 2017-06-01 13:15:26,179 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 13:18:26,558 INFO repeater task posmv_hcorrstats starting 0.8 seconds late 2017-06-01 13:18:27,448 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 13:18:27,449 INFO repeater task posmv_plotposmv starting 1.7 seconds late 2017-06-01 13:18:28,048 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 13:18:38,061 INFO repeater task os150bb_lastensq starting 2.1 seconds late 2017-06-01 13:18:39,791 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 13:18:39,791 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 13:18:26 2017-06-01 13:18:39,792 INFO repeater task os150nb_lastensq starting 3.8 seconds late 2017-06-01 13:18:40,953 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 13:18:40,954 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 13:18:26 2017-06-01 13:20:46,149 INFO repeater task os150bb_10pings starting 0.4 seconds late 2017-06-01 13:20:46,244 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 13:20:46,244 INFO repeater task os150nb_10pings starting 0.5 seconds late 2017-06-01 13:20:46,339 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 13:23:26,598 INFO repeater task os150bb_quick starting 0.8 seconds late 2017-06-01 13:23:29,792 INFO repeater task os150bb_quick ended after 3.2 seconds 2017-06-01 13:23:29,792 INFO repeater task os150bb_3dayplots starting 4.0 seconds late 2017-06-01 13:23:36,492 INFO repeater task os150bb_3dayplots ended after 6.7 seconds 2017-06-01 13:23:36,492 INFO repeater task os150bb_tsstats starting 10.7 seconds late 2017-06-01 13:23:37,048 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-01 13:23:37,048 INFO repeater task os150nb_lastensq starting 1.0 seconds late 2017-06-01 13:23:38,292 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 13:23:38,292 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 13:23:26 2017-06-01 13:23:38,293 INFO repeater task os150nb_quick starting 12.5 seconds late 2017-06-01 13:23:41,425 INFO repeater task os150nb_quick ended after 3.1 seconds 2017-06-01 13:23:41,425 INFO repeater task os150nb_3dayplots starting 15.6 seconds late 2017-06-01 13:23:48,124 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 13:23:48,124 INFO repeater task os150nb_3dayplots ended after 6.7 seconds 2017-06-01 13:23:48,124 INFO repeater task os150nb_tsstats starting 22.3 seconds late 2017-06-01 13:23:48,689 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-01 13:23:48,689 INFO repeater task os150bb_beamplots starting 22.9 seconds late 2017-06-01 13:23:49,585 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 13:23:49,585 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 13:23:49,585 INFO repeater task os150bb_beamplots starting 23.8 seconds late 2017-06-01 13:23:50,523 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 13:23:50,523 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 13:23:50,523 INFO repeater task os150nb_beamplots starting 24.7 seconds late 2017-06-01 13:23:51,497 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 13:23:51,497 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-01 13:23:51,497 INFO repeater task os150nb_beamplots starting 25.7 seconds late 2017-06-01 13:23:52,502 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 13:23:52,503 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-01 13:23:52,503 INFO repeater task posmv_hcorrstats starting 26.7 seconds late 2017-06-01 13:23:53,339 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 13:23:53,340 INFO repeater task posmv_plotposmv starting 27.6 seconds late 2017-06-01 13:23:53,969 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 13:23:58,975 INFO repeater task os150bb_lastensq starting 23.0 seconds late 2017-06-01 13:24:00,786 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 13:24:00,787 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 13:23:26 2017-06-01 13:26:06,107 INFO repeater task os150bb_10pings starting 0.3 seconds late 2017-06-01 13:26:06,301 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 13:26:06,302 INFO repeater task os150nb_10pings starting 0.5 seconds late 2017-06-01 13:26:06,398 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 13:28:26,684 INFO repeater task posmv_hcorrstats starting 0.9 seconds late 2017-06-01 13:28:27,601 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 13:28:27,601 INFO repeater task posmv_plotposmv starting 1.8 seconds late 2017-06-01 13:28:28,266 INFO repeater task posmv_plotposmv ended after 0.7 seconds 2017-06-01 13:28:38,280 INFO repeater task os150bb_lastensq starting 2.3 seconds late 2017-06-01 13:28:40,073 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 13:28:40,073 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 13:28:26 2017-06-01 13:28:40,074 INFO repeater task os150nb_lastensq starting 4.1 seconds late 2017-06-01 13:28:41,327 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-01 13:28:41,327 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 13:28:26 2017-06-01 13:31:26,652 INFO repeater task os150bb_10pings starting 0.9 seconds late 2017-06-01 13:31:26,819 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 13:31:26,820 INFO repeater task os150nb_10pings starting 1.0 seconds late 2017-06-01 13:31:26,913 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 13:33:27,146 INFO repeater task posmv_hcorrstats starting 1.4 seconds late 2017-06-01 13:33:28,090 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 13:33:28,090 INFO repeater task posmv_plotposmv starting 2.3 seconds late 2017-06-01 13:33:28,693 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 13:33:38,710 INFO repeater task os150bb_lastensq starting 2.7 seconds late 2017-06-01 13:33:40,517 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 13:33:40,517 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 13:33:26 2017-06-01 13:33:40,518 INFO repeater task os150nb_lastensq starting 4.5 seconds late 2017-06-01 13:33:41,763 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 13:33:41,763 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 13:33:26 2017-06-01 13:36:47,249 INFO repeater task os150bb_10pings starting 1.5 seconds late 2017-06-01 13:36:47,408 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 13:36:47,409 INFO repeater task os150nb_10pings starting 1.6 seconds late 2017-06-01 13:36:47,504 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 13:38:27,701 INFO repeater task os150bb_quick starting 1.9 seconds late 2017-06-01 13:38:30,899 INFO repeater task os150bb_quick ended after 3.2 seconds 2017-06-01 13:38:30,900 INFO repeater task os150bb_3dayplots starting 5.1 seconds late 2017-06-01 13:38:37,168 INFO repeater task os150bb_3dayplots ended after 6.3 seconds 2017-06-01 13:38:37,168 INFO repeater task os150bb_tsstats starting 11.4 seconds late 2017-06-01 13:38:37,705 INFO repeater task os150bb_tsstats ended after 0.5 seconds 2017-06-01 13:38:37,705 INFO repeater task os150nb_lastensq starting 1.7 seconds late 2017-06-01 13:38:38,970 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-01 13:38:38,971 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 13:38:26 2017-06-01 13:38:38,971 INFO repeater task os150nb_quick starting 13.2 seconds late 2017-06-01 13:38:42,068 INFO repeater task os150nb_quick ended after 3.1 seconds 2017-06-01 13:38:42,068 INFO repeater task os150nb_3dayplots starting 16.3 seconds late 2017-06-01 13:38:48,979 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 13:38:48,980 INFO repeater task os150nb_3dayplots ended after 6.9 seconds 2017-06-01 13:38:48,980 INFO repeater task os150nb_tsstats starting 23.2 seconds late 2017-06-01 13:38:49,537 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-01 13:38:49,538 INFO repeater task os150bb_beamplots starting 23.8 seconds late 2017-06-01 13:38:50,492 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 13:38:50,493 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-01 13:38:50,493 INFO repeater task os150bb_beamplots starting 24.7 seconds late 2017-06-01 13:38:51,439 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 13:38:51,439 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 13:38:51,439 INFO repeater task os150nb_beamplots starting 25.7 seconds late 2017-06-01 13:38:52,392 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 13:38:52,393 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-01 13:38:52,393 INFO repeater task os150nb_beamplots starting 26.6 seconds late 2017-06-01 13:38:53,375 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 13:38:53,376 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-01 13:38:53,376 INFO repeater task posmv_hcorrstats starting 27.6 seconds late 2017-06-01 13:38:54,265 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 13:38:54,265 INFO repeater task posmv_plotposmv starting 28.5 seconds late 2017-06-01 13:38:54,908 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 13:38:59,919 INFO repeater task os150bb_lastensq starting 23.9 seconds late 2017-06-01 13:39:01,698 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 13:39:01,699 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 13:38:26 2017-06-01 13:42:07,023 INFO repeater task os150bb_10pings starting 1.2 seconds late 2017-06-01 13:42:07,122 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 13:42:07,123 INFO repeater task os150nb_10pings starting 1.3 seconds late 2017-06-01 13:42:07,224 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 13:43:27,368 INFO repeater task posmv_hcorrstats starting 1.6 seconds late 2017-06-01 13:43:28,232 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 13:43:28,232 INFO repeater task posmv_plotposmv starting 2.5 seconds late 2017-06-01 13:43:28,866 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 13:43:38,947 INFO repeater task os150bb_lastensq starting 2.9 seconds late 2017-06-01 13:43:40,720 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 13:43:40,721 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 13:43:26 2017-06-01 13:43:40,721 INFO repeater task os150nb_lastensq starting 4.7 seconds late 2017-06-01 13:43:42,012 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-01 13:43:42,012 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 13:43:26 2017-06-01 13:47:27,480 INFO repeater task os150bb_10pings starting 1.7 seconds late 2017-06-01 13:47:27,630 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 13:47:27,631 INFO repeater task os150nb_10pings starting 1.9 seconds late 2017-06-01 13:47:27,722 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 13:48:27,860 INFO repeater task posmv_hcorrstats starting 2.1 seconds late 2017-06-01 13:48:28,780 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 13:48:28,780 INFO repeater task posmv_plotposmv starting 3.0 seconds late 2017-06-01 13:48:29,433 INFO repeater task posmv_plotposmv ended after 0.7 seconds 2017-06-01 13:48:39,446 INFO repeater task os150bb_lastensq starting 3.4 seconds late 2017-06-01 13:48:41,240 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 13:48:41,240 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 13:48:26 2017-06-01 13:48:41,241 INFO repeater task os150nb_lastensq starting 5.2 seconds late 2017-06-01 13:48:42,416 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 13:48:42,416 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 13:48:26 2017-06-01 13:52:47,937 INFO repeater task os150bb_10pings starting 2.2 seconds late 2017-06-01 13:52:48,107 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 13:52:48,108 INFO repeater task os150nb_10pings starting 2.3 seconds late 2017-06-01 13:52:48,203 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 13:53:28,287 INFO repeater task os150bb_quick starting 2.5 seconds late 2017-06-01 13:53:31,502 INFO repeater task os150bb_quick ended after 3.2 seconds 2017-06-01 13:53:31,503 INFO repeater task os150bb_3dayplots starting 5.7 seconds late 2017-06-01 13:53:38,245 INFO repeater task os150bb_3dayplots ended after 6.7 seconds 2017-06-01 13:53:38,245 INFO repeater task os150bb_tsstats starting 12.5 seconds late 2017-06-01 13:53:38,802 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-01 13:53:38,803 INFO repeater task os150nb_lastensq starting 2.8 seconds late 2017-06-01 13:53:40,161 INFO repeater task os150nb_lastensq ended after 1.4 seconds 2017-06-01 13:53:40,161 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 13:53:26 2017-06-01 13:53:40,161 INFO repeater task os150nb_quick starting 14.4 seconds late 2017-06-01 13:53:43,340 INFO repeater task os150nb_quick ended after 3.2 seconds 2017-06-01 13:53:43,354 INFO repeater task os150nb_3dayplots starting 17.6 seconds late 2017-06-01 13:53:50,220 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 13:53:50,220 INFO repeater task os150nb_3dayplots ended after 6.9 seconds 2017-06-01 13:53:50,220 INFO repeater task os150nb_tsstats starting 24.4 seconds late 2017-06-01 13:53:50,785 INFO repeater task os150nb_tsstats output: /usr/lib/python2.7/dist-packages/numpy/ma/core.py:1073: RuntimeWarning: overflow encountered in true_divide result = self.f(da, db, *args, **kwargs) 2017-06-01 13:53:50,785 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-01 13:53:50,785 INFO repeater task os150bb_beamplots starting 25.0 seconds late 2017-06-01 13:53:51,778 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 13:53:51,778 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-01 13:53:51,778 INFO repeater task os150bb_beamplots starting 26.0 seconds late 2017-06-01 13:53:52,721 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 13:53:52,721 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 13:53:52,721 INFO repeater task os150nb_beamplots starting 26.9 seconds late 2017-06-01 13:53:53,681 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 13:53:53,681 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-01 13:53:53,681 INFO repeater task os150nb_beamplots starting 27.9 seconds late 2017-06-01 13:53:54,651 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 13:53:54,651 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-01 13:53:54,652 INFO repeater task posmv_hcorrstats starting 28.9 seconds late 2017-06-01 13:53:55,533 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 13:53:55,533 INFO repeater task posmv_plotposmv starting 29.8 seconds late 2017-06-01 13:53:56,147 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 13:54:01,153 INFO repeater task os150bb_lastensq starting 25.2 seconds late 2017-06-01 13:54:02,990 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 13:54:02,990 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 13:53:26 2017-06-01 13:58:08,462 INFO repeater task os150bb_10pings starting 2.7 seconds late 2017-06-01 13:58:08,638 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 13:58:08,639 INFO repeater task os150nb_10pings starting 2.9 seconds late 2017-06-01 13:58:08,733 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 13:58:28,771 INFO repeater task posmv_hcorrstats starting 3.0 seconds late 2017-06-01 13:58:29,737 INFO repeater task posmv_hcorrstats ended after 1.0 seconds 2017-06-01 13:58:29,737 INFO repeater task posmv_plotposmv starting 4.0 seconds late 2017-06-01 13:58:30,360 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 13:58:40,378 INFO repeater task os150bb_lastensq starting 4.4 seconds late 2017-06-01 13:58:42,110 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 13:58:42,110 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 13:58:26 2017-06-01 13:58:42,111 INFO repeater task os150nb_lastensq starting 6.1 seconds late 2017-06-01 13:58:43,278 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 13:58:43,278 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 13:58:26 2017-06-01 14:03:28,814 INFO repeater task os150bb_10pings starting 3.0 seconds late 2017-06-01 14:03:28,917 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 14:03:28,918 INFO repeater task os150nb_10pings starting 3.1 seconds late 2017-06-01 14:03:29,014 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 14:03:29,015 INFO repeater task posmv_hcorrstats starting 3.2 seconds late 2017-06-01 14:03:29,863 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 14:03:29,863 INFO repeater task posmv_plotposmv starting 4.1 seconds late 2017-06-01 14:03:30,472 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 14:03:40,490 INFO repeater task os150bb_lastensq starting 4.5 seconds late 2017-06-01 14:03:42,222 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 14:03:42,223 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 14:03:26 2017-06-01 14:03:42,223 INFO repeater task os150nb_lastensq starting 6.2 seconds late 2017-06-01 14:03:43,499 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-01 14:03:43,499 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 14:03:26 2017-06-01 14:08:29,077 INFO repeater task os150bb_quick starting 3.3 seconds late 2017-06-01 14:08:32,277 INFO repeater task os150bb_quick ended after 3.2 seconds 2017-06-01 14:08:32,278 INFO repeater task os150bb_3dayplots starting 6.5 seconds late 2017-06-01 14:08:38,653 INFO repeater task os150bb_3dayplots ended after 6.4 seconds 2017-06-01 14:08:38,653 INFO repeater task os150bb_tsstats starting 12.9 seconds late 2017-06-01 14:08:39,226 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-01 14:08:39,226 INFO repeater task os150nb_lastensq starting 3.2 seconds late 2017-06-01 14:08:40,418 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 14:08:40,418 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 14:08:26 2017-06-01 14:08:40,418 INFO repeater task os150nb_quick starting 14.6 seconds late 2017-06-01 14:08:43,454 INFO repeater task os150nb_quick ended after 3.0 seconds 2017-06-01 14:08:43,455 INFO repeater task os150nb_3dayplots starting 17.7 seconds late 2017-06-01 14:08:49,897 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 14:08:49,897 INFO repeater task os150nb_3dayplots ended after 6.4 seconds 2017-06-01 14:08:49,898 INFO repeater task os150nb_tsstats starting 24.1 seconds late 2017-06-01 14:08:50,457 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-01 14:08:50,457 INFO repeater task os150bb_beamplots starting 24.7 seconds late 2017-06-01 14:08:51,397 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 14:08:51,397 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 14:08:51,397 INFO repeater task os150bb_beamplots starting 25.6 seconds late 2017-06-01 14:08:52,333 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 14:08:52,334 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 14:08:52,334 INFO repeater task os150nb_beamplots starting 26.6 seconds late 2017-06-01 14:08:53,249 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 14:08:53,249 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 14:08:53,249 INFO repeater task os150nb_beamplots starting 27.5 seconds late 2017-06-01 14:08:54,205 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 14:08:54,205 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-01 14:08:54,205 INFO repeater task posmv_hcorrstats starting 28.4 seconds late 2017-06-01 14:08:55,081 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 14:08:55,082 INFO repeater task posmv_plotposmv starting 29.3 seconds late 2017-06-01 14:08:55,695 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 14:09:00,700 INFO repeater task os150bb_lastensq starting 24.7 seconds late 2017-06-01 14:09:02,417 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 14:09:02,417 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 14:08:26 2017-06-01 14:09:02,417 INFO repeater task os150bb_10pings starting 16.6 seconds late 2017-06-01 14:09:02,510 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 14:09:02,511 INFO repeater task os150nb_10pings starting 16.7 seconds late 2017-06-01 14:09:02,604 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 14:13:28,153 INFO repeater task posmv_hcorrstats starting 2.4 seconds late 2017-06-01 14:13:29,083 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 14:13:29,084 INFO repeater task posmv_plotposmv starting 3.3 seconds late 2017-06-01 14:13:29,700 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 14:13:39,717 INFO repeater task os150bb_lastensq starting 3.7 seconds late 2017-06-01 14:13:41,416 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 14:13:41,417 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 14:13:26 2017-06-01 14:13:41,417 INFO repeater task os150nb_lastensq starting 5.4 seconds late 2017-06-01 14:13:42,615 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 14:13:42,615 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 14:13:26 2017-06-01 14:14:07,661 INFO repeater task os150bb_10pings starting 1.9 seconds late 2017-06-01 14:14:07,818 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 14:14:07,818 INFO repeater task os150nb_10pings starting 2.0 seconds late 2017-06-01 14:14:07,911 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 14:18:28,488 INFO repeater task posmv_hcorrstats starting 2.7 seconds late 2017-06-01 14:18:29,301 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 14:18:29,301 INFO repeater task posmv_plotposmv starting 3.5 seconds late 2017-06-01 14:18:29,933 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 14:18:39,951 INFO repeater task os150bb_lastensq starting 4.0 seconds late 2017-06-01 14:18:41,527 INFO repeater task os150bb_lastensq ended after 1.6 seconds 2017-06-01 14:18:41,528 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 14:18:26 2017-06-01 14:18:41,528 INFO repeater task os150nb_lastensq starting 5.5 seconds late 2017-06-01 14:18:42,640 INFO repeater task os150nb_lastensq ended after 1.1 seconds 2017-06-01 14:18:42,640 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 14:18:26 2017-06-01 14:19:27,738 INFO repeater task os150bb_10pings starting 2.0 seconds late 2017-06-01 14:19:27,890 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 14:19:27,891 INFO repeater task os150nb_10pings starting 2.1 seconds late 2017-06-01 14:19:27,985 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 14:23:28,495 INFO repeater task os150bb_quick starting 2.7 seconds late 2017-06-01 14:23:31,725 INFO repeater task os150bb_quick ended after 3.2 seconds 2017-06-01 14:23:31,725 INFO repeater task os150bb_3dayplots starting 5.9 seconds late 2017-06-01 14:23:37,977 INFO repeater task os150bb_3dayplots ended after 6.3 seconds 2017-06-01 14:23:37,978 INFO repeater task os150bb_tsstats starting 12.2 seconds late 2017-06-01 14:23:38,538 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-01 14:23:38,538 INFO repeater task os150nb_lastensq starting 2.5 seconds late 2017-06-01 14:23:39,668 INFO repeater task os150nb_lastensq ended after 1.1 seconds 2017-06-01 14:23:39,669 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 14:23:26 2017-06-01 14:23:39,669 INFO repeater task os150nb_quick starting 13.9 seconds late 2017-06-01 14:23:42,801 INFO repeater task os150nb_quick ended after 3.1 seconds 2017-06-01 14:23:42,801 INFO repeater task os150nb_3dayplots starting 17.0 seconds late 2017-06-01 14:23:49,095 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 14:23:49,095 INFO repeater task os150nb_3dayplots ended after 6.3 seconds 2017-06-01 14:23:49,095 INFO repeater task os150nb_tsstats starting 23.3 seconds late 2017-06-01 14:23:49,654 INFO repeater task os150nb_tsstats output: /usr/lib/python2.7/dist-packages/numpy/ma/core.py:1073: RuntimeWarning: overflow encountered in true_divide result = self.f(da, db, *args, **kwargs) 2017-06-01 14:23:49,654 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-01 14:23:49,654 INFO repeater task os150bb_beamplots starting 23.9 seconds late 2017-06-01 14:23:50,557 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 14:23:50,557 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 14:23:50,558 INFO repeater task os150bb_beamplots starting 24.8 seconds late 2017-06-01 14:23:51,494 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 14:23:51,495 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 14:23:51,495 INFO repeater task os150nb_beamplots starting 25.7 seconds late 2017-06-01 14:23:52,402 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 14:23:52,403 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 14:23:52,403 INFO repeater task os150nb_beamplots starting 26.6 seconds late 2017-06-01 14:23:53,344 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 14:23:53,344 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 14:23:53,345 INFO repeater task posmv_hcorrstats starting 27.6 seconds late 2017-06-01 14:23:54,172 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 14:23:54,172 INFO repeater task posmv_plotposmv starting 28.4 seconds late 2017-06-01 14:23:54,767 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 14:23:59,773 INFO repeater task os150bb_lastensq starting 23.8 seconds late 2017-06-01 14:24:01,421 INFO repeater task os150bb_lastensq ended after 1.6 seconds 2017-06-01 14:24:01,421 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 14:23:26 2017-06-01 14:24:46,516 INFO repeater task os150bb_10pings starting 0.7 seconds late 2017-06-01 14:24:46,683 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 14:24:46,684 INFO repeater task os150nb_10pings starting 0.9 seconds late 2017-06-01 14:24:46,783 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 14:28:27,274 INFO repeater task posmv_hcorrstats starting 1.5 seconds late 2017-06-01 14:28:28,199 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 14:28:28,200 INFO repeater task posmv_plotposmv starting 2.4 seconds late 2017-06-01 14:28:28,825 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 14:28:38,843 INFO repeater task os150bb_lastensq starting 2.8 seconds late 2017-06-01 14:28:40,802 INFO repeater task os150bb_lastensq ended after 2.0 seconds 2017-06-01 14:28:40,802 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 14:28:26 2017-06-01 14:28:40,803 INFO repeater task os150nb_lastensq starting 4.8 seconds late 2017-06-01 14:28:42,023 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 14:28:42,023 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 14:28:26 2017-06-01 14:30:07,177 INFO repeater task os150bb_10pings starting 1.4 seconds late 2017-06-01 14:30:07,319 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 14:30:07,320 INFO repeater task os150nb_10pings starting 1.5 seconds late 2017-06-01 14:30:07,415 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 14:33:27,774 INFO repeater task posmv_hcorrstats starting 2.0 seconds late 2017-06-01 14:33:28,650 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 14:33:28,650 INFO repeater task posmv_plotposmv starting 2.9 seconds late 2017-06-01 14:33:29,309 INFO repeater task posmv_plotposmv ended after 0.7 seconds 2017-06-01 14:33:39,326 INFO repeater task os150bb_lastensq starting 3.3 seconds late 2017-06-01 14:33:41,009 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 14:33:41,010 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 14:33:26 2017-06-01 14:33:41,010 INFO repeater task os150nb_lastensq starting 5.0 seconds late 2017-06-01 14:33:42,224 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 14:33:42,224 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 14:33:26 2017-06-01 14:35:27,448 INFO repeater task os150bb_10pings starting 1.7 seconds late 2017-06-01 14:35:27,609 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 14:35:27,611 INFO repeater task os150nb_10pings starting 1.8 seconds late 2017-06-01 14:35:27,704 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 14:38:28,143 INFO repeater task os150bb_quick starting 2.4 seconds late 2017-06-01 14:38:31,268 INFO repeater task os150bb_quick ended after 3.1 seconds 2017-06-01 14:38:31,268 INFO repeater task os150bb_3dayplots starting 5.5 seconds late 2017-06-01 14:38:38,235 INFO repeater task os150bb_3dayplots ended after 7.0 seconds 2017-06-01 14:38:38,235 INFO repeater task os150bb_tsstats starting 12.5 seconds late 2017-06-01 14:38:38,848 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-01 14:38:38,848 INFO repeater task os150nb_lastensq starting 2.8 seconds late 2017-06-01 14:38:40,118 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-01 14:38:40,118 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 14:38:26 2017-06-01 14:38:40,118 INFO repeater task os150nb_quick starting 14.3 seconds late 2017-06-01 14:38:43,304 INFO repeater task os150nb_quick ended after 3.2 seconds 2017-06-01 14:38:43,304 INFO repeater task os150nb_3dayplots starting 17.5 seconds late 2017-06-01 14:38:50,147 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 14:38:50,147 INFO repeater task os150nb_3dayplots ended after 6.8 seconds 2017-06-01 14:38:50,148 INFO repeater task os150nb_tsstats starting 24.4 seconds late 2017-06-01 14:38:50,763 INFO repeater task os150nb_tsstats output: /usr/lib/python2.7/dist-packages/numpy/ma/core.py:1073: RuntimeWarning: overflow encountered in true_divide result = self.f(da, db, *args, **kwargs) 2017-06-01 14:38:50,763 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-01 14:38:50,763 INFO repeater task os150bb_beamplots starting 25.0 seconds late 2017-06-01 14:38:51,731 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 14:38:51,731 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-01 14:38:51,732 INFO repeater task os150bb_beamplots starting 26.0 seconds late 2017-06-01 14:38:52,678 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 14:38:52,678 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 14:38:52,678 INFO repeater task os150nb_beamplots starting 26.9 seconds late 2017-06-01 14:38:53,621 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 14:38:53,621 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 14:38:53,621 INFO repeater task os150nb_beamplots starting 27.8 seconds late 2017-06-01 14:38:54,630 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 14:38:54,630 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-01 14:38:54,630 INFO repeater task posmv_hcorrstats starting 28.8 seconds late 2017-06-01 14:38:55,452 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 14:38:55,452 INFO repeater task posmv_plotposmv starting 29.7 seconds late 2017-06-01 14:38:56,102 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 14:39:01,108 INFO repeater task os150bb_lastensq starting 25.1 seconds late 2017-06-01 14:39:02,848 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 14:39:02,848 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 14:38:26 2017-06-01 14:40:48,143 INFO repeater task os150bb_10pings starting 2.4 seconds late 2017-06-01 14:40:48,322 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 14:40:48,323 INFO repeater task os150nb_10pings starting 2.5 seconds late 2017-06-01 14:40:48,419 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 14:43:28,747 INFO repeater task posmv_hcorrstats starting 3.0 seconds late 2017-06-01 14:43:29,609 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 14:43:29,609 INFO repeater task posmv_plotposmv starting 3.8 seconds late 2017-06-01 14:43:30,270 INFO repeater task posmv_plotposmv ended after 0.7 seconds 2017-06-01 14:43:40,286 INFO repeater task os150bb_lastensq starting 4.3 seconds late 2017-06-01 14:43:41,991 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 14:43:41,991 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 14:43:26 2017-06-01 14:43:41,992 INFO repeater task os150nb_lastensq starting 6.0 seconds late 2017-06-01 14:43:43,169 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 14:43:43,169 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 14:43:26 2017-06-01 14:46:08,466 INFO repeater task os150bb_10pings starting 2.7 seconds late 2017-06-01 14:46:08,632 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 14:46:08,633 INFO repeater task os150nb_10pings starting 2.9 seconds late 2017-06-01 14:46:08,729 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 14:48:29,043 INFO repeater task posmv_hcorrstats starting 3.3 seconds late 2017-06-01 14:48:29,975 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 14:48:29,975 INFO repeater task posmv_plotposmv starting 4.2 seconds late 2017-06-01 14:48:30,578 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 14:48:40,594 INFO repeater task os150bb_lastensq starting 4.6 seconds late 2017-06-01 14:48:42,282 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 14:48:42,282 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 14:48:26 2017-06-01 14:48:42,283 INFO repeater task os150nb_lastensq starting 6.3 seconds late 2017-06-01 14:48:43,442 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 14:48:43,442 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 14:48:26 2017-06-01 14:51:28,792 INFO repeater task os150bb_10pings starting 3.0 seconds late 2017-06-01 14:51:28,933 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 14:51:28,934 INFO repeater task os150nb_10pings starting 3.2 seconds late 2017-06-01 14:51:29,025 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 14:53:29,238 INFO repeater task os150bb_quick starting 3.5 seconds late 2017-06-01 14:53:32,468 INFO repeater task os150bb_quick ended after 3.2 seconds 2017-06-01 14:53:32,468 INFO repeater task os150bb_3dayplots starting 6.7 seconds late 2017-06-01 14:53:38,827 INFO repeater task os150bb_3dayplots ended after 6.4 seconds 2017-06-01 14:53:38,828 INFO repeater task os150bb_tsstats starting 13.0 seconds late 2017-06-01 14:53:39,403 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-01 14:53:39,403 INFO repeater task os150nb_lastensq starting 3.4 seconds late 2017-06-01 14:53:40,551 INFO repeater task os150nb_lastensq ended after 1.1 seconds 2017-06-01 14:53:40,551 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 14:53:26 2017-06-01 14:53:40,552 INFO repeater task os150nb_quick starting 14.8 seconds late 2017-06-01 14:53:43,681 INFO repeater task os150nb_quick ended after 3.1 seconds 2017-06-01 14:53:43,681 INFO repeater task os150nb_3dayplots starting 17.9 seconds late 2017-06-01 14:53:50,046 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 14:53:50,046 INFO repeater task os150nb_3dayplots ended after 6.4 seconds 2017-06-01 14:53:50,046 INFO repeater task os150nb_tsstats starting 24.3 seconds late 2017-06-01 14:53:50,615 INFO repeater task os150nb_tsstats output: /usr/lib/python2.7/dist-packages/numpy/ma/core.py:1073: RuntimeWarning: overflow encountered in true_divide result = self.f(da, db, *args, **kwargs) 2017-06-01 14:53:50,615 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-01 14:53:50,616 INFO repeater task os150bb_beamplots starting 24.8 seconds late 2017-06-01 14:53:51,537 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 14:53:51,537 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 14:53:51,537 INFO repeater task os150bb_beamplots starting 25.8 seconds late 2017-06-01 14:53:52,495 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 14:53:52,495 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-01 14:53:52,496 INFO repeater task os150nb_beamplots starting 26.7 seconds late 2017-06-01 14:53:53,416 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 14:53:53,416 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 14:53:53,416 INFO repeater task os150nb_beamplots starting 27.6 seconds late 2017-06-01 14:53:54,348 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 14:53:54,348 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 14:53:54,348 INFO repeater task posmv_hcorrstats starting 28.6 seconds late 2017-06-01 14:53:55,184 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 14:53:55,184 INFO repeater task posmv_plotposmv starting 29.4 seconds late 2017-06-01 14:53:55,802 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 14:54:00,808 INFO repeater task os150bb_lastensq starting 24.8 seconds late 2017-06-01 14:54:02,526 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 14:54:02,527 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 14:53:26 2017-06-01 14:56:47,907 INFO repeater task os150bb_10pings starting 2.1 seconds late 2017-06-01 14:56:48,001 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 14:56:48,001 INFO repeater task os150nb_10pings starting 2.2 seconds late 2017-06-01 14:56:48,095 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 14:58:28,317 INFO repeater task posmv_hcorrstats starting 2.5 seconds late 2017-06-01 14:58:29,231 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 14:58:29,231 INFO repeater task posmv_plotposmv starting 3.4 seconds late 2017-06-01 14:58:29,860 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 14:58:39,877 INFO repeater task os150bb_lastensq starting 3.9 seconds late 2017-06-01 14:58:41,545 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 14:58:41,545 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 14:58:26 2017-06-01 14:58:41,546 INFO repeater task os150nb_lastensq starting 5.5 seconds late 2017-06-01 14:58:42,696 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 14:58:42,696 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 14:58:26 2017-06-01 15:02:08,123 INFO repeater task os150bb_10pings starting 2.3 seconds late 2017-06-01 15:02:08,239 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 15:02:08,240 INFO repeater task os150nb_10pings starting 2.5 seconds late 2017-06-01 15:02:08,335 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 15:03:28,509 INFO repeater task posmv_hcorrstats starting 2.7 seconds late 2017-06-01 15:03:29,394 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 15:03:29,394 INFO repeater task posmv_plotposmv starting 3.6 seconds late 2017-06-01 15:03:30,035 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 15:03:40,047 INFO repeater task os150bb_lastensq starting 4.0 seconds late 2017-06-01 15:03:41,811 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 15:03:41,812 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 15:03:26 2017-06-01 15:03:41,812 INFO repeater task os150nb_lastensq starting 5.8 seconds late 2017-06-01 15:03:43,045 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 15:03:43,045 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 15:03:26 2017-06-01 15:07:28,449 INFO repeater task os150bb_10pings starting 2.7 seconds late 2017-06-01 15:07:28,539 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 15:07:28,540 INFO repeater task os150nb_10pings starting 2.8 seconds late 2017-06-01 15:07:28,631 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 15:08:28,723 INFO repeater task os150bb_quick starting 2.9 seconds late 2017-06-01 15:08:31,830 INFO repeater task os150bb_quick ended after 3.1 seconds 2017-06-01 15:08:31,830 INFO repeater task os150bb_3dayplots starting 6.0 seconds late 2017-06-01 15:08:38,071 INFO repeater task os150bb_3dayplots ended after 6.2 seconds 2017-06-01 15:08:38,071 INFO repeater task os150bb_tsstats starting 12.3 seconds late 2017-06-01 15:08:38,644 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-01 15:08:38,644 INFO repeater task os150nb_lastensq starting 2.6 seconds late 2017-06-01 15:08:39,772 INFO repeater task os150nb_lastensq ended after 1.1 seconds 2017-06-01 15:08:39,772 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 15:08:26 2017-06-01 15:08:39,773 INFO repeater task os150nb_quick starting 14.0 seconds late 2017-06-01 15:08:42,859 INFO repeater task os150nb_quick ended after 3.1 seconds 2017-06-01 15:08:42,859 INFO repeater task os150nb_3dayplots starting 17.1 seconds late 2017-06-01 15:08:49,213 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 15:08:49,213 INFO repeater task os150nb_3dayplots ended after 6.4 seconds 2017-06-01 15:08:49,213 INFO repeater task os150nb_tsstats starting 23.4 seconds late 2017-06-01 15:08:49,787 INFO repeater task os150nb_tsstats output: /usr/lib/python2.7/dist-packages/numpy/ma/core.py:1073: RuntimeWarning: overflow encountered in true_divide result = self.f(da, db, *args, **kwargs) 2017-06-01 15:08:49,788 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-01 15:08:49,788 INFO repeater task os150bb_beamplots starting 24.0 seconds late 2017-06-01 15:08:50,689 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 15:08:50,689 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 15:08:50,690 INFO repeater task os150bb_beamplots starting 24.9 seconds late 2017-06-01 15:08:51,626 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 15:08:51,626 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 15:08:51,626 INFO repeater task os150nb_beamplots starting 25.8 seconds late 2017-06-01 15:08:52,548 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 15:08:52,549 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 15:08:52,549 INFO repeater task os150nb_beamplots starting 26.8 seconds late 2017-06-01 15:08:53,481 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 15:08:53,481 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 15:08:53,482 INFO repeater task posmv_hcorrstats starting 27.7 seconds late 2017-06-01 15:08:54,327 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 15:08:54,328 INFO repeater task posmv_plotposmv starting 28.5 seconds late 2017-06-01 15:08:54,945 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 15:08:59,956 INFO repeater task os150bb_lastensq starting 24.0 seconds late 2017-06-01 15:09:01,631 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 15:09:01,631 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 15:08:26 2017-06-01 15:12:47,202 INFO repeater task os150bb_10pings starting 1.4 seconds late 2017-06-01 15:12:47,291 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 15:12:47,292 INFO repeater task os150nb_10pings starting 1.5 seconds late 2017-06-01 15:12:47,384 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 15:13:27,476 INFO repeater task posmv_hcorrstats starting 1.7 seconds late 2017-06-01 15:13:28,405 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 15:13:28,405 INFO repeater task posmv_plotposmv starting 2.6 seconds late 2017-06-01 15:13:29,018 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 15:13:39,067 INFO repeater task os150bb_lastensq starting 3.1 seconds late 2017-06-01 15:13:40,725 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 15:13:40,725 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 15:13:26 2017-06-01 15:13:40,726 INFO repeater task os150nb_lastensq starting 4.7 seconds late 2017-06-01 15:13:41,902 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 15:13:41,903 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 15:13:26 2017-06-01 15:18:07,478 INFO repeater task os150bb_10pings starting 1.7 seconds late 2017-06-01 15:18:07,632 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 15:18:07,632 INFO repeater task os150nb_10pings starting 1.9 seconds late 2017-06-01 15:18:07,727 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 15:18:27,765 INFO repeater task posmv_hcorrstats starting 2.0 seconds late 2017-06-01 15:18:28,707 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 15:18:28,708 INFO repeater task posmv_plotposmv starting 2.9 seconds late 2017-06-01 15:18:29,360 INFO repeater task posmv_plotposmv ended after 0.7 seconds 2017-06-01 15:18:39,378 INFO repeater task os150bb_lastensq starting 3.4 seconds late 2017-06-01 15:18:41,179 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 15:18:41,179 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 15:18:26 2017-06-01 15:18:41,180 INFO repeater task os150nb_lastensq starting 5.2 seconds late 2017-06-01 15:18:42,411 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 15:18:42,411 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 15:18:26 2017-06-01 15:23:27,900 INFO repeater task os150bb_10pings starting 2.1 seconds late 2017-06-01 15:23:28,002 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 15:23:28,003 INFO repeater task os150bb_quick starting 2.2 seconds late 2017-06-01 15:23:31,066 INFO repeater task os150bb_quick ended after 3.1 seconds 2017-06-01 15:23:31,066 INFO repeater task os150bb_3dayplots starting 5.3 seconds late 2017-06-01 15:23:37,329 INFO repeater task os150bb_3dayplots ended after 6.3 seconds 2017-06-01 15:23:37,330 INFO repeater task os150bb_tsstats starting 11.5 seconds late 2017-06-01 15:23:37,872 INFO repeater task os150bb_tsstats ended after 0.5 seconds 2017-06-01 15:23:37,872 INFO repeater task os150nb_lastensq starting 1.9 seconds late 2017-06-01 15:23:39,035 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 15:23:39,035 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 15:23:26 2017-06-01 15:23:39,035 INFO repeater task os150nb_10pings starting 13.3 seconds late 2017-06-01 15:23:39,125 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 15:23:39,125 INFO repeater task os150nb_quick starting 13.3 seconds late 2017-06-01 15:23:42,167 INFO repeater task os150nb_quick ended after 3.0 seconds 2017-06-01 15:23:42,167 INFO repeater task os150nb_3dayplots starting 16.4 seconds late 2017-06-01 15:23:48,478 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 15:23:48,479 INFO repeater task os150nb_3dayplots ended after 6.3 seconds 2017-06-01 15:23:48,479 INFO repeater task os150nb_tsstats starting 22.7 seconds late 2017-06-01 15:23:49,056 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-01 15:23:49,056 INFO repeater task os150bb_beamplots starting 23.3 seconds late 2017-06-01 15:23:49,962 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 15:23:49,962 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 15:23:49,963 INFO repeater task os150bb_beamplots starting 24.2 seconds late 2017-06-01 15:23:50,874 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 15:23:50,874 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 15:23:50,874 INFO repeater task os150nb_beamplots starting 25.1 seconds late 2017-06-01 15:23:51,777 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 15:23:51,777 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 15:23:51,777 INFO repeater task os150nb_beamplots starting 26.0 seconds late 2017-06-01 15:23:52,705 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 15:23:52,705 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 15:23:52,706 INFO repeater task posmv_hcorrstats starting 26.9 seconds late 2017-06-01 15:23:53,537 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 15:23:53,538 INFO repeater task posmv_plotposmv starting 27.8 seconds late 2017-06-01 15:23:54,170 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 15:23:59,173 INFO repeater task os150bb_lastensq starting 23.2 seconds late 2017-06-01 15:24:00,860 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 15:24:00,860 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 15:23:26 2017-06-01 15:28:26,470 INFO repeater task posmv_hcorrstats starting 0.7 seconds late 2017-06-01 15:28:27,348 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 15:28:27,349 INFO repeater task posmv_plotposmv starting 1.6 seconds late 2017-06-01 15:28:27,972 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 15:28:37,985 INFO repeater task os150bb_lastensq starting 2.0 seconds late 2017-06-01 15:28:39,824 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 15:28:39,824 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 15:28:26 2017-06-01 15:28:39,824 INFO repeater task os150nb_lastensq starting 3.8 seconds late 2017-06-01 15:28:41,084 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-01 15:28:41,085 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 15:28:26 2017-06-01 15:28:46,092 INFO repeater task os150bb_10pings starting 0.3 seconds late 2017-06-01 15:28:46,259 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 15:28:46,260 INFO repeater task os150nb_10pings starting 0.5 seconds late 2017-06-01 15:28:46,357 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 15:33:26,895 INFO repeater task posmv_hcorrstats starting 1.1 seconds late 2017-06-01 15:33:27,827 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 15:33:27,827 INFO repeater task posmv_plotposmv starting 2.0 seconds late 2017-06-01 15:33:28,437 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 15:33:38,448 INFO repeater task os150bb_lastensq starting 2.4 seconds late 2017-06-01 15:33:40,073 INFO repeater task os150bb_lastensq ended after 1.6 seconds 2017-06-01 15:33:40,073 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 15:33:26 2017-06-01 15:33:40,074 INFO repeater task os150nb_lastensq starting 4.1 seconds late 2017-06-01 15:33:41,234 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 15:33:41,234 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 15:33:26 2017-06-01 15:34:06,275 INFO repeater task os150bb_10pings starting 0.5 seconds late 2017-06-01 15:34:06,446 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 15:34:06,447 INFO repeater task os150nb_10pings starting 0.7 seconds late 2017-06-01 15:34:06,540 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 15:38:26,977 INFO repeater task os150bb_quick starting 1.2 seconds late 2017-06-01 15:38:30,092 INFO repeater task os150bb_quick ended after 3.1 seconds 2017-06-01 15:38:30,092 INFO repeater task os150bb_3dayplots starting 4.3 seconds late 2017-06-01 15:38:36,360 INFO repeater task os150bb_3dayplots ended after 6.3 seconds 2017-06-01 15:38:36,360 INFO repeater task os150bb_tsstats starting 10.6 seconds late 2017-06-01 15:38:36,899 INFO repeater task os150bb_tsstats ended after 0.5 seconds 2017-06-01 15:38:36,900 INFO repeater task os150nb_lastensq starting 0.9 seconds late 2017-06-01 15:38:38,075 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 15:38:38,076 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 15:38:26 2017-06-01 15:38:38,076 INFO repeater task os150nb_quick starting 12.3 seconds late 2017-06-01 15:38:41,120 INFO repeater task os150nb_quick ended after 3.0 seconds 2017-06-01 15:38:41,120 INFO repeater task os150nb_3dayplots starting 15.3 seconds late 2017-06-01 15:38:47,463 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 15:38:47,463 INFO repeater task os150nb_3dayplots ended after 6.3 seconds 2017-06-01 15:38:47,463 INFO repeater task os150nb_tsstats starting 21.7 seconds late 2017-06-01 15:38:48,018 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-01 15:38:48,018 INFO repeater task os150bb_beamplots starting 22.2 seconds late 2017-06-01 15:38:48,936 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 15:38:48,936 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 15:38:48,936 INFO repeater task os150bb_beamplots starting 23.2 seconds late 2017-06-01 15:38:49,845 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 15:38:49,845 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 15:38:49,845 INFO repeater task os150nb_beamplots starting 24.1 seconds late 2017-06-01 15:38:50,761 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 15:38:50,761 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 15:38:50,761 INFO repeater task os150nb_beamplots starting 25.0 seconds late 2017-06-01 15:38:51,715 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 15:38:51,716 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-01 15:38:51,716 INFO repeater task posmv_hcorrstats starting 25.9 seconds late 2017-06-01 15:38:52,545 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 15:38:52,545 INFO repeater task posmv_plotposmv starting 26.8 seconds late 2017-06-01 15:38:53,141 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 15:38:58,185 INFO repeater task os150bb_lastensq starting 22.2 seconds late 2017-06-01 15:38:59,823 INFO repeater task os150bb_lastensq ended after 1.6 seconds 2017-06-01 15:38:59,824 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 15:38:26 2017-06-01 15:39:29,889 INFO repeater task os150bb_10pings starting 4.1 seconds late 2017-06-01 15:39:30,000 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 15:39:30,001 INFO repeater task os150nb_10pings starting 4.2 seconds late 2017-06-01 15:39:30,091 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 15:43:30,541 INFO repeater task posmv_hcorrstats starting 4.8 seconds late 2017-06-01 15:43:31,420 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 15:43:31,420 INFO repeater task posmv_plotposmv starting 5.6 seconds late 2017-06-01 15:43:32,023 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 15:43:37,028 INFO repeater task os150bb_lastensq starting 1.0 seconds late 2017-06-01 15:43:38,771 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 15:43:38,772 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 15:43:26 2017-06-01 15:43:38,773 INFO repeater task os150nb_lastensq starting 2.8 seconds late 2017-06-01 15:43:39,911 INFO repeater task os150nb_lastensq ended after 1.1 seconds 2017-06-01 15:43:39,912 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 15:43:26 2017-06-01 15:44:50,055 INFO repeater task os150bb_10pings starting 4.3 seconds late 2017-06-01 15:44:50,198 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 15:44:50,199 INFO repeater task os150nb_10pings starting 4.4 seconds late 2017-06-01 15:44:50,292 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 15:48:30,747 INFO repeater task posmv_hcorrstats starting 5.0 seconds late 2017-06-01 15:48:31,694 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 15:48:31,695 INFO repeater task posmv_plotposmv starting 5.9 seconds late 2017-06-01 15:48:32,312 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 15:48:37,318 INFO repeater task os150bb_lastensq starting 1.3 seconds late 2017-06-01 15:48:38,981 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 15:48:38,981 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 15:48:26 2017-06-01 15:48:38,982 INFO repeater task os150nb_lastensq starting 3.0 seconds late 2017-06-01 15:48:40,236 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-01 15:48:40,236 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 15:48:26 2017-06-01 15:50:10,377 INFO repeater task os150bb_10pings starting 4.6 seconds late 2017-06-01 15:50:10,483 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 15:50:10,484 INFO repeater task os150nb_10pings starting 4.7 seconds late 2017-06-01 15:50:10,583 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 15:53:25,986 INFO repeater task os150bb_quick starting 0.2 seconds late 2017-06-01 15:53:29,235 INFO repeater task os150bb_quick ended after 3.2 seconds 2017-06-01 15:53:29,235 INFO repeater task os150bb_3dayplots starting 3.5 seconds late 2017-06-01 15:53:35,485 INFO repeater task os150bb_3dayplots ended after 6.2 seconds 2017-06-01 15:53:35,485 INFO repeater task os150bb_tsstats starting 9.7 seconds late 2017-06-01 15:53:36,044 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-01 15:53:36,045 INFO repeater task os150nb_lastensq starting 0.0 seconds late 2017-06-01 15:53:37,198 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 15:53:37,199 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 15:53:26 2017-06-01 15:53:37,199 INFO repeater task os150nb_quick starting 11.4 seconds late 2017-06-01 15:53:40,244 INFO repeater task os150nb_quick ended after 3.0 seconds 2017-06-01 15:53:40,245 INFO repeater task os150nb_3dayplots starting 14.5 seconds late 2017-06-01 15:53:46,564 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 15:53:46,564 INFO repeater task os150nb_3dayplots ended after 6.3 seconds 2017-06-01 15:53:46,564 INFO repeater task os150nb_tsstats starting 20.8 seconds late 2017-06-01 15:53:47,115 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-01 15:53:47,115 INFO repeater task os150bb_beamplots starting 21.3 seconds late 2017-06-01 15:53:48,011 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 15:53:48,011 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 15:53:48,012 INFO repeater task os150bb_beamplots starting 22.2 seconds late 2017-06-01 15:53:48,918 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 15:53:48,919 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 15:53:48,919 INFO repeater task os150nb_beamplots starting 23.1 seconds late 2017-06-01 15:53:49,810 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 15:53:49,810 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 15:53:49,810 INFO repeater task os150nb_beamplots starting 24.0 seconds late 2017-06-01 15:53:50,729 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 15:53:50,729 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 15:53:50,730 INFO repeater task posmv_hcorrstats starting 24.9 seconds late 2017-06-01 15:53:51,586 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 15:53:51,586 INFO repeater task posmv_plotposmv starting 25.8 seconds late 2017-06-01 15:53:52,190 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 15:53:57,196 INFO repeater task os150bb_lastensq starting 21.2 seconds late 2017-06-01 15:53:58,882 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 15:53:58,882 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 15:53:26 2017-06-01 15:55:29,064 INFO repeater task os150bb_10pings starting 3.3 seconds late 2017-06-01 15:55:29,219 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 15:55:29,220 INFO repeater task os150nb_10pings starting 3.4 seconds late 2017-06-01 15:55:29,311 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 15:58:29,660 INFO repeater task posmv_hcorrstats starting 3.9 seconds late 2017-06-01 15:58:30,635 INFO repeater task posmv_hcorrstats ended after 1.0 seconds 2017-06-01 15:58:30,635 INFO repeater task posmv_plotposmv starting 4.9 seconds late 2017-06-01 15:58:31,264 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 15:58:36,270 INFO repeater task os150bb_lastensq starting 0.3 seconds late 2017-06-01 15:58:38,031 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 15:58:38,031 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 15:58:26 2017-06-01 15:58:38,032 INFO repeater task os150nb_lastensq starting 2.0 seconds late 2017-06-01 15:58:39,252 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 15:58:39,253 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 15:58:26 2017-06-01 16:00:49,519 INFO repeater task os150bb_10pings starting 3.7 seconds late 2017-06-01 16:00:49,694 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 16:00:49,695 INFO repeater task os150nb_10pings starting 3.9 seconds late 2017-06-01 16:00:49,795 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 16:03:30,125 INFO repeater task posmv_hcorrstats starting 4.3 seconds late 2017-06-01 16:03:30,963 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 16:03:30,964 INFO repeater task posmv_plotposmv starting 5.2 seconds late 2017-06-01 16:03:31,573 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 16:03:36,578 INFO repeater task os150bb_lastensq starting 0.6 seconds late 2017-06-01 16:03:38,284 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 16:03:38,284 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 16:03:26 2017-06-01 16:03:38,285 INFO repeater task os150nb_lastensq starting 2.3 seconds late 2017-06-01 16:03:39,431 INFO repeater task os150nb_lastensq ended after 1.1 seconds 2017-06-01 16:03:39,432 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 16:03:26 2017-06-01 16:06:09,760 INFO repeater task os150bb_10pings starting 4.0 seconds late 2017-06-01 16:06:09,919 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 16:06:09,920 INFO repeater task os150nb_10pings starting 4.1 seconds late 2017-06-01 16:06:10,020 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 16:08:30,303 INFO repeater task os150bb_quick starting 4.5 seconds late 2017-06-01 16:08:33,446 INFO repeater task os150bb_quick ended after 3.1 seconds 2017-06-01 16:08:33,447 INFO repeater task os150bb_3dayplots starting 7.7 seconds late 2017-06-01 16:08:39,725 INFO repeater task os150bb_3dayplots ended after 6.3 seconds 2017-06-01 16:08:39,725 INFO repeater task os150bb_tsstats starting 13.9 seconds late 2017-06-01 16:08:40,274 INFO repeater task os150bb_tsstats ended after 0.5 seconds 2017-06-01 16:08:40,275 INFO repeater task os150nb_lastensq starting 4.3 seconds late 2017-06-01 16:08:41,448 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 16:08:41,448 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 16:08:26 2017-06-01 16:08:41,448 INFO repeater task os150nb_quick starting 15.7 seconds late 2017-06-01 16:08:44,509 INFO repeater task os150nb_quick ended after 3.1 seconds 2017-06-01 16:08:44,509 INFO repeater task os150nb_3dayplots starting 18.7 seconds late 2017-06-01 16:08:50,827 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 16:08:50,828 INFO repeater task os150nb_3dayplots ended after 6.3 seconds 2017-06-01 16:08:50,828 INFO repeater task os150nb_tsstats starting 25.0 seconds late 2017-06-01 16:08:51,411 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-01 16:08:51,412 INFO repeater task os150bb_beamplots starting 25.6 seconds late 2017-06-01 16:08:52,311 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 16:08:52,311 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 16:08:52,311 INFO repeater task os150bb_beamplots starting 26.5 seconds late 2017-06-01 16:08:53,213 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 16:08:53,213 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 16:08:53,213 INFO repeater task os150nb_beamplots starting 27.4 seconds late 2017-06-01 16:08:54,102 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 16:08:54,103 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 16:08:54,103 INFO repeater task os150nb_beamplots starting 28.3 seconds late 2017-06-01 16:08:55,029 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 16:08:55,029 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 16:08:55,029 INFO repeater task posmv_hcorrstats starting 29.2 seconds late 2017-06-01 16:08:55,875 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 16:08:55,875 INFO repeater task posmv_plotposmv starting 30.1 seconds late 2017-06-01 16:08:56,499 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 16:09:01,505 INFO repeater task os150bb_lastensq starting 25.5 seconds late 2017-06-01 16:09:03,201 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 16:09:03,201 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 16:08:26 2017-06-01 16:11:28,530 INFO repeater task os150bb_10pings starting 2.8 seconds late 2017-06-01 16:11:28,689 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 16:11:28,690 INFO repeater task os150nb_10pings starting 2.9 seconds late 2017-06-01 16:11:28,787 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 16:13:29,036 INFO repeater task posmv_hcorrstats starting 3.3 seconds late 2017-06-01 16:13:29,995 INFO repeater task posmv_hcorrstats ended after 1.0 seconds 2017-06-01 16:13:29,995 INFO repeater task posmv_plotposmv starting 4.2 seconds late 2017-06-01 16:13:30,647 INFO repeater task posmv_plotposmv ended after 0.7 seconds 2017-06-01 16:13:40,665 INFO repeater task os150bb_lastensq starting 4.7 seconds late 2017-06-01 16:13:42,488 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 16:13:42,488 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 16:13:26 2017-06-01 16:13:42,489 INFO repeater task os150nb_lastensq starting 6.5 seconds late 2017-06-01 16:13:43,725 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 16:13:43,725 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 16:13:26 2017-06-01 16:16:49,101 INFO repeater task os150bb_10pings starting 3.3 seconds late 2017-06-01 16:16:49,268 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 16:16:49,269 INFO repeater task os150nb_10pings starting 3.5 seconds late 2017-06-01 16:16:49,362 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 16:18:29,554 INFO repeater task posmv_hcorrstats starting 3.8 seconds late 2017-06-01 16:18:30,492 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 16:18:30,492 INFO repeater task posmv_plotposmv starting 4.7 seconds late 2017-06-01 16:18:31,111 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 16:18:36,117 INFO repeater task os150bb_lastensq starting 0.1 seconds late 2017-06-01 16:18:37,798 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 16:18:37,798 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 16:18:26 2017-06-01 16:18:37,799 INFO repeater task os150nb_lastensq starting 1.8 seconds late 2017-06-01 16:18:38,937 INFO repeater task os150nb_lastensq ended after 1.1 seconds 2017-06-01 16:18:38,938 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 16:18:26 2017-06-01 16:22:09,372 INFO repeater task os150bb_10pings starting 3.6 seconds late 2017-06-01 16:22:09,516 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 16:22:09,517 INFO repeater task os150nb_10pings starting 3.7 seconds late 2017-06-01 16:22:09,609 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 16:23:29,776 INFO repeater task os150bb_quick starting 4.0 seconds late 2017-06-01 16:23:32,950 INFO repeater task os150bb_quick ended after 3.2 seconds 2017-06-01 16:23:32,950 INFO repeater task os150bb_3dayplots starting 7.2 seconds late 2017-06-01 16:23:39,430 INFO repeater task os150bb_3dayplots ended after 6.5 seconds 2017-06-01 16:23:39,431 INFO repeater task os150bb_tsstats starting 13.7 seconds late 2017-06-01 16:23:40,014 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-01 16:23:40,014 INFO repeater task os150nb_lastensq starting 4.0 seconds late 2017-06-01 16:23:41,186 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 16:23:41,187 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 16:23:26 2017-06-01 16:23:41,187 INFO repeater task os150nb_quick starting 15.4 seconds late 2017-06-01 16:23:44,360 INFO repeater task os150nb_quick ended after 3.2 seconds 2017-06-01 16:23:44,360 INFO repeater task os150nb_3dayplots starting 18.6 seconds late 2017-06-01 16:23:51,266 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 16:23:51,266 INFO repeater task os150nb_3dayplots ended after 6.9 seconds 2017-06-01 16:23:51,266 INFO repeater task os150nb_tsstats starting 25.5 seconds late 2017-06-01 16:23:51,857 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-01 16:23:51,857 INFO repeater task os150bb_beamplots starting 26.1 seconds late 2017-06-01 16:23:52,778 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 16:23:52,778 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 16:23:52,778 INFO repeater task os150bb_beamplots starting 27.0 seconds late 2017-06-01 16:23:53,720 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 16:23:53,720 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 16:23:53,721 INFO repeater task os150nb_beamplots starting 27.9 seconds late 2017-06-01 16:23:54,681 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin