2017-06-01 09:23:41,960 INFO repeater task os150nb_3dayplots starting 16.2 seconds late 2017-06-01 09:23:48,259 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 09:23:48,260 INFO repeater task os150nb_3dayplots ended after 6.3 seconds 2017-06-01 09:23:48,260 INFO repeater task os150nb_tsstats starting 22.5 seconds late 2017-06-01 09:23:48,802 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2017-06-01 09:23:48,803 INFO repeater task os150bb_beamplots starting 23.0 seconds late 2017-06-01 09:23:49,721 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 09:23:49,721 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 09:23:49,721 INFO repeater task os150bb_beamplots starting 23.9 seconds late 2017-06-01 09:23:50,634 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 09:23:50,634 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 09:23:50,634 INFO repeater task os150nb_beamplots starting 24.9 seconds late 2017-06-01 09:23:51,545 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 09:23:51,545 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 09:23:51,545 INFO repeater task os150nb_beamplots starting 25.8 seconds late 2017-06-01 09:23:52,470 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 09:23:52,470 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 09:23:52,470 INFO repeater task posmv_hcorrstats starting 26.7 seconds late 2017-06-01 09:23:53,293 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 09:23:53,294 INFO repeater task posmv_plotposmv starting 27.5 seconds late 2017-06-01 09:23:53,921 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 09:23:58,927 INFO repeater task os150bb_lastensq starting 22.9 seconds late 2017-06-01 09:24:00,648 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 09:24:00,648 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 09:23:26 2017-06-01 09:26:05,888 INFO repeater task os150bb_10pings starting 0.1 seconds late 2017-06-01 09:26:06,054 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 09:26:06,055 INFO repeater task os150nb_10pings starting 0.3 seconds late 2017-06-01 09:26:06,149 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 09:28:26,516 INFO repeater task posmv_hcorrstats starting 0.7 seconds late 2017-06-01 09:28:27,357 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 09:28:27,358 INFO repeater task posmv_plotposmv starting 1.6 seconds late 2017-06-01 09:28:27,982 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 09:28:37,999 INFO repeater task os150bb_lastensq starting 2.0 seconds late 2017-06-01 09:28:39,769 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 09:28:39,769 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 09:28:26 2017-06-01 09:28:39,770 INFO repeater task os150nb_lastensq starting 3.8 seconds late 2017-06-01 09:28:40,935 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 09:28:40,935 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 09:28:26 2017-06-01 09:31:26,244 INFO repeater task os150bb_10pings starting 0.5 seconds late 2017-06-01 09:31:26,337 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 09:31:26,338 INFO repeater task os150nb_10pings starting 0.6 seconds late 2017-06-01 09:31:26,431 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 09:33:26,701 INFO repeater task posmv_hcorrstats starting 0.9 seconds late 2017-06-01 09:33:27,522 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 09:33:27,522 INFO repeater task posmv_plotposmv starting 1.7 seconds late 2017-06-01 09:33:28,124 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 09:33:38,142 INFO repeater task os150bb_lastensq starting 2.1 seconds late 2017-06-01 09:33:39,872 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 09:33:39,873 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 09:33:26 2017-06-01 09:33:39,873 INFO repeater task os150nb_lastensq starting 3.9 seconds late 2017-06-01 09:33:41,054 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 09:33:41,054 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 09:33:26 2017-06-01 09:36:46,457 INFO repeater task os150bb_10pings starting 0.7 seconds late 2017-06-01 09:36:46,625 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 09:36:46,626 INFO repeater task os150nb_10pings starting 0.8 seconds late 2017-06-01 09:36:46,718 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 09:38:26,899 INFO repeater task os150bb_quick starting 1.1 seconds late 2017-06-01 09:38:29,973 INFO repeater task os150bb_quick ended after 3.1 seconds 2017-06-01 09:38:29,973 INFO repeater task os150bb_3dayplots starting 4.2 seconds late 2017-06-01 09:38:36,197 INFO repeater task os150bb_3dayplots ended after 6.2 seconds 2017-06-01 09:38:36,197 INFO repeater task os150bb_tsstats starting 10.4 seconds late 2017-06-01 09:38:36,742 INFO repeater task os150bb_tsstats ended after 0.5 seconds 2017-06-01 09:38:36,742 INFO repeater task os150nb_lastensq starting 0.7 seconds late 2017-06-01 09:38:37,884 INFO repeater task os150nb_lastensq ended after 1.1 seconds 2017-06-01 09:38:37,885 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 09:38:26 2017-06-01 09:38:37,885 INFO repeater task os150nb_quick starting 12.1 seconds late 2017-06-01 09:38:40,928 INFO repeater task os150nb_quick ended after 3.0 seconds 2017-06-01 09:38:40,928 INFO repeater task os150nb_3dayplots starting 15.1 seconds late 2017-06-01 09:38:47,315 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 09:38:47,315 INFO repeater task os150nb_3dayplots ended after 6.4 seconds 2017-06-01 09:38:47,316 INFO repeater task os150nb_tsstats starting 21.5 seconds late 2017-06-01 09:38:47,851 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2017-06-01 09:38:47,851 INFO repeater task os150bb_beamplots starting 22.1 seconds late 2017-06-01 09:38:48,746 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 09:38:48,746 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 09:38:48,746 INFO repeater task os150bb_beamplots starting 23.0 seconds late 2017-06-01 09:38:49,690 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 09:38:49,690 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 09:38:49,691 INFO repeater task os150nb_beamplots starting 23.9 seconds late 2017-06-01 09:38:50,579 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 09:38:50,579 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 09:38:50,579 INFO repeater task os150nb_beamplots starting 24.8 seconds late 2017-06-01 09:38:51,529 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 09:38:51,529 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 09:38:51,529 INFO repeater task posmv_hcorrstats starting 25.7 seconds late 2017-06-01 09:38:52,356 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 09:38:52,356 INFO repeater task posmv_plotposmv starting 26.6 seconds late 2017-06-01 09:38:52,956 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 09:38:57,962 INFO repeater task os150bb_lastensq starting 22.0 seconds late 2017-06-01 09:38:59,766 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 09:38:59,766 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 09:38:26 2017-06-01 09:42:10,174 INFO repeater task os150bb_10pings starting 4.4 seconds late 2017-06-01 09:42:10,343 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 09:42:10,344 INFO repeater task os150nb_10pings starting 4.6 seconds late 2017-06-01 09:42:10,442 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 09:43:30,686 INFO repeater task posmv_hcorrstats starting 4.9 seconds late 2017-06-01 09:43:31,603 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 09:43:31,604 INFO repeater task posmv_plotposmv starting 5.8 seconds late 2017-06-01 09:43:32,275 INFO repeater task posmv_plotposmv ended after 0.7 seconds 2017-06-01 09:43:37,280 INFO repeater task os150bb_lastensq starting 1.3 seconds late 2017-06-01 09:43:38,982 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 09:43:38,982 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 09:43:26 2017-06-01 09:43:38,983 INFO repeater task os150nb_lastensq starting 3.0 seconds late 2017-06-01 09:43:40,190 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 09:43:40,190 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 09:43:26 2017-06-01 09:47:30,739 INFO repeater task os150bb_10pings starting 5.0 seconds late 2017-06-01 09:47:30,833 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 09:47:30,837 INFO repeater task os150nb_10pings starting 5.1 seconds late 2017-06-01 09:47:30,930 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 09:48:26,044 INFO repeater task posmv_hcorrstats starting 0.3 seconds late 2017-06-01 09:48:27,021 INFO repeater task posmv_hcorrstats ended after 1.0 seconds 2017-06-01 09:48:27,021 INFO repeater task posmv_plotposmv starting 1.2 seconds late 2017-06-01 09:48:27,690 INFO repeater task posmv_plotposmv ended after 0.7 seconds 2017-06-01 09:48:37,708 INFO repeater task os150bb_lastensq starting 1.7 seconds late 2017-06-01 09:48:39,465 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 09:48:39,465 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 09:48:26 2017-06-01 09:48:39,466 INFO repeater task os150nb_lastensq starting 3.5 seconds late 2017-06-01 09:48:40,712 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 09:48:40,712 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 09:48:26 2017-06-01 09:52:46,237 INFO repeater task os150bb_10pings starting 0.5 seconds late 2017-06-01 09:52:46,329 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 09:52:46,330 INFO repeater task os150nb_10pings starting 0.5 seconds late 2017-06-01 09:52:46,425 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 09:53:26,546 INFO repeater task os150bb_quick starting 0.8 seconds late 2017-06-01 09:53:29,633 INFO repeater task os150bb_quick ended after 3.1 seconds 2017-06-01 09:53:29,633 INFO repeater task os150bb_3dayplots starting 3.9 seconds late 2017-06-01 09:53:36,474 INFO repeater task os150bb_3dayplots ended after 6.8 seconds 2017-06-01 09:53:36,474 INFO repeater task os150bb_tsstats starting 10.7 seconds late 2017-06-01 09:53:37,048 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-01 09:53:37,048 INFO repeater task os150nb_lastensq starting 1.0 seconds late 2017-06-01 09:53:38,354 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-01 09:53:38,354 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 09:53:26 2017-06-01 09:53:38,355 INFO repeater task os150nb_quick starting 12.6 seconds late 2017-06-01 09:53:41,471 INFO repeater task os150nb_quick ended after 3.1 seconds 2017-06-01 09:53:41,471 INFO repeater task os150nb_3dayplots starting 15.7 seconds late 2017-06-01 09:53:48,415 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 09:53:48,415 INFO repeater task os150nb_3dayplots ended after 6.9 seconds 2017-06-01 09:53:48,415 INFO repeater task os150nb_tsstats starting 22.6 seconds late 2017-06-01 09:53:48,969 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-01 09:53:48,969 INFO repeater task os150bb_beamplots starting 23.2 seconds late 2017-06-01 09:53:49,878 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 09:53:49,878 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 09:53:49,878 INFO repeater task os150bb_beamplots starting 24.1 seconds late 2017-06-01 09:53:50,812 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 09:53:50,812 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 09:53:50,812 INFO repeater task os150nb_beamplots starting 25.0 seconds late 2017-06-01 09:53:51,722 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 09:53:51,722 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 09:53:51,722 INFO repeater task os150nb_beamplots starting 25.9 seconds late 2017-06-01 09:53:52,656 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 09:53:52,657 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 09:53:52,657 INFO repeater task posmv_hcorrstats starting 26.9 seconds late 2017-06-01 09:53:53,517 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 09:53:53,517 INFO repeater task posmv_plotposmv starting 27.7 seconds late 2017-06-01 09:53:54,161 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 09:53:59,166 INFO repeater task os150bb_lastensq starting 23.2 seconds late 2017-06-01 09:54:00,977 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 09:54:00,977 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 09:53:26 2017-06-01 09:58:06,582 INFO repeater task os150bb_10pings starting 0.8 seconds late 2017-06-01 09:58:06,728 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 09:58:06,729 INFO repeater task os150nb_10pings starting 0.9 seconds late 2017-06-01 09:58:06,823 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 09:58:26,862 INFO repeater task posmv_hcorrstats starting 1.1 seconds late 2017-06-01 09:58:27,764 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 09:58:27,764 INFO repeater task posmv_plotposmv starting 2.0 seconds late 2017-06-01 09:58:28,391 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 09:58:38,401 INFO repeater task os150bb_lastensq starting 2.4 seconds late 2017-06-01 09:58:40,225 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 09:58:40,225 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 09:58:26 2017-06-01 09:58:40,226 INFO repeater task os150nb_lastensq starting 4.2 seconds late 2017-06-01 09:58:41,404 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 09:58:41,404 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 09:58:26 2017-06-01 10:03:27,069 INFO repeater task os150bb_10pings starting 1.3 seconds late 2017-06-01 10:03:27,244 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 10:03:27,245 INFO repeater task os150nb_10pings starting 1.5 seconds late 2017-06-01 10:03:27,338 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 10:03:27,340 INFO repeater task posmv_hcorrstats starting 1.6 seconds late 2017-06-01 10:03:28,167 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 10:03:28,167 INFO repeater task posmv_plotposmv starting 2.4 seconds late 2017-06-01 10:03:28,773 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 10:03:38,785 INFO repeater task os150bb_lastensq starting 2.8 seconds late 2017-06-01 10:03:40,523 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 10:03:40,523 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 10:03:26 2017-06-01 10:03:40,524 INFO repeater task os150nb_lastensq starting 4.5 seconds late 2017-06-01 10:03:41,701 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 10:03:41,702 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 10:03:26 2017-06-01 10:08:27,290 INFO repeater task os150bb_quick starting 1.5 seconds late 2017-06-01 10:08:30,426 INFO repeater task os150bb_quick ended after 3.1 seconds 2017-06-01 10:08:30,426 INFO repeater task os150bb_3dayplots starting 4.6 seconds late 2017-06-01 10:08:36,703 INFO repeater task os150bb_3dayplots ended after 6.3 seconds 2017-06-01 10:08:36,703 INFO repeater task os150bb_tsstats starting 10.9 seconds late 2017-06-01 10:08:37,243 INFO repeater task os150bb_tsstats ended after 0.5 seconds 2017-06-01 10:08:37,243 INFO repeater task os150nb_lastensq starting 1.2 seconds late 2017-06-01 10:08:38,397 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 10:08:38,397 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 10:08:26 2017-06-01 10:08:38,397 INFO repeater task os150nb_quick starting 12.6 seconds late 2017-06-01 10:08:41,423 INFO repeater task os150nb_quick ended after 3.0 seconds 2017-06-01 10:08:41,423 INFO repeater task os150nb_3dayplots starting 15.6 seconds late 2017-06-01 10:08:47,738 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 10:08:47,738 INFO repeater task os150nb_3dayplots ended after 6.3 seconds 2017-06-01 10:08:47,739 INFO repeater task os150nb_tsstats starting 22.0 seconds late 2017-06-01 10:08:48,277 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2017-06-01 10:08:48,277 INFO repeater task os150bb_beamplots starting 22.5 seconds late 2017-06-01 10:08:49,191 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 10:08:49,191 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 10:08:49,191 INFO repeater task os150bb_beamplots starting 23.4 seconds late 2017-06-01 10:08:50,119 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 10:08:50,119 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 10:08:50,120 INFO repeater task os150nb_beamplots starting 24.3 seconds late 2017-06-01 10:08:51,028 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 10:08:51,028 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 10:08:51,029 INFO repeater task os150nb_beamplots starting 25.2 seconds late 2017-06-01 10:08:51,940 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 10:08:51,941 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 10:08:51,941 INFO repeater task posmv_hcorrstats starting 26.2 seconds late 2017-06-01 10:08:52,764 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 10:08:52,764 INFO repeater task posmv_plotposmv starting 27.0 seconds late 2017-06-01 10:08:53,352 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 10:08:58,356 INFO repeater task os150bb_lastensq starting 22.4 seconds late 2017-06-01 10:09:00,069 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 10:09:00,069 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 10:08:26 2017-06-01 10:09:00,069 INFO repeater task os150bb_10pings starting 14.3 seconds late 2017-06-01 10:09:00,165 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 10:09:00,167 INFO repeater task os150nb_10pings starting 14.4 seconds late 2017-06-01 10:09:00,260 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 10:13:30,786 INFO repeater task posmv_hcorrstats starting 5.0 seconds late 2017-06-01 10:13:31,706 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 10:13:31,707 INFO repeater task posmv_plotposmv starting 5.9 seconds late 2017-06-01 10:13:32,310 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 10:13:37,316 INFO repeater task os150bb_lastensq starting 1.3 seconds late 2017-06-01 10:13:38,933 INFO repeater task os150bb_lastensq ended after 1.6 seconds 2017-06-01 10:13:38,933 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 10:13:26 2017-06-01 10:13:38,934 INFO repeater task os150nb_lastensq starting 2.9 seconds late 2017-06-01 10:13:40,109 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 10:13:40,110 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 10:13:26 2017-06-01 10:14:10,167 INFO repeater task os150bb_10pings starting 4.4 seconds late 2017-06-01 10:14:10,330 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 10:14:10,332 INFO repeater task os150nb_10pings starting 4.6 seconds late 2017-06-01 10:14:10,425 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 10:18:25,937 INFO repeater task posmv_hcorrstats starting 0.2 seconds late 2017-06-01 10:18:26,890 INFO repeater task posmv_hcorrstats ended after 1.0 seconds 2017-06-01 10:18:26,891 INFO repeater task posmv_plotposmv starting 1.1 seconds late 2017-06-01 10:18:27,506 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 10:18:37,523 INFO repeater task os150bb_lastensq starting 1.5 seconds late 2017-06-01 10:18:39,232 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 10:18:39,232 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 10:18:26 2017-06-01 10:18:39,233 INFO repeater task os150nb_lastensq starting 3.2 seconds late 2017-06-01 10:18:40,418 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 10:18:40,418 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 10:18:26 2017-06-01 10:19:30,515 INFO repeater task os150bb_10pings starting 4.7 seconds late 2017-06-01 10:19:30,636 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 10:19:30,637 INFO repeater task os150nb_10pings starting 4.9 seconds late 2017-06-01 10:19:30,730 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 10:23:26,184 INFO repeater task os150bb_quick starting 0.4 seconds late 2017-06-01 10:23:29,330 INFO repeater task os150bb_quick ended after 3.1 seconds 2017-06-01 10:23:29,331 INFO repeater task os150bb_3dayplots starting 3.6 seconds late 2017-06-01 10:23:35,524 INFO repeater task os150bb_3dayplots ended after 6.2 seconds 2017-06-01 10:23:35,524 INFO repeater task os150bb_tsstats starting 9.7 seconds late 2017-06-01 10:23:36,070 INFO repeater task os150bb_tsstats ended after 0.5 seconds 2017-06-01 10:23:36,070 INFO repeater task os150nb_lastensq starting 0.1 seconds late 2017-06-01 10:23:37,192 INFO repeater task os150nb_lastensq ended after 1.1 seconds 2017-06-01 10:23:37,192 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 10:23:26 2017-06-01 10:23:37,192 INFO repeater task os150nb_quick starting 11.4 seconds late 2017-06-01 10:23:40,225 INFO repeater task os150nb_quick ended after 3.0 seconds 2017-06-01 10:23:40,225 INFO repeater task os150nb_3dayplots starting 14.4 seconds late 2017-06-01 10:23:46,545 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 10:23:46,545 INFO repeater task os150nb_3dayplots ended after 6.3 seconds 2017-06-01 10:23:46,545 INFO repeater task os150nb_tsstats starting 20.8 seconds late 2017-06-01 10:23:47,106 INFO repeater task os150nb_tsstats output: /usr/lib/python2.7/dist-packages/numpy/ma/core.py:1073: RuntimeWarning: overflow encountered in true_divide result = self.f(da, db, *args, **kwargs) 2017-06-01 10:23:47,107 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-01 10:23:47,107 INFO repeater task os150bb_beamplots starting 21.3 seconds late 2017-06-01 10:23:48,006 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 10:23:48,007 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 10:23:48,007 INFO repeater task os150bb_beamplots starting 22.2 seconds late 2017-06-01 10:23:48,939 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 10:23:48,939 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 10:23:48,939 INFO repeater task os150nb_beamplots starting 23.2 seconds late 2017-06-01 10:23:49,854 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 10:23:49,854 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 10:23:49,854 INFO repeater task os150nb_beamplots starting 24.1 seconds late 2017-06-01 10:23:50,760 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 10:23:50,760 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 10:23:50,760 INFO repeater task posmv_hcorrstats starting 25.0 seconds late 2017-06-01 10:23:51,606 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 10:23:51,606 INFO repeater task posmv_plotposmv starting 25.8 seconds late 2017-06-01 10:23:52,201 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 10:23:57,207 INFO repeater task os150bb_lastensq starting 21.2 seconds late 2017-06-01 10:23:58,931 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 10:23:58,931 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 10:23:26 2017-06-01 10:24:49,040 INFO repeater task os150bb_10pings starting 3.3 seconds late 2017-06-01 10:24:49,203 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 10:24:49,204 INFO repeater task os150nb_10pings starting 3.4 seconds late 2017-06-01 10:24:49,294 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 10:28:29,796 INFO repeater task posmv_hcorrstats starting 4.0 seconds late 2017-06-01 10:28:30,749 INFO repeater task posmv_hcorrstats ended after 1.0 seconds 2017-06-01 10:28:30,749 INFO repeater task posmv_plotposmv starting 5.0 seconds late 2017-06-01 10:28:31,384 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 10:28:36,390 INFO repeater task os150bb_lastensq starting 0.4 seconds late 2017-06-01 10:28:38,209 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 10:28:38,209 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 10:28:26 2017-06-01 10:28:38,210 INFO repeater task os150nb_lastensq starting 2.2 seconds late 2017-06-01 10:28:39,481 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-01 10:28:39,482 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 10:28:26 2017-06-01 10:30:09,652 INFO repeater task os150bb_10pings starting 3.9 seconds late 2017-06-01 10:30:09,830 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 10:30:09,830 INFO repeater task os150nb_10pings starting 4.0 seconds late 2017-06-01 10:30:09,927 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 10:33:30,356 INFO repeater task posmv_hcorrstats starting 4.6 seconds late 2017-06-01 10:33:31,237 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 10:33:31,238 INFO repeater task posmv_plotposmv starting 5.5 seconds late 2017-06-01 10:33:31,886 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 10:33:36,892 INFO repeater task os150bb_lastensq starting 0.9 seconds late 2017-06-01 10:33:38,707 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 10:33:38,708 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 10:33:26 2017-06-01 10:33:38,708 INFO repeater task os150nb_lastensq starting 2.7 seconds late 2017-06-01 10:33:39,911 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 10:33:39,911 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 10:33:26 2017-06-01 10:35:30,133 INFO repeater task os150bb_10pings starting 4.4 seconds late 2017-06-01 10:35:30,311 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 10:35:30,312 INFO repeater task os150nb_10pings starting 4.5 seconds late 2017-06-01 10:35:30,404 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 10:38:25,842 INFO repeater task os150bb_quick starting 0.1 seconds late 2017-06-01 10:38:29,051 INFO repeater task os150bb_quick ended after 3.2 seconds 2017-06-01 10:38:29,052 INFO repeater task os150bb_3dayplots starting 3.3 seconds late 2017-06-01 10:38:35,703 INFO repeater task os150bb_3dayplots ended after 6.7 seconds 2017-06-01 10:38:35,704 INFO repeater task os150bb_tsstats starting 9.9 seconds late 2017-06-01 10:38:36,262 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-01 10:38:36,263 INFO repeater task os150nb_lastensq starting 0.3 seconds late 2017-06-01 10:38:37,505 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 10:38:37,505 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 10:38:26 2017-06-01 10:38:37,505 INFO repeater task os150nb_quick starting 11.7 seconds late 2017-06-01 10:38:40,566 INFO repeater task os150nb_quick ended after 3.1 seconds 2017-06-01 10:38:40,566 INFO repeater task os150nb_3dayplots starting 14.8 seconds late 2017-06-01 10:38:46,941 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 10:38:46,942 INFO repeater task os150nb_3dayplots ended after 6.4 seconds 2017-06-01 10:38:46,942 INFO repeater task os150nb_tsstats starting 21.2 seconds late 2017-06-01 10:38:47,501 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-01 10:38:47,501 INFO repeater task os150bb_beamplots starting 21.7 seconds late 2017-06-01 10:38:48,469 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 10:38:48,469 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-01 10:38:48,470 INFO repeater task os150bb_beamplots starting 22.7 seconds late 2017-06-01 10:38:49,434 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 10:38:49,434 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-01 10:38:49,434 INFO repeater task os150nb_beamplots starting 23.7 seconds late 2017-06-01 10:38:50,408 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 10:38:50,408 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-01 10:38:50,408 INFO repeater task os150nb_beamplots starting 24.6 seconds late 2017-06-01 10:38:51,330 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 10:38:51,330 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 10:38:51,330 INFO repeater task posmv_hcorrstats starting 25.5 seconds late 2017-06-01 10:38:52,209 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 10:38:52,209 INFO repeater task posmv_plotposmv starting 26.4 seconds late 2017-06-01 10:38:52,881 INFO repeater task posmv_plotposmv ended after 0.7 seconds 2017-06-01 10:38:57,887 INFO repeater task os150bb_lastensq starting 21.9 seconds late 2017-06-01 10:38:59,671 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 10:38:59,671 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 10:38:26 2017-06-01 10:40:49,879 INFO repeater task os150bb_10pings starting 4.1 seconds late 2017-06-01 10:40:50,048 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 10:40:50,048 INFO repeater task os150nb_10pings starting 4.3 seconds late 2017-06-01 10:40:50,140 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 10:43:30,460 INFO repeater task posmv_hcorrstats starting 4.7 seconds late 2017-06-01 10:43:31,368 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 10:43:31,369 INFO repeater task posmv_plotposmv starting 5.6 seconds late 2017-06-01 10:43:31,977 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 10:43:36,983 INFO repeater task os150bb_lastensq starting 1.0 seconds late 2017-06-01 10:43:38,805 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 10:43:38,805 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 10:43:26 2017-06-01 10:43:38,806 INFO repeater task os150nb_lastensq starting 2.8 seconds late 2017-06-01 10:43:40,008 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 10:43:40,008 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 10:43:26 2017-06-01 10:46:10,318 INFO repeater task os150bb_10pings starting 4.5 seconds late 2017-06-01 10:46:10,413 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 10:46:10,414 INFO repeater task os150nb_10pings starting 4.6 seconds late 2017-06-01 10:46:10,510 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 10:48:30,787 INFO repeater task posmv_hcorrstats starting 5.0 seconds late 2017-06-01 10:48:31,794 INFO repeater task posmv_hcorrstats ended after 1.0 seconds 2017-06-01 10:48:31,795 INFO repeater task posmv_plotposmv starting 6.0 seconds late 2017-06-01 10:48:32,475 INFO repeater task posmv_plotposmv ended after 0.7 seconds 2017-06-01 10:48:37,481 INFO repeater task os150bb_lastensq starting 1.5 seconds late 2017-06-01 10:48:39,257 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 10:48:39,257 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 10:48:26 2017-06-01 10:48:39,258 INFO repeater task os150nb_lastensq starting 3.3 seconds late 2017-06-01 10:48:40,476 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 10:48:40,476 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 10:48:26 2017-06-01 10:51:25,872 INFO repeater task os150bb_10pings starting 0.1 seconds late 2017-06-01 10:51:26,056 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 10:51:26,057 INFO repeater task os150nb_10pings starting 0.3 seconds late 2017-06-01 10:51:26,155 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 10:53:26,390 INFO repeater task os150bb_quick starting 0.6 seconds late 2017-06-01 10:53:29,485 INFO repeater task os150bb_quick ended after 3.1 seconds 2017-06-01 10:53:29,486 INFO repeater task os150bb_3dayplots starting 3.7 seconds late 2017-06-01 10:53:35,864 INFO repeater task os150bb_3dayplots ended after 6.4 seconds 2017-06-01 10:53:35,865 INFO repeater task os150bb_tsstats starting 10.1 seconds late 2017-06-01 10:53:36,408 INFO repeater task os150bb_tsstats ended after 0.5 seconds 2017-06-01 10:53:36,408 INFO repeater task os150nb_lastensq starting 0.4 seconds late 2017-06-01 10:53:37,615 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 10:53:37,615 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 10:53:26 2017-06-01 10:53:37,615 INFO repeater task os150nb_quick starting 11.8 seconds late 2017-06-01 10:53:40,649 INFO repeater task os150nb_quick ended after 3.0 seconds 2017-06-01 10:53:40,649 INFO repeater task os150nb_3dayplots starting 14.9 seconds late 2017-06-01 10:53:46,983 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 10:53:46,983 INFO repeater task os150nb_3dayplots ended after 6.3 seconds 2017-06-01 10:53:46,983 INFO repeater task os150nb_tsstats starting 21.2 seconds late 2017-06-01 10:53:47,529 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2017-06-01 10:53:47,529 INFO repeater task os150bb_beamplots starting 21.7 seconds late 2017-06-01 10:53:48,466 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 10:53:48,466 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 10:53:48,466 INFO repeater task os150bb_beamplots starting 22.7 seconds late 2017-06-01 10:53:49,372 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 10:53:49,372 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 10:53:49,373 INFO repeater task os150nb_beamplots starting 23.6 seconds late 2017-06-01 10:53:50,286 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 10:53:50,286 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 10:53:50,286 INFO repeater task os150nb_beamplots starting 24.5 seconds late 2017-06-01 10:53:51,185 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 10:53:51,185 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 10:53:51,185 INFO repeater task posmv_hcorrstats starting 25.4 seconds late 2017-06-01 10:53:52,005 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 10:53:52,006 INFO repeater task posmv_plotposmv starting 26.2 seconds late 2017-06-01 10:53:52,607 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 10:53:57,613 INFO repeater task os150bb_lastensq starting 21.6 seconds late 2017-06-01 10:53:59,304 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 10:53:59,304 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 10:53:26 2017-06-01 10:56:49,632 INFO repeater task os150bb_10pings starting 3.9 seconds late 2017-06-01 10:56:49,788 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 10:56:49,789 INFO repeater task os150nb_10pings starting 4.0 seconds late 2017-06-01 10:56:49,884 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 10:58:30,066 INFO repeater task posmv_hcorrstats starting 4.3 seconds late 2017-06-01 10:58:30,907 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 10:58:30,907 INFO repeater task posmv_plotposmv starting 5.1 seconds late 2017-06-01 10:58:31,541 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 10:58:36,546 INFO repeater task os150bb_lastensq starting 0.5 seconds late 2017-06-01 10:58:38,264 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 10:58:38,265 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 10:58:26 2017-06-01 10:58:38,265 INFO repeater task os150nb_lastensq starting 2.3 seconds late 2017-06-01 10:58:39,433 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 10:58:39,433 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 10:58:26 2017-06-01 11:02:09,873 INFO repeater task os150bb_10pings starting 4.1 seconds late 2017-06-01 11:02:10,004 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 11:02:10,005 INFO repeater task os150nb_10pings starting 4.2 seconds late 2017-06-01 11:02:10,095 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 11:03:30,260 INFO repeater task posmv_hcorrstats starting 4.5 seconds late 2017-06-01 11:03:31,203 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 11:03:31,203 INFO repeater task posmv_plotposmv starting 5.4 seconds late 2017-06-01 11:03:31,803 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 11:03:36,809 INFO repeater task os150bb_lastensq starting 0.8 seconds late 2017-06-01 11:03:38,504 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 11:03:38,504 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 11:03:26 2017-06-01 11:03:38,505 INFO repeater task os150nb_lastensq starting 2.5 seconds late 2017-06-01 11:03:39,682 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 11:03:39,682 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 11:03:26 2017-06-01 11:07:30,158 INFO repeater task os150bb_10pings starting 4.4 seconds late 2017-06-01 11:07:30,334 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 11:07:30,335 INFO repeater task os150nb_10pings starting 4.6 seconds late 2017-06-01 11:07:30,427 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 11:08:30,551 INFO repeater task os150bb_quick starting 4.8 seconds late 2017-06-01 11:08:33,652 INFO repeater task os150bb_quick ended after 3.1 seconds 2017-06-01 11:08:33,652 INFO repeater task os150bb_3dayplots starting 7.9 seconds late 2017-06-01 11:08:39,878 INFO repeater task os150bb_3dayplots ended after 6.2 seconds 2017-06-01 11:08:39,879 INFO repeater task os150bb_tsstats starting 14.1 seconds late 2017-06-01 11:08:40,422 INFO repeater task os150bb_tsstats ended after 0.5 seconds 2017-06-01 11:08:40,422 INFO repeater task os150nb_lastensq starting 4.4 seconds late 2017-06-01 11:08:41,562 INFO repeater task os150nb_lastensq ended after 1.1 seconds 2017-06-01 11:08:41,563 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 11:08:26 2017-06-01 11:08:41,563 INFO repeater task os150nb_quick starting 15.8 seconds late 2017-06-01 11:08:44,566 INFO repeater task os150nb_quick ended after 3.0 seconds 2017-06-01 11:08:44,566 INFO repeater task os150nb_3dayplots starting 18.8 seconds late 2017-06-01 11:08:50,868 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 11:08:50,868 INFO repeater task os150nb_3dayplots ended after 6.3 seconds 2017-06-01 11:08:50,868 INFO repeater task os150nb_tsstats starting 25.1 seconds late 2017-06-01 11:08:51,435 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-01 11:08:51,435 INFO repeater task os150bb_beamplots starting 25.7 seconds late 2017-06-01 11:08:52,383 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 11:08:52,383 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 11:08:52,384 INFO repeater task os150bb_beamplots starting 26.6 seconds late 2017-06-01 11:08:53,288 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 11:08:53,288 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 11:08:53,288 INFO repeater task os150nb_beamplots starting 27.5 seconds late 2017-06-01 11:08:54,191 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 11:08:54,191 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 11:08:54,191 INFO repeater task os150nb_beamplots starting 28.4 seconds late 2017-06-01 11:08:55,108 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 11:08:55,108 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 11:08:55,108 INFO repeater task posmv_hcorrstats starting 29.3 seconds late 2017-06-01 11:08:55,921 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 11:08:55,921 INFO repeater task posmv_plotposmv starting 30.1 seconds late 2017-06-01 11:08:56,523 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 11:09:01,525 INFO repeater task os150bb_lastensq starting 25.5 seconds late 2017-06-01 11:09:03,208 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 11:09:03,209 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 11:08:26 2017-06-01 11:12:48,623 INFO repeater task os150bb_10pings starting 2.8 seconds late 2017-06-01 11:12:48,716 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 11:12:48,717 INFO repeater task os150nb_10pings starting 2.9 seconds late 2017-06-01 11:12:48,808 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 11:13:28,862 INFO repeater task posmv_hcorrstats starting 3.1 seconds late 2017-06-01 11:13:29,729 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 11:13:29,730 INFO repeater task posmv_plotposmv starting 3.9 seconds late 2017-06-01 11:13:30,340 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 11:13:40,358 INFO repeater task os150bb_lastensq starting 4.4 seconds late 2017-06-01 11:13:42,164 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 11:13:42,164 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 11:13:26 2017-06-01 11:13:42,165 INFO repeater task os150nb_lastensq starting 6.2 seconds late 2017-06-01 11:13:43,434 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-01 11:13:43,434 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 11:13:26 2017-06-01 11:18:09,036 INFO repeater task os150bb_10pings starting 3.3 seconds late 2017-06-01 11:18:09,204 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 11:18:09,205 INFO repeater task os150nb_10pings starting 3.4 seconds late 2017-06-01 11:18:09,295 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 11:18:29,341 INFO repeater task posmv_hcorrstats starting 3.6 seconds late 2017-06-01 11:18:30,292 INFO repeater task posmv_hcorrstats ended after 1.0 seconds 2017-06-01 11:18:30,292 INFO repeater task posmv_plotposmv starting 4.5 seconds late 2017-06-01 11:18:30,904 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 11:18:40,922 INFO repeater task os150bb_lastensq starting 4.9 seconds late 2017-06-01 11:18:42,563 INFO repeater task os150bb_lastensq ended after 1.6 seconds 2017-06-01 11:18:42,563 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 11:18:26 2017-06-01 11:18:42,565 INFO repeater task os150nb_lastensq starting 6.6 seconds late 2017-06-01 11:18:43,725 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 11:18:43,725 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 11:18:26 2017-06-01 11:23:29,330 INFO repeater task os150bb_10pings starting 3.6 seconds late 2017-06-01 11:23:29,479 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 11:23:29,479 INFO repeater task os150bb_quick starting 3.7 seconds late 2017-06-01 11:23:32,700 INFO repeater task os150bb_quick ended after 3.2 seconds 2017-06-01 11:23:32,700 INFO repeater task os150bb_3dayplots starting 6.9 seconds late 2017-06-01 11:23:38,944 INFO repeater task os150bb_3dayplots ended after 6.2 seconds 2017-06-01 11:23:38,944 INFO repeater task os150bb_tsstats starting 13.2 seconds late 2017-06-01 11:23:39,500 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-01 11:23:39,500 INFO repeater task os150nb_lastensq starting 3.5 seconds late 2017-06-01 11:23:40,745 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 11:23:40,745 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 11:23:26 2017-06-01 11:23:40,745 INFO repeater task os150nb_10pings starting 15.0 seconds late 2017-06-01 11:23:40,839 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 11:23:40,839 INFO repeater task os150nb_quick starting 15.1 seconds late 2017-06-01 11:23:44,100 INFO repeater task os150nb_quick ended after 3.3 seconds 2017-06-01 11:23:44,100 INFO repeater task os150nb_3dayplots starting 18.3 seconds late 2017-06-01 11:23:50,345 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 11:23:50,345 INFO repeater task os150nb_3dayplots ended after 6.2 seconds 2017-06-01 11:23:50,345 INFO repeater task os150nb_tsstats starting 24.6 seconds late 2017-06-01 11:23:50,896 INFO repeater task os150nb_tsstats output: /usr/lib/python2.7/dist-packages/numpy/ma/core.py:1073: RuntimeWarning: overflow encountered in true_divide result = self.f(da, db, *args, **kwargs) 2017-06-01 11:23:50,897 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-01 11:23:50,897 INFO repeater task os150bb_beamplots starting 25.1 seconds late 2017-06-01 11:23:51,820 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 11:23:51,820 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 11:23:51,820 INFO repeater task os150bb_beamplots starting 26.0 seconds late 2017-06-01 11:23:52,806 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 11:23:52,806 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-01 11:23:52,806 INFO repeater task os150nb_beamplots starting 27.0 seconds late 2017-06-01 11:23:53,796 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 11:23:53,797 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-01 11:23:53,797 INFO repeater task os150nb_beamplots starting 28.0 seconds late 2017-06-01 11:23:54,745 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 11:23:54,745 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 11:23:54,745 INFO repeater task posmv_hcorrstats starting 29.0 seconds late 2017-06-01 11:23:55,587 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 11:23:55,587 INFO repeater task posmv_plotposmv starting 29.8 seconds late 2017-06-01 11:23:56,210 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 11:24:01,213 INFO repeater task os150bb_lastensq starting 25.2 seconds late 2017-06-01 11:24:02,964 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 11:24:02,964 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 11:23:26 2017-06-01 11:28:28,455 INFO repeater task posmv_hcorrstats starting 2.7 seconds late 2017-06-01 11:28:29,274 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 11:28:29,274 INFO repeater task posmv_plotposmv starting 3.5 seconds late 2017-06-01 11:28:29,883 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 11:28:39,896 INFO repeater task os150bb_lastensq starting 3.9 seconds late 2017-06-01 11:28:41,608 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 11:28:41,608 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 11:28:26 2017-06-01 11:28:41,609 INFO repeater task os150nb_lastensq starting 5.6 seconds late 2017-06-01 11:28:42,786 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 11:28:42,786 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 11:28:26 2017-06-01 11:28:47,794 INFO repeater task os150bb_10pings starting 2.0 seconds late 2017-06-01 11:28:47,960 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 11:28:47,961 INFO repeater task os150nb_10pings starting 2.2 seconds late 2017-06-01 11:28:48,053 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 11:33:28,710 INFO repeater task posmv_hcorrstats starting 2.9 seconds late 2017-06-01 11:33:29,533 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 11:33:29,534 INFO repeater task posmv_plotposmv starting 3.8 seconds late 2017-06-01 11:33:30,138 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 11:33:40,151 INFO repeater task os150bb_lastensq starting 4.2 seconds late 2017-06-01 11:33:41,838 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 11:33:41,838 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 11:33:26 2017-06-01 11:33:41,839 INFO repeater task os150nb_lastensq starting 5.8 seconds late 2017-06-01 11:33:42,995 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 11:33:42,995 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 11:33:26 2017-06-01 11:34:08,046 INFO repeater task os150bb_10pings starting 2.3 seconds late 2017-06-01 11:34:08,208 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 11:34:08,209 INFO repeater task os150nb_10pings starting 2.4 seconds late 2017-06-01 11:34:08,299 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 11:38:28,881 INFO repeater task os150bb_quick starting 3.1 seconds late 2017-06-01 11:38:32,065 INFO repeater task os150bb_quick ended after 3.2 seconds 2017-06-01 11:38:32,066 INFO repeater task os150bb_3dayplots starting 6.3 seconds late 2017-06-01 11:38:38,340 INFO repeater task os150bb_3dayplots ended after 6.3 seconds 2017-06-01 11:38:38,341 INFO repeater task os150bb_tsstats starting 12.6 seconds late 2017-06-01 11:38:38,881 INFO repeater task os150bb_tsstats ended after 0.5 seconds 2017-06-01 11:38:38,882 INFO repeater task os150nb_lastensq starting 2.9 seconds late 2017-06-01 11:38:40,029 INFO repeater task os150nb_lastensq ended after 1.1 seconds 2017-06-01 11:38:40,029 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 11:38:26 2017-06-01 11:38:40,030 INFO repeater task os150nb_quick starting 14.2 seconds late 2017-06-01 11:38:43,051 INFO repeater task os150nb_quick ended after 3.0 seconds 2017-06-01 11:38:43,051 INFO repeater task os150nb_3dayplots starting 17.3 seconds late 2017-06-01 11:38:49,377 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 11:38:49,378 INFO repeater task os150nb_3dayplots ended after 6.3 seconds 2017-06-01 11:38:49,378 INFO repeater task os150nb_tsstats starting 23.6 seconds late 2017-06-01 11:38:49,916 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2017-06-01 11:38:49,916 INFO repeater task os150bb_beamplots starting 24.1 seconds late 2017-06-01 11:38:50,802 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 11:38:50,803 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 11:38:50,803 INFO repeater task os150bb_beamplots starting 25.0 seconds late 2017-06-01 11:38:51,730 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 11:38:51,730 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 11:38:51,730 INFO repeater task os150nb_beamplots starting 25.9 seconds late 2017-06-01 11:38:52,629 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 11:38:52,629 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 11:38:52,630 INFO repeater task os150nb_beamplots starting 26.8 seconds late 2017-06-01 11:38:53,554 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 11:38:53,555 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 11:38:53,555 INFO repeater task posmv_hcorrstats starting 27.8 seconds late 2017-06-01 11:38:54,395 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 11:38:54,395 INFO repeater task posmv_plotposmv starting 28.6 seconds late 2017-06-01 11:38:54,999 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 11:39:00,005 INFO repeater task os150bb_lastensq starting 24.0 seconds late 2017-06-01 11:39:01,745 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 11:39:01,745 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 11:38:26 2017-06-01 11:39:26,792 INFO repeater task os150bb_10pings starting 1.0 seconds late 2017-06-01 11:39:26,950 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 11:39:26,951 INFO repeater task os150nb_10pings starting 1.2 seconds late 2017-06-01 11:39:27,044 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 11:43:27,476 INFO repeater task posmv_hcorrstats starting 1.7 seconds late 2017-06-01 11:43:28,329 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 11:43:28,330 INFO repeater task posmv_plotposmv starting 2.5 seconds late 2017-06-01 11:43:28,950 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 11:43:38,959 INFO repeater task os150bb_lastensq starting 3.0 seconds late 2017-06-01 11:43:40,723 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 11:43:40,723 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 11:43:26 2017-06-01 11:43:40,724 INFO repeater task os150nb_lastensq starting 4.7 seconds late 2017-06-01 11:43:41,898 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 11:43:41,898 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 11:43:26 2017-06-01 11:44:47,024 INFO repeater task os150bb_10pings starting 1.2 seconds late 2017-06-01 11:44:47,171 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 11:44:47,171 INFO repeater task os150nb_10pings starting 1.4 seconds late 2017-06-01 11:44:47,265 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 11:48:27,767 INFO repeater task posmv_hcorrstats starting 2.0 seconds late 2017-06-01 11:48:28,681 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 11:48:28,681 INFO repeater task posmv_plotposmv starting 2.9 seconds late 2017-06-01 11:48:29,318 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 11:48:39,328 INFO repeater task os150bb_lastensq starting 3.3 seconds late 2017-06-01 11:48:40,974 INFO repeater task os150bb_lastensq ended after 1.6 seconds 2017-06-01 11:48:40,975 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 11:48:26 2017-06-01 11:48:40,975 INFO repeater task os150nb_lastensq starting 5.0 seconds late 2017-06-01 11:48:42,148 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 11:48:42,148 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 11:48:26 2017-06-01 11:50:07,323 INFO repeater task os150bb_10pings starting 1.5 seconds late 2017-06-01 11:50:07,472 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 11:50:07,473 INFO repeater task os150nb_10pings starting 1.7 seconds late 2017-06-01 11:50:07,567 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 11:53:28,024 INFO repeater task os150bb_quick starting 2.2 seconds late 2017-06-01 11:53:31,131 INFO repeater task os150bb_quick ended after 3.1 seconds 2017-06-01 11:53:31,131 INFO repeater task os150bb_3dayplots starting 5.4 seconds late 2017-06-01 11:53:37,363 INFO repeater task os150bb_3dayplots ended after 6.2 seconds 2017-06-01 11:53:37,363 INFO repeater task os150bb_tsstats starting 11.6 seconds late 2017-06-01 11:53:37,903 INFO repeater task os150bb_tsstats ended after 0.5 seconds 2017-06-01 11:53:37,903 INFO repeater task os150nb_lastensq starting 1.9 seconds late 2017-06-01 11:53:39,062 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 11:53:39,062 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 11:53:26 2017-06-01 11:53:39,063 INFO repeater task os150nb_quick starting 13.3 seconds late 2017-06-01 11:53:42,096 INFO repeater task os150nb_quick ended after 3.0 seconds 2017-06-01 11:53:42,097 INFO repeater task os150nb_3dayplots starting 16.3 seconds late 2017-06-01 11:53:48,365 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 11:53:48,366 INFO repeater task os150nb_3dayplots ended after 6.3 seconds 2017-06-01 11:53:48,366 INFO repeater task os150nb_tsstats starting 22.6 seconds late 2017-06-01 11:53:48,905 INFO repeater task os150nb_tsstats output: /usr/lib/python2.7/dist-packages/numpy/ma/core.py:1073: RuntimeWarning: overflow encountered in true_divide result = self.f(da, db, *args, **kwargs) 2017-06-01 11:53:48,906 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2017-06-01 11:53:48,906 INFO repeater task os150bb_beamplots starting 23.1 seconds late 2017-06-01 11:53:49,813 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 11:53:49,813 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 11:53:49,813 INFO repeater task os150bb_beamplots starting 24.0 seconds late 2017-06-01 11:53:50,753 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 11:53:50,754 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 11:53:50,754 INFO repeater task os150nb_beamplots starting 25.0 seconds late 2017-06-01 11:53:51,647 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 11:53:51,647 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 11:53:51,648 INFO repeater task os150nb_beamplots starting 25.9 seconds late 2017-06-01 11:53:52,577 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 11:53:52,578 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 11:53:52,578 INFO repeater task posmv_hcorrstats starting 26.8 seconds late 2017-06-01 11:53:53,420 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 11:53:53,420 INFO repeater task posmv_plotposmv starting 27.6 seconds late 2017-06-01 11:53:54,026 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 11:53:59,032 INFO repeater task os150bb_lastensq starting 23.0 seconds late 2017-06-01 11:54:00,703 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 11:54:00,704 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 11:53:26 2017-06-01 11:55:25,819 INFO repeater task os150bb_10pings starting 0.0 seconds late 2017-06-01 11:55:25,919 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 11:55:25,920 INFO repeater task os150nb_10pings starting 0.1 seconds late 2017-06-01 11:55:26,012 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 11:58:26,297 INFO repeater task posmv_hcorrstats starting 0.5 seconds late 2017-06-01 11:58:27,193 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 11:58:27,193 INFO repeater task posmv_plotposmv starting 1.4 seconds late 2017-06-01 11:58:27,852 INFO repeater task posmv_plotposmv ended after 0.7 seconds 2017-06-01 11:58:37,867 INFO repeater task os150bb_lastensq starting 1.9 seconds late 2017-06-01 11:58:39,583 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 11:58:39,583 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 11:58:26 2017-06-01 11:58:39,584 INFO repeater task os150nb_lastensq starting 3.6 seconds late 2017-06-01 11:58:40,827 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 11:58:40,828 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 11:58:26 2017-06-01 12:00:46,073 INFO repeater task os150bb_10pings starting 0.3 seconds late 2017-06-01 12:00:46,243 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 12:00:46,243 INFO repeater task os150nb_10pings starting 0.5 seconds late 2017-06-01 12:00:46,337 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 12:03:26,681 INFO repeater task posmv_hcorrstats starting 0.9 seconds late 2017-06-01 12:03:27,667 INFO repeater task posmv_hcorrstats ended after 1.0 seconds 2017-06-01 12:03:27,668 INFO repeater task posmv_plotposmv starting 1.9 seconds late 2017-06-01 12:03:28,300 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 12:03:38,312 INFO repeater task os150bb_lastensq starting 2.3 seconds late 2017-06-01 12:03:40,162 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 12:03:40,162 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 12:03:26 2017-06-01 12:03:40,163 INFO repeater task os150nb_lastensq starting 4.2 seconds late 2017-06-01 12:03:41,422 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-01 12:03:41,423 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 12:03:26 2017-06-01 12:06:06,734 INFO repeater task os150bb_10pings starting 1.0 seconds late 2017-06-01 12:06:06,901 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 12:06:06,902 INFO repeater task os150nb_10pings starting 1.1 seconds late 2017-06-01 12:06:06,999 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 12:08:27,325 INFO repeater task os150bb_quick starting 1.5 seconds late 2017-06-01 12:08:30,526 INFO repeater task os150bb_quick ended after 3.2 seconds 2017-06-01 12:08:30,527 INFO repeater task os150bb_3dayplots starting 4.7 seconds late 2017-06-01 12:08:37,286 INFO repeater task os150bb_3dayplots ended after 6.8 seconds 2017-06-01 12:08:37,287 INFO repeater task os150bb_tsstats starting 11.5 seconds late 2017-06-01 12:08:37,870 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-01 12:08:37,870 INFO repeater task os150nb_lastensq starting 1.9 seconds late 2017-06-01 12:08:39,078 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 12:08:39,078 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 12:08:26 2017-06-01 12:08:39,078 INFO repeater task os150nb_quick starting 13.3 seconds late 2017-06-01 12:08:42,179 INFO repeater task os150nb_quick ended after 3.1 seconds 2017-06-01 12:08:42,179 INFO repeater task os150nb_3dayplots starting 16.4 seconds late 2017-06-01 12:08:49,099 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 12:08:49,099 INFO repeater task os150nb_3dayplots ended after 6.9 seconds 2017-06-01 12:08:49,099 INFO repeater task os150nb_tsstats starting 23.3 seconds late 2017-06-01 12:08:49,691 INFO repeater task os150nb_tsstats output: /usr/lib/python2.7/dist-packages/numpy/ma/core.py:1073: RuntimeWarning: overflow encountered in true_divide result = self.f(da, db, *args, **kwargs) 2017-06-01 12:08:49,692 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-01 12:08:49,692 INFO repeater task os150bb_beamplots starting 23.9 seconds late 2017-06-01 12:08:50,715 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 12:08:50,716 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-01 12:08:50,716 INFO repeater task os150bb_beamplots starting 24.9 seconds late 2017-06-01 12:08:51,669 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 12:08:51,669 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-01 12:08:51,669 INFO repeater task os150nb_beamplots starting 25.9 seconds late 2017-06-01 12:08:52,567 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 12:08:52,567 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 12:08:52,567 INFO repeater task os150nb_beamplots starting 26.8 seconds late 2017-06-01 12:08:53,512 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 12:08:53,512 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 12:08:53,512 INFO repeater task posmv_hcorrstats starting 27.7 seconds late 2017-06-01 12:08:54,420 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 12:08:54,420 INFO repeater task posmv_plotposmv starting 28.6 seconds late 2017-06-01 12:08:55,075 INFO repeater task posmv_plotposmv ended after 0.7 seconds 2017-06-01 12:09:00,081 INFO repeater task os150bb_lastensq starting 24.1 seconds late 2017-06-01 12:09:01,831 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 12:09:01,831 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 12:08:26 2017-06-01 12:11:27,113 INFO repeater task os150bb_10pings starting 1.3 seconds late 2017-06-01 12:11:27,273 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 12:11:27,274 INFO repeater task os150nb_10pings starting 1.5 seconds late 2017-06-01 12:11:27,371 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 12:13:27,635 INFO repeater task posmv_hcorrstats starting 1.9 seconds late 2017-06-01 12:13:28,577 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 12:13:28,577 INFO repeater task posmv_plotposmv starting 2.8 seconds late 2017-06-01 12:13:29,195 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 12:13:39,213 INFO repeater task os150bb_lastensq starting 3.2 seconds late 2017-06-01 12:13:40,943 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 12:13:40,943 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 12:13:26 2017-06-01 12:13:40,944 INFO repeater task os150nb_lastensq starting 4.9 seconds late 2017-06-01 12:13:42,216 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-01 12:13:42,216 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 12:13:26 2017-06-01 12:16:47,558 INFO repeater task os150bb_10pings starting 1.8 seconds late 2017-06-01 12:16:47,661 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 12:16:47,662 INFO repeater task os150nb_10pings starting 1.9 seconds late 2017-06-01 12:16:47,758 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 12:18:27,930 INFO repeater task posmv_hcorrstats starting 2.1 seconds late 2017-06-01 12:18:28,830 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 12:18:28,831 INFO repeater task posmv_plotposmv starting 3.0 seconds late 2017-06-01 12:18:29,465 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 12:18:39,482 INFO repeater task os150bb_lastensq starting 3.5 seconds late 2017-06-01 12:18:41,200 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 12:18:41,200 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 12:18:26 2017-06-01 12:18:41,201 INFO repeater task os150nb_lastensq starting 5.2 seconds late 2017-06-01 12:18:42,434 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 12:18:42,434 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 12:18:26 2017-06-01 12:22:07,869 INFO repeater task os150bb_10pings starting 2.1 seconds late 2017-06-01 12:22:08,012 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 12:22:08,013 INFO repeater task os150nb_10pings starting 2.2 seconds late 2017-06-01 12:22:08,103 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 12:23:28,288 INFO repeater task os150bb_quick starting 2.5 seconds late 2017-06-01 12:23:31,414 INFO repeater task os150bb_quick ended after 3.1 seconds 2017-06-01 12:23:31,415 INFO repeater task os150bb_3dayplots starting 5.6 seconds late 2017-06-01 12:23:37,646 INFO repeater task os150bb_3dayplots ended after 6.2 seconds 2017-06-01 12:23:37,646 INFO repeater task os150bb_tsstats starting 11.9 seconds late 2017-06-01 12:23:38,176 INFO repeater task os150bb_tsstats ended after 0.5 seconds 2017-06-01 12:23:38,176 INFO repeater task os150nb_lastensq starting 2.2 seconds late 2017-06-01 12:23:39,340 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 12:23:39,340 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 12:23:26 2017-06-01 12:23:39,340 INFO repeater task os150nb_quick starting 13.6 seconds late 2017-06-01 12:23:42,414 INFO repeater task os150nb_quick ended after 3.1 seconds 2017-06-01 12:23:42,415 INFO repeater task os150nb_3dayplots starting 16.6 seconds late 2017-06-01 12:23:48,721 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 12:23:48,721 INFO repeater task os150nb_3dayplots ended after 6.3 seconds 2017-06-01 12:23:48,722 INFO repeater task os150nb_tsstats starting 22.9 seconds late 2017-06-01 12:23:49,252 INFO repeater task os150nb_tsstats output: /usr/lib/python2.7/dist-packages/numpy/ma/core.py:1073: RuntimeWarning: overflow encountered in true_divide result = self.f(da, db, *args, **kwargs) 2017-06-01 12:23:49,252 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2017-06-01 12:23:49,252 INFO repeater task os150bb_beamplots starting 23.5 seconds late 2017-06-01 12:23:50,172 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 12:23:50,173 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 12:23:50,173 INFO repeater task os150bb_beamplots starting 24.4 seconds late 2017-06-01 12:23:51,092 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 12:23:51,092 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 12:23:51,092 INFO repeater task os150nb_beamplots starting 25.3 seconds late 2017-06-01 12:23:51,993 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 12:23:51,993 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 12:23:51,993 INFO repeater task os150nb_beamplots starting 26.2 seconds late 2017-06-01 12:23:52,940 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 12:23:52,940 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 12:23:52,941 INFO repeater task posmv_hcorrstats starting 27.2 seconds late 2017-06-01 12:23:53,787 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 12:23:53,788 INFO repeater task posmv_plotposmv starting 28.0 seconds late 2017-06-01 12:23:54,409 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 12:23:59,415 INFO repeater task os150bb_lastensq starting 23.4 seconds late 2017-06-01 12:24:01,024 INFO repeater task os150bb_lastensq ended after 1.6 seconds 2017-06-01 12:24:01,024 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 12:23:26 2017-06-01 12:27:26,444 INFO repeater task os150bb_10pings starting 0.7 seconds late 2017-06-01 12:27:26,607 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 12:27:26,608 INFO repeater task os150nb_10pings starting 0.8 seconds late 2017-06-01 12:27:26,701 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 12:28:26,817 INFO repeater task posmv_hcorrstats starting 1.0 seconds late 2017-06-01 12:28:27,701 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 12:28:27,702 INFO repeater task posmv_plotposmv starting 1.9 seconds late 2017-06-01 12:28:28,297 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 12:28:38,315 INFO repeater task os150bb_lastensq starting 2.3 seconds late 2017-06-01 12:28:40,030 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 12:28:40,030 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 12:28:26 2017-06-01 12:28:40,031 INFO repeater task os150nb_lastensq starting 4.0 seconds late 2017-06-01 12:28:41,221 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 12:28:41,221 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 12:28:26 2017-06-01 12:32:46,661 INFO repeater task os150bb_10pings starting 0.9 seconds late 2017-06-01 12:32:46,814 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 12:32:46,814 INFO repeater task os150nb_10pings starting 1.0 seconds late 2017-06-01 12:32:46,913 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 12:33:27,002 INFO repeater task posmv_hcorrstats starting 1.2 seconds late 2017-06-01 12:33:27,957 INFO repeater task posmv_hcorrstats ended after 1.0 seconds 2017-06-01 12:33:27,957 INFO repeater task posmv_plotposmv starting 2.2 seconds late 2017-06-01 12:33:28,581 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 12:33:38,598 INFO repeater task os150bb_lastensq starting 2.6 seconds late 2017-06-01 12:33:40,400 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 12:33:40,400 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 12:33:26 2017-06-01 12:33:40,402 INFO repeater task os150nb_lastensq starting 4.4 seconds late 2017-06-01 12:33:41,602 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 12:33:41,603 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 12:33:26 2017-06-01 12:38:07,160 INFO repeater task os150bb_10pings starting 1.4 seconds late 2017-06-01 12:38:07,328 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 12:38:07,329 INFO repeater task os150nb_10pings starting 1.5 seconds late 2017-06-01 12:38:07,424 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 12:38:27,462 INFO repeater task os150bb_quick starting 1.7 seconds late 2017-06-01 12:38:30,780 INFO repeater task os150bb_quick ended after 3.3 seconds 2017-06-01 12:38:30,780 INFO repeater task os150bb_3dayplots starting 5.0 seconds late 2017-06-01 12:38:37,699 INFO repeater task os150bb_3dayplots ended after 6.9 seconds 2017-06-01 12:38:37,699 INFO repeater task os150bb_tsstats starting 11.9 seconds late 2017-06-01 12:38:38,253 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-01 12:38:38,253 INFO repeater task os150nb_lastensq starting 2.3 seconds late 2017-06-01 12:38:39,491 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 12:38:39,491 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 12:38:26 2017-06-01 12:38:39,491 INFO repeater task os150nb_quick starting 13.7 seconds late 2017-06-01 12:38:42,683 INFO repeater task os150nb_quick ended after 3.2 seconds 2017-06-01 12:38:42,684 INFO repeater task os150nb_3dayplots starting 16.9 seconds late 2017-06-01 12:38:49,317 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 12:38:49,317 INFO repeater task os150nb_3dayplots ended after 6.6 seconds 2017-06-01 12:38:49,317 INFO repeater task os150nb_tsstats starting 23.5 seconds late 2017-06-01 12:38:49,874 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-01 12:38:49,874 INFO repeater task os150bb_beamplots starting 24.1 seconds late 2017-06-01 12:38:50,833 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 12:38:50,833 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-01 12:38:50,833 INFO repeater task os150bb_beamplots starting 25.1 seconds late 2017-06-01 12:38:51,815 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 12:38:51,815 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-01 12:38:51,816 INFO repeater task os150nb_beamplots starting 26.0 seconds late 2017-06-01 12:38:52,778 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 12:38:52,778 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-01 12:38:52,778 INFO repeater task os150nb_beamplots starting 27.0 seconds late 2017-06-01 12:38:53,785 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 12:38:53,786 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-01 12:38:53,786 INFO repeater task posmv_hcorrstats starting 28.0 seconds late 2017-06-01 12:38:54,675 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 12:38:54,676 INFO repeater task posmv_plotposmv starting 28.9 seconds late 2017-06-01 12:38:55,285 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 12:39:00,291 INFO repeater task os150bb_lastensq starting 24.3 seconds late 2017-06-01 12:39:02,062 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 12:39:02,062 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 12:38:26 2017-06-01 12:43:27,639 INFO repeater task os150bb_10pings starting 1.9 seconds late 2017-06-01 12:43:27,809 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 12:43:27,810 INFO repeater task os150nb_10pings starting 2.0 seconds late 2017-06-01 12:43:27,902 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 12:43:27,904 INFO repeater task posmv_hcorrstats starting 2.1 seconds late 2017-06-01 12:43:28,743 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 12:43:28,743 INFO repeater task posmv_plotposmv starting 3.0 seconds late 2017-06-01 12:43:29,346 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 12:43:39,358 INFO repeater task os150bb_lastensq starting 3.4 seconds late 2017-06-01 12:43:41,077 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 12:43:41,077 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 12:43:26 2017-06-01 12:43:41,078 INFO repeater task os150nb_lastensq starting 5.1 seconds late 2017-06-01 12:43:42,226 INFO repeater task os150nb_lastensq ended after 1.1 seconds 2017-06-01 12:43:42,226 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 12:43:26 2017-06-01 12:48:27,845 INFO repeater task posmv_hcorrstats starting 2.1 seconds late 2017-06-01 12:48:28,822 INFO repeater task posmv_hcorrstats ended after 1.0 seconds 2017-06-01 12:48:28,822 INFO repeater task posmv_plotposmv starting 3.0 seconds late 2017-06-01 12:48:29,437 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 12:48:39,451 INFO repeater task os150bb_lastensq starting 3.5 seconds late 2017-06-01 12:48:41,172 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 12:48:41,172 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 12:48:26 2017-06-01 12:48:41,173 INFO repeater task os150nb_lastensq starting 5.2 seconds late 2017-06-01 12:48:42,366 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 12:48:42,367 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 12:48:26 2017-06-01 12:48:47,375 INFO repeater task os150bb_10pings starting 1.6 seconds late 2017-06-01 12:48:47,528 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 12:48:47,529 INFO repeater task os150nb_10pings starting 1.7 seconds late 2017-06-01 12:48:47,622 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 12:53:28,079 INFO repeater task os150bb_quick starting 2.3 seconds late 2017-06-01 12:53:31,217 INFO repeater task os150bb_quick ended after 3.1 seconds 2017-06-01 12:53:31,217 INFO repeater task os150bb_3dayplots starting 5.4 seconds late 2017-06-01 12:53:37,495 INFO repeater task os150bb_3dayplots ended after 6.3 seconds 2017-06-01 12:53:37,495 INFO repeater task os150bb_tsstats starting 11.7 seconds late 2017-06-01 12:53:38,033 INFO repeater task os150bb_tsstats ended after 0.5 seconds 2017-06-01 12:53:38,033 INFO repeater task os150nb_lastensq starting 2.0 seconds late 2017-06-01 12:53:39,167 INFO repeater task os150nb_lastensq ended after 1.1 seconds 2017-06-01 12:53:39,167 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 12:53:26 2017-06-01 12:53:39,167 INFO repeater task os150nb_quick starting 13.4 seconds late 2017-06-01 12:53:42,236 INFO repeater task os150nb_quick ended after 3.1 seconds 2017-06-01 12:53:42,237 INFO repeater task os150nb_3dayplots starting 16.5 seconds late 2017-06-01 12:53:48,636 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 12:53:48,636 INFO repeater task os150nb_3dayplots ended after 6.4 seconds 2017-06-01 12:53:48,637 INFO repeater task os150nb_tsstats starting 22.9 seconds late 2017-06-01 12:53:49,175 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2017-06-01 12:53:49,175 INFO repeater task os150bb_beamplots starting 23.4 seconds late 2017-06-01 12:53:50,095 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin