2017-06-01 23:34:09,359 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 23:38:29,894 INFO repeater task os150bb_quick starting 4.1 seconds late 2017-06-01 23:38:33,026 INFO repeater task os150bb_quick ended after 3.1 seconds 2017-06-01 23:38:33,026 INFO repeater task os150bb_3dayplots starting 7.2 seconds late 2017-06-01 23:38:39,297 INFO repeater task os150bb_3dayplots ended after 6.3 seconds 2017-06-01 23:38:39,297 INFO repeater task os150bb_tsstats starting 13.5 seconds late 2017-06-01 23:38:39,841 INFO repeater task os150bb_tsstats ended after 0.5 seconds 2017-06-01 23:38:39,841 INFO repeater task os150nb_lastensq starting 3.8 seconds late 2017-06-01 23:38:41,017 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 23:38:41,017 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 23:38:26 2017-06-01 23:38:41,017 INFO repeater task os150nb_quick starting 15.2 seconds late 2017-06-01 23:38:44,103 INFO repeater task os150nb_quick ended after 3.1 seconds 2017-06-01 23:38:44,103 INFO repeater task os150nb_3dayplots starting 18.3 seconds late 2017-06-01 23:38:50,495 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 23:38:50,495 INFO repeater task os150nb_3dayplots ended after 6.4 seconds 2017-06-01 23:38:50,495 INFO repeater task os150nb_tsstats starting 24.7 seconds late 2017-06-01 23:38:51,032 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2017-06-01 23:38:51,032 INFO repeater task os150bb_beamplots starting 25.3 seconds late 2017-06-01 23:38:51,947 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 23:38:51,947 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 23:38:51,947 INFO repeater task os150bb_beamplots starting 26.2 seconds late 2017-06-01 23:38:52,866 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 23:38:52,866 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 23:38:52,866 INFO repeater task os150nb_beamplots starting 27.1 seconds late 2017-06-01 23:38:53,766 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 23:38:53,766 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 23:38:53,767 INFO repeater task os150nb_beamplots starting 28.0 seconds late 2017-06-01 23:38:54,693 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 23:38:54,693 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 23:38:54,693 INFO repeater task posmv_hcorrstats starting 28.9 seconds late 2017-06-01 23:38:55,542 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 23:38:55,542 INFO repeater task posmv_plotposmv starting 29.8 seconds late 2017-06-01 23:38:56,162 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 23:39:01,168 INFO repeater task os150bb_lastensq starting 25.2 seconds late 2017-06-01 23:39:02,930 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 23:39:02,930 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 23:38:26 2017-06-01 23:39:28,127 INFO repeater task os150bb_10pings starting 2.3 seconds late 2017-06-01 23:39:28,220 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 23:39:28,221 INFO repeater task os150nb_10pings starting 2.4 seconds late 2017-06-01 23:39:28,312 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 23:43:28,808 INFO repeater task posmv_hcorrstats starting 3.0 seconds late 2017-06-01 23:43:29,743 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-01 23:43:29,744 INFO repeater task posmv_plotposmv starting 4.0 seconds late 2017-06-01 23:43:30,355 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 23:43:40,367 INFO repeater task os150bb_lastensq starting 4.4 seconds late 2017-06-01 23:43:42,097 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 23:43:42,097 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 23:43:26 2017-06-01 23:43:42,098 INFO repeater task os150nb_lastensq starting 6.1 seconds late 2017-06-01 23:43:43,250 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 23:43:43,250 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 23:43:26 2017-06-01 23:44:48,377 INFO repeater task os150bb_10pings starting 2.6 seconds late 2017-06-01 23:44:48,551 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 23:44:48,552 INFO repeater task os150nb_10pings starting 2.8 seconds late 2017-06-01 23:44:48,645 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 23:48:29,154 INFO repeater task posmv_hcorrstats starting 3.4 seconds late 2017-06-01 23:48:29,986 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 23:48:29,986 INFO repeater task posmv_plotposmv starting 4.2 seconds late 2017-06-01 23:48:30,589 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 23:48:40,605 INFO repeater task os150bb_lastensq starting 4.6 seconds late 2017-06-01 23:48:42,221 INFO repeater task os150bb_lastensq ended after 1.6 seconds 2017-06-01 23:48:42,221 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 23:48:26 2017-06-01 23:48:42,222 INFO repeater task os150nb_lastensq starting 6.2 seconds late 2017-06-01 23:48:43,379 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 23:48:43,380 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 23:48:26 2017-06-01 23:50:08,547 INFO repeater task os150bb_10pings starting 2.8 seconds late 2017-06-01 23:50:08,683 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-01 23:50:08,684 INFO repeater task os150nb_10pings starting 2.9 seconds late 2017-06-01 23:50:08,774 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 23:53:29,234 INFO repeater task os150bb_quick starting 3.5 seconds late 2017-06-01 23:53:32,261 INFO repeater task os150bb_quick ended after 3.0 seconds 2017-06-01 23:53:32,261 INFO repeater task os150bb_3dayplots starting 6.5 seconds late 2017-06-01 23:53:38,576 INFO repeater task os150bb_3dayplots ended after 6.3 seconds 2017-06-01 23:53:38,576 INFO repeater task os150bb_tsstats starting 12.8 seconds late 2017-06-01 23:53:39,114 INFO repeater task os150bb_tsstats ended after 0.5 seconds 2017-06-01 23:53:39,115 INFO repeater task os150nb_lastensq starting 3.1 seconds late 2017-06-01 23:53:40,301 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 23:53:40,302 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 23:53:26 2017-06-01 23:53:40,302 INFO repeater task os150nb_quick starting 14.5 seconds late 2017-06-01 23:53:43,388 INFO repeater task os150nb_quick ended after 3.1 seconds 2017-06-01 23:53:43,388 INFO repeater task os150nb_3dayplots starting 17.6 seconds late 2017-06-01 23:53:49,666 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-01 23:53:49,666 INFO repeater task os150nb_3dayplots ended after 6.3 seconds 2017-06-01 23:53:49,666 INFO repeater task os150nb_tsstats starting 23.9 seconds late 2017-06-01 23:53:50,210 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2017-06-01 23:53:50,211 INFO repeater task os150bb_beamplots starting 24.4 seconds late 2017-06-01 23:53:51,115 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 23:53:51,115 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-01 23:53:51,115 INFO repeater task os150bb_beamplots starting 25.3 seconds late 2017-06-01 23:53:52,063 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 23:53:52,071 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-01 23:53:52,071 INFO repeater task os150nb_beamplots starting 26.3 seconds late 2017-06-01 23:53:52,993 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 23:53:52,993 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 23:53:52,993 INFO repeater task os150nb_beamplots starting 27.2 seconds late 2017-06-01 23:53:53,906 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-01 23:53:53,906 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-01 23:53:53,906 INFO repeater task posmv_hcorrstats starting 28.1 seconds late 2017-06-01 23:53:54,730 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-01 23:53:54,730 INFO repeater task posmv_plotposmv starting 28.9 seconds late 2017-06-01 23:53:55,345 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 23:54:00,351 INFO repeater task os150bb_lastensq starting 24.4 seconds late 2017-06-01 23:54:02,097 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-01 23:54:02,097 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 23:53:26 2017-06-01 23:55:27,265 INFO repeater task os150bb_10pings starting 1.5 seconds late 2017-06-01 23:55:27,436 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-01 23:55:27,437 INFO repeater task os150nb_10pings starting 1.7 seconds late 2017-06-01 23:55:27,533 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-01 23:58:27,911 INFO repeater task posmv_hcorrstats starting 2.1 seconds late 2017-06-01 23:58:28,878 INFO repeater task posmv_hcorrstats ended after 1.0 seconds 2017-06-01 23:58:28,879 INFO repeater task posmv_plotposmv starting 3.1 seconds late 2017-06-01 23:58:29,503 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-01 23:58:39,522 INFO repeater task os150bb_lastensq starting 3.5 seconds late 2017-06-01 23:58:41,357 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-01 23:58:41,357 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/01 23:58:26 2017-06-01 23:58:41,357 INFO repeater task os150nb_lastensq starting 5.4 seconds late 2017-06-01 23:58:42,565 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-01 23:58:42,565 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/01 23:58:26 2017-06-02 00:00:47,820 INFO repeater task os150bb_10pings starting 2.0 seconds late 2017-06-02 00:00:47,964 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 00:00:47,965 INFO repeater task os150nb_10pings starting 2.2 seconds late 2017-06-02 00:00:48,058 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 00:03:28,432 INFO repeater task posmv_hcorrstats starting 2.7 seconds late 2017-06-02 00:03:29,411 INFO repeater task posmv_hcorrstats ended after 1.0 seconds 2017-06-02 00:03:29,411 INFO repeater task posmv_plotposmv starting 3.6 seconds late 2017-06-02 00:03:30,011 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 00:03:40,025 INFO repeater task os150bb_lastensq starting 4.0 seconds late 2017-06-02 00:03:41,882 INFO repeater task os150bb_lastensq ended after 1.9 seconds 2017-06-02 00:03:41,883 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 00:03:26 2017-06-02 00:03:41,883 INFO repeater task os150nb_lastensq starting 5.9 seconds late 2017-06-02 00:03:43,120 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 00:03:43,120 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 00:03:26 2017-06-02 00:06:08,436 INFO repeater task os150bb_10pings starting 2.7 seconds late 2017-06-02 00:06:08,563 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 00:06:08,564 INFO repeater task os150nb_10pings starting 2.8 seconds late 2017-06-02 00:06:08,666 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 00:08:28,949 INFO repeater task os150bb_quick starting 3.2 seconds late 2017-06-02 00:08:32,071 INFO repeater task os150bb_quick ended after 3.1 seconds 2017-06-02 00:08:32,072 INFO repeater task os150bb_3dayplots starting 6.3 seconds late 2017-06-02 00:08:38,453 INFO repeater task os150bb_3dayplots output: could not parse 152_os150bb_ddaycont.png could not parse 152_os150bb_latcont.png could not parse 152_os150bb_loncont.png could not parse 152_os150bb_ddaycont.png could not parse 152_os150bb_latcont.png could not parse 152_os150bb_loncont.png 2017-06-02 00:08:38,453 INFO repeater task os150bb_3dayplots ended after 6.4 seconds 2017-06-02 00:08:38,453 INFO repeater task os150bb_tsstats starting 12.7 seconds late 2017-06-02 00:08:39,063 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-02 00:08:39,063 INFO repeater task os150nb_lastensq starting 3.1 seconds late 2017-06-02 00:08:40,274 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 00:08:40,274 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 00:08:26 2017-06-02 00:08:40,274 INFO repeater task os150nb_quick starting 14.5 seconds late 2017-06-02 00:08:43,436 INFO repeater task os150nb_quick ended after 3.2 seconds 2017-06-02 00:08:43,436 INFO repeater task os150nb_3dayplots starting 17.7 seconds late 2017-06-02 00:08:49,930 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) could not parse 152_os150nb_ddaycont.png could not parse 152_os150nb_latcont.png could not parse 152_os150nb_loncont.png could not parse 152_os150nb_ddaycont.png could not parse 152_os150nb_latcont.png could not parse 152_os150nb_loncont.png 2017-06-02 00:08:49,930 INFO repeater task os150nb_3dayplots ended after 6.5 seconds 2017-06-02 00:08:49,930 INFO repeater task os150nb_tsstats starting 24.1 seconds late 2017-06-02 00:08:50,525 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-02 00:08:50,526 INFO repeater task os150bb_beamplots starting 24.7 seconds late 2017-06-02 00:08:51,495 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 00:08:51,496 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-02 00:08:51,496 INFO repeater task os150bb_beamplots starting 25.7 seconds late 2017-06-02 00:08:52,467 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 00:08:52,468 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-02 00:08:52,468 INFO repeater task os150nb_beamplots starting 26.7 seconds late 2017-06-02 00:08:53,395 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 00:08:53,395 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 00:08:53,395 INFO repeater task os150nb_beamplots starting 27.6 seconds late 2017-06-02 00:08:54,364 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 00:08:54,364 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-02 00:08:54,364 INFO repeater task posmv_hcorrstats starting 28.6 seconds late 2017-06-02 00:08:55,187 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 00:08:55,187 INFO repeater task posmv_plotposmv starting 29.4 seconds late 2017-06-02 00:08:55,818 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 00:09:00,824 INFO repeater task os150bb_lastensq starting 24.8 seconds late 2017-06-02 00:09:02,588 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 00:09:02,588 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 00:08:26 2017-06-02 00:11:27,907 INFO repeater task os150bb_10pings starting 2.1 seconds late 2017-06-02 00:11:28,062 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 00:11:28,063 INFO repeater task os150nb_10pings starting 2.3 seconds late 2017-06-02 00:11:28,157 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 00:13:28,487 INFO repeater task posmv_hcorrstats starting 2.7 seconds late 2017-06-02 00:13:29,440 INFO repeater task posmv_hcorrstats ended after 1.0 seconds 2017-06-02 00:13:29,441 INFO repeater task posmv_plotposmv starting 3.7 seconds late 2017-06-02 00:13:30,063 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 00:13:40,077 INFO repeater task os150bb_lastensq starting 4.1 seconds late 2017-06-02 00:13:41,830 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 00:13:41,830 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 00:13:26 2017-06-02 00:13:41,831 INFO repeater task os150nb_lastensq starting 5.8 seconds late 2017-06-02 00:13:42,997 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 00:13:42,997 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 00:13:26 2017-06-02 00:16:48,357 INFO repeater task os150bb_10pings starting 2.6 seconds late 2017-06-02 00:16:48,468 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 00:16:48,469 INFO repeater task os150nb_10pings starting 2.7 seconds late 2017-06-02 00:16:48,562 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 00:18:28,768 INFO repeater task posmv_hcorrstats starting 3.0 seconds late 2017-06-02 00:18:29,727 INFO repeater task posmv_hcorrstats ended after 1.0 seconds 2017-06-02 00:18:29,728 INFO repeater task posmv_plotposmv starting 3.9 seconds late 2017-06-02 00:18:30,348 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 00:18:40,365 INFO repeater task os150bb_lastensq starting 4.4 seconds late 2017-06-02 00:18:42,189 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 00:18:42,189 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 00:18:26 2017-06-02 00:18:42,190 INFO repeater task os150nb_lastensq starting 6.2 seconds late 2017-06-02 00:18:43,451 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-02 00:18:43,451 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 00:18:26 2017-06-02 00:22:08,875 INFO repeater task os150bb_10pings starting 3.1 seconds late 2017-06-02 00:22:09,050 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 00:22:09,050 INFO repeater task os150nb_10pings starting 3.3 seconds late 2017-06-02 00:22:09,144 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 00:23:29,305 INFO repeater task os150bb_quick starting 3.5 seconds late 2017-06-02 00:23:32,523 INFO repeater task os150bb_quick ended after 3.2 seconds 2017-06-02 00:23:32,523 INFO repeater task os150bb_3dayplots starting 6.7 seconds late 2017-06-02 00:23:39,265 INFO repeater task os150bb_3dayplots ended after 6.7 seconds 2017-06-02 00:23:39,266 INFO repeater task os150bb_tsstats starting 13.5 seconds late 2017-06-02 00:23:39,839 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-02 00:23:39,839 INFO repeater task os150nb_lastensq starting 3.8 seconds late 2017-06-02 00:23:41,021 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 00:23:41,021 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 00:23:26 2017-06-02 00:23:41,022 INFO repeater task os150nb_quick starting 15.2 seconds late 2017-06-02 00:23:44,166 INFO repeater task os150nb_quick ended after 3.1 seconds 2017-06-02 00:23:44,166 INFO repeater task os150nb_3dayplots starting 18.4 seconds late 2017-06-02 00:23:51,031 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-02 00:23:51,031 INFO repeater task os150nb_3dayplots ended after 6.9 seconds 2017-06-02 00:23:51,031 INFO repeater task os150nb_tsstats starting 25.3 seconds late 2017-06-02 00:23:51,593 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-02 00:23:51,593 INFO repeater task os150bb_beamplots starting 25.8 seconds late 2017-06-02 00:23:52,508 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 00:23:52,508 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-02 00:23:52,509 INFO repeater task os150bb_beamplots starting 26.7 seconds late 2017-06-02 00:23:53,438 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 00:23:53,438 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-02 00:23:53,438 INFO repeater task os150nb_beamplots starting 27.7 seconds late 2017-06-02 00:23:54,367 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 00:23:54,367 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 00:23:54,367 INFO repeater task os150nb_beamplots starting 28.6 seconds late 2017-06-02 00:23:55,306 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 00:23:55,306 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 00:23:55,306 INFO repeater task posmv_hcorrstats starting 29.5 seconds late 2017-06-02 00:23:56,134 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 00:23:56,134 INFO repeater task posmv_plotposmv starting 30.4 seconds late 2017-06-02 00:23:56,778 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 00:24:01,784 INFO repeater task os150bb_lastensq starting 25.8 seconds late 2017-06-02 00:24:03,562 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 00:24:03,562 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 00:23:26 2017-06-02 00:27:28,925 INFO repeater task os150bb_10pings starting 3.1 seconds late 2017-06-02 00:27:29,028 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 00:27:29,029 INFO repeater task os150nb_10pings starting 3.2 seconds late 2017-06-02 00:27:29,126 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 00:28:29,215 INFO repeater task posmv_hcorrstats starting 3.4 seconds late 2017-06-02 00:28:30,031 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 00:28:30,032 INFO repeater task posmv_plotposmv starting 4.2 seconds late 2017-06-02 00:28:30,638 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 00:28:40,653 INFO repeater task os150bb_lastensq starting 4.7 seconds late 2017-06-02 00:28:42,422 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 00:28:42,422 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 00:28:26 2017-06-02 00:28:42,423 INFO repeater task os150nb_lastensq starting 6.4 seconds late 2017-06-02 00:28:43,569 INFO repeater task os150nb_lastensq ended after 1.1 seconds 2017-06-02 00:28:43,569 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 00:28:26 2017-06-02 00:32:49,093 INFO repeater task os150bb_10pings starting 3.3 seconds late 2017-06-02 00:32:49,191 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 00:32:49,192 INFO repeater task os150nb_10pings starting 3.4 seconds late 2017-06-02 00:32:49,289 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 00:33:29,360 INFO repeater task posmv_hcorrstats starting 3.6 seconds late 2017-06-02 00:33:30,227 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 00:33:30,228 INFO repeater task posmv_plotposmv starting 4.4 seconds late 2017-06-02 00:33:30,840 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 00:33:40,852 INFO repeater task os150bb_lastensq starting 4.9 seconds late 2017-06-02 00:33:42,573 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-02 00:33:42,573 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 00:33:26 2017-06-02 00:33:42,574 INFO repeater task os150nb_lastensq starting 6.6 seconds late 2017-06-02 00:33:43,731 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 00:33:43,732 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 00:33:26 2017-06-02 00:38:09,301 INFO repeater task os150bb_10pings starting 3.5 seconds late 2017-06-02 00:38:09,462 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 00:38:09,463 INFO repeater task os150nb_10pings starting 3.7 seconds late 2017-06-02 00:38:09,558 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 00:38:29,599 INFO repeater task os150bb_quick starting 3.8 seconds late 2017-06-02 00:38:32,956 INFO repeater task os150bb_quick ended after 3.4 seconds 2017-06-02 00:38:32,956 INFO repeater task os150bb_3dayplots starting 7.2 seconds late 2017-06-02 00:38:40,100 INFO repeater task os150bb_3dayplots ended after 7.1 seconds 2017-06-02 00:38:40,101 INFO repeater task os150bb_tsstats starting 14.3 seconds late 2017-06-02 00:38:40,693 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-02 00:38:40,693 INFO repeater task os150nb_lastensq starting 4.7 seconds late 2017-06-02 00:38:41,887 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 00:38:41,887 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 00:38:26 2017-06-02 00:38:41,887 INFO repeater task os150nb_quick starting 16.1 seconds late 2017-06-02 00:38:45,074 INFO repeater task os150nb_quick ended after 3.2 seconds 2017-06-02 00:38:45,074 INFO repeater task os150nb_3dayplots starting 19.3 seconds late 2017-06-02 00:38:52,032 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-02 00:38:52,033 INFO repeater task os150nb_3dayplots ended after 7.0 seconds 2017-06-02 00:38:52,033 INFO repeater task os150nb_tsstats starting 26.3 seconds late 2017-06-02 00:38:52,612 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-02 00:38:52,612 INFO repeater task os150bb_beamplots starting 26.8 seconds late 2017-06-02 00:38:53,577 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 00:38:53,577 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-02 00:38:53,577 INFO repeater task os150bb_beamplots starting 27.8 seconds late 2017-06-02 00:38:54,495 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 00:38:54,495 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-02 00:38:54,495 INFO repeater task os150nb_beamplots starting 28.7 seconds late 2017-06-02 00:38:55,420 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 00:38:55,420 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 00:38:55,421 INFO repeater task os150nb_beamplots starting 29.6 seconds late 2017-06-02 00:38:56,405 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 00:38:56,405 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-02 00:38:56,405 INFO repeater task posmv_hcorrstats starting 30.6 seconds late 2017-06-02 00:38:57,236 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 00:38:57,236 INFO repeater task posmv_plotposmv starting 31.5 seconds late 2017-06-02 00:38:57,884 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 00:39:02,889 INFO repeater task os150bb_lastensq starting 26.9 seconds late 2017-06-02 00:39:04,663 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 00:39:04,664 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 00:38:26 2017-06-02 00:43:30,213 INFO repeater task os150bb_10pings starting 4.4 seconds late 2017-06-02 00:43:30,364 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 00:43:30,365 INFO repeater task os150nb_10pings starting 4.6 seconds late 2017-06-02 00:43:30,457 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 00:43:30,459 INFO repeater task posmv_hcorrstats starting 4.7 seconds late 2017-06-02 00:43:31,270 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 00:43:31,271 INFO repeater task posmv_plotposmv starting 5.5 seconds late 2017-06-02 00:43:31,874 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 00:43:36,880 INFO repeater task os150bb_lastensq starting 0.9 seconds late 2017-06-02 00:43:38,535 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-02 00:43:38,536 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 00:43:26 2017-06-02 00:43:38,536 INFO repeater task os150nb_lastensq starting 2.5 seconds late 2017-06-02 00:43:39,696 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 00:43:39,697 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 00:43:26 2017-06-02 00:48:30,186 INFO repeater task posmv_hcorrstats starting 4.4 seconds late 2017-06-02 00:48:31,136 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 00:48:31,136 INFO repeater task posmv_plotposmv starting 5.4 seconds late 2017-06-02 00:48:31,758 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 00:48:36,763 INFO repeater task os150bb_lastensq starting 0.8 seconds late 2017-06-02 00:48:38,532 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 00:48:38,532 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 00:48:26 2017-06-02 00:48:38,533 INFO repeater task os150nb_lastensq starting 2.5 seconds late 2017-06-02 00:48:39,683 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 00:48:39,684 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 00:48:26 2017-06-02 00:48:49,702 INFO repeater task os150bb_10pings starting 3.9 seconds late 2017-06-02 00:48:49,833 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 00:48:49,834 INFO repeater task os150nb_10pings starting 4.1 seconds late 2017-06-02 00:48:49,928 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 00:53:30,520 INFO repeater task os150bb_quick starting 4.7 seconds late 2017-06-02 00:53:33,786 INFO repeater task os150bb_quick ended after 3.3 seconds 2017-06-02 00:53:33,786 INFO repeater task os150bb_3dayplots starting 8.0 seconds late 2017-06-02 00:53:40,778 INFO repeater task os150bb_3dayplots ended after 7.0 seconds 2017-06-02 00:53:40,778 INFO repeater task os150bb_tsstats starting 15.0 seconds late 2017-06-02 00:53:41,342 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-02 00:53:41,342 INFO repeater task os150nb_lastensq starting 5.3 seconds late 2017-06-02 00:53:42,602 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-02 00:53:42,602 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 00:53:26 2017-06-02 00:53:42,602 INFO repeater task os150nb_quick starting 16.8 seconds late 2017-06-02 00:53:45,765 INFO repeater task os150nb_quick ended after 3.2 seconds 2017-06-02 00:53:45,765 INFO repeater task os150nb_3dayplots starting 20.0 seconds late 2017-06-02 00:53:53,146 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-02 00:53:53,146 INFO repeater task os150nb_3dayplots ended after 7.4 seconds 2017-06-02 00:53:53,146 INFO repeater task os150nb_tsstats starting 27.4 seconds late 2017-06-02 00:53:53,737 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-02 00:53:53,737 INFO repeater task os150bb_beamplots starting 28.0 seconds late 2017-06-02 00:53:54,713 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 00:53:54,713 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-02 00:53:54,714 INFO repeater task os150bb_beamplots starting 28.9 seconds late 2017-06-02 00:53:55,717 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 00:53:55,717 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-02 00:53:55,717 INFO repeater task os150nb_beamplots starting 29.9 seconds late 2017-06-02 00:53:56,692 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 00:53:56,692 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-02 00:53:56,692 INFO repeater task os150nb_beamplots starting 30.9 seconds late 2017-06-02 00:53:57,689 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 00:53:57,690 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-02 00:53:57,690 INFO repeater task posmv_hcorrstats starting 31.9 seconds late 2017-06-02 00:53:58,533 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 00:53:58,533 INFO repeater task posmv_plotposmv starting 32.8 seconds late 2017-06-02 00:53:59,196 INFO repeater task posmv_plotposmv ended after 0.7 seconds 2017-06-02 00:54:04,202 INFO repeater task os150bb_lastensq starting 28.2 seconds late 2017-06-02 00:54:06,123 INFO repeater task os150bb_lastensq ended after 1.9 seconds 2017-06-02 00:54:06,123 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 00:53:26 2017-06-02 00:54:06,124 INFO repeater task os150bb_10pings starting 0.3 seconds late 2017-06-02 00:54:06,219 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 00:54:06,219 INFO repeater task os150nb_10pings starting 0.4 seconds late 2017-06-02 00:54:06,312 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 00:58:26,852 INFO repeater task posmv_hcorrstats starting 1.1 seconds late 2017-06-02 00:58:27,683 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 00:58:27,683 INFO repeater task posmv_plotposmv starting 1.9 seconds late 2017-06-02 00:58:28,321 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 00:58:38,333 INFO repeater task os150bb_lastensq starting 2.3 seconds late 2017-06-02 00:58:39,987 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-02 00:58:39,988 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 00:58:26 2017-06-02 00:58:39,988 INFO repeater task os150nb_lastensq starting 4.0 seconds late 2017-06-02 00:58:41,174 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 00:58:41,174 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 00:58:26 2017-06-02 00:59:26,285 INFO repeater task os150bb_10pings starting 0.5 seconds late 2017-06-02 00:59:26,421 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 00:59:26,423 INFO repeater task os150nb_10pings starting 0.6 seconds late 2017-06-02 00:59:26,516 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 01:03:27,007 INFO repeater task posmv_hcorrstats starting 1.2 seconds late 2017-06-02 01:03:27,922 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 01:03:27,922 INFO repeater task posmv_plotposmv starting 2.1 seconds late 2017-06-02 01:03:28,540 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 01:03:38,551 INFO repeater task os150bb_lastensq starting 2.6 seconds late 2017-06-02 01:03:40,329 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 01:03:40,330 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 01:03:26 2017-06-02 01:03:40,330 INFO repeater task os150nb_lastensq starting 4.3 seconds late 2017-06-02 01:03:41,492 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 01:03:41,493 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 01:03:26 2017-06-02 01:04:46,618 INFO repeater task os150bb_10pings starting 0.8 seconds late 2017-06-02 01:04:46,750 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 01:04:46,751 INFO repeater task os150nb_10pings starting 1.0 seconds late 2017-06-02 01:04:46,844 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 01:08:27,316 INFO repeater task os150bb_quick starting 1.5 seconds late 2017-06-02 01:08:30,523 INFO repeater task os150bb_quick ended after 3.2 seconds 2017-06-02 01:08:30,523 INFO repeater task os150bb_3dayplots starting 4.7 seconds late 2017-06-02 01:08:37,258 INFO repeater task os150bb_3dayplots ended after 6.7 seconds 2017-06-02 01:08:37,258 INFO repeater task os150bb_tsstats starting 11.5 seconds late 2017-06-02 01:08:37,820 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-02 01:08:37,821 INFO repeater task os150nb_lastensq starting 1.8 seconds late 2017-06-02 01:08:38,976 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 01:08:38,976 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 01:08:26 2017-06-02 01:08:38,976 INFO repeater task os150nb_quick starting 13.2 seconds late 2017-06-02 01:08:42,153 INFO repeater task os150nb_quick ended after 3.2 seconds 2017-06-02 01:08:42,153 INFO repeater task os150nb_3dayplots starting 16.4 seconds late 2017-06-02 01:08:48,935 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-02 01:08:48,936 INFO repeater task os150nb_3dayplots ended after 6.8 seconds 2017-06-02 01:08:48,936 INFO repeater task os150nb_tsstats starting 23.2 seconds late 2017-06-02 01:08:49,485 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2017-06-02 01:08:49,486 INFO repeater task os150bb_beamplots starting 23.7 seconds late 2017-06-02 01:08:50,403 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 01:08:50,403 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-02 01:08:50,404 INFO repeater task os150bb_beamplots starting 24.6 seconds late 2017-06-02 01:08:51,345 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 01:08:51,345 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-02 01:08:51,346 INFO repeater task os150nb_beamplots starting 25.6 seconds late 2017-06-02 01:08:52,253 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 01:08:52,253 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 01:08:52,254 INFO repeater task os150nb_beamplots starting 26.5 seconds late 2017-06-02 01:08:53,163 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 01:08:53,164 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 01:08:53,164 INFO repeater task posmv_hcorrstats starting 27.4 seconds late 2017-06-02 01:08:53,981 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 01:08:53,982 INFO repeater task posmv_plotposmv starting 28.2 seconds late 2017-06-02 01:08:54,593 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 01:08:59,599 INFO repeater task os150bb_lastensq starting 23.6 seconds late 2017-06-02 01:09:01,318 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-02 01:09:01,318 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 01:08:26 2017-06-02 01:10:06,451 INFO repeater task os150bb_10pings starting 0.7 seconds late 2017-06-02 01:10:06,617 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 01:10:06,617 INFO repeater task os150nb_10pings starting 0.8 seconds late 2017-06-02 01:10:06,718 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 01:13:27,123 INFO repeater task posmv_hcorrstats starting 1.3 seconds late 2017-06-02 01:13:28,073 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 01:13:28,074 INFO repeater task posmv_plotposmv starting 2.3 seconds late 2017-06-02 01:13:28,681 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 01:13:38,699 INFO repeater task os150bb_lastensq starting 2.7 seconds late 2017-06-02 01:13:40,537 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 01:13:40,537 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 01:13:26 2017-06-02 01:13:40,538 INFO repeater task os150nb_lastensq starting 4.5 seconds late 2017-06-02 01:13:41,782 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 01:13:41,782 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 01:13:26 2017-06-02 01:15:26,992 INFO repeater task os150bb_10pings starting 1.2 seconds late 2017-06-02 01:15:27,111 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 01:15:27,112 INFO repeater task os150nb_10pings starting 1.3 seconds late 2017-06-02 01:15:27,205 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 01:18:27,589 INFO repeater task posmv_hcorrstats starting 1.8 seconds late 2017-06-02 01:18:28,450 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 01:18:28,450 INFO repeater task posmv_plotposmv starting 2.7 seconds late 2017-06-02 01:18:29,067 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 01:18:39,085 INFO repeater task os150bb_lastensq starting 3.1 seconds late 2017-06-02 01:18:40,835 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-02 01:18:40,835 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 01:18:26 2017-06-02 01:18:40,836 INFO repeater task os150nb_lastensq starting 4.8 seconds late 2017-06-02 01:18:41,992 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 01:18:41,992 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 01:18:26 2017-06-02 01:20:47,342 INFO repeater task os150bb_10pings starting 1.6 seconds late 2017-06-02 01:20:47,478 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 01:20:47,479 INFO repeater task os150nb_10pings starting 1.7 seconds late 2017-06-02 01:20:47,572 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 01:23:27,915 INFO repeater task os150bb_quick starting 2.1 seconds late 2017-06-02 01:23:31,179 INFO repeater task os150bb_quick ended after 3.3 seconds 2017-06-02 01:23:31,179 INFO repeater task os150bb_3dayplots starting 5.4 seconds late 2017-06-02 01:23:37,910 INFO repeater task os150bb_3dayplots ended after 6.7 seconds 2017-06-02 01:23:37,911 INFO repeater task os150bb_tsstats starting 12.1 seconds late 2017-06-02 01:23:38,459 INFO repeater task os150bb_tsstats ended after 0.5 seconds 2017-06-02 01:23:38,460 INFO repeater task os150nb_lastensq starting 2.5 seconds late 2017-06-02 01:23:39,651 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 01:23:39,651 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 01:23:26 2017-06-02 01:23:39,652 INFO repeater task os150nb_quick starting 13.9 seconds late 2017-06-02 01:23:42,748 INFO repeater task os150nb_quick ended after 3.1 seconds 2017-06-02 01:23:42,748 INFO repeater task os150nb_3dayplots starting 17.0 seconds late 2017-06-02 01:23:49,512 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-02 01:23:49,513 INFO repeater task os150nb_3dayplots ended after 6.8 seconds 2017-06-02 01:23:49,513 INFO repeater task os150nb_tsstats starting 23.7 seconds late 2017-06-02 01:23:50,056 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2017-06-02 01:23:50,057 INFO repeater task os150bb_beamplots starting 24.3 seconds late 2017-06-02 01:23:50,978 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 01:23:50,978 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-02 01:23:50,979 INFO repeater task os150bb_beamplots starting 25.2 seconds late 2017-06-02 01:23:51,877 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 01:23:51,878 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-02 01:23:51,878 INFO repeater task os150nb_beamplots starting 26.1 seconds late 2017-06-02 01:23:52,781 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 01:23:52,781 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 01:23:52,781 INFO repeater task os150nb_beamplots starting 27.0 seconds late 2017-06-02 01:23:53,697 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 01:23:53,697 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 01:23:53,697 INFO repeater task posmv_hcorrstats starting 27.9 seconds late 2017-06-02 01:23:54,511 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 01:23:54,511 INFO repeater task posmv_plotposmv starting 28.7 seconds late 2017-06-02 01:23:55,113 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 01:24:00,118 INFO repeater task os150bb_lastensq starting 24.1 seconds late 2017-06-02 01:24:01,843 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-02 01:24:01,843 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 01:23:26 2017-06-02 01:26:07,077 INFO repeater task os150bb_10pings starting 1.3 seconds late 2017-06-02 01:26:07,194 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 01:26:07,195 INFO repeater task os150nb_10pings starting 1.4 seconds late 2017-06-02 01:26:07,286 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 01:28:27,620 INFO repeater task posmv_hcorrstats starting 1.8 seconds late 2017-06-02 01:28:28,437 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 01:28:28,438 INFO repeater task posmv_plotposmv starting 2.7 seconds late 2017-06-02 01:28:29,046 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 01:28:39,064 INFO repeater task os150bb_lastensq starting 3.1 seconds late 2017-06-02 01:28:40,779 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-02 01:28:40,779 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 01:28:26 2017-06-02 01:28:40,780 INFO repeater task os150nb_lastensq starting 4.8 seconds late 2017-06-02 01:28:41,946 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 01:28:41,947 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 01:28:26 2017-06-02 01:31:27,406 INFO repeater task os150bb_10pings starting 1.6 seconds late 2017-06-02 01:31:27,525 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 01:31:27,526 INFO repeater task os150nb_10pings starting 1.7 seconds late 2017-06-02 01:31:27,621 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 01:33:27,870 INFO repeater task posmv_hcorrstats starting 2.1 seconds late 2017-06-02 01:33:28,796 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 01:33:28,796 INFO repeater task posmv_plotposmv starting 3.0 seconds late 2017-06-02 01:33:29,407 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 01:33:39,421 INFO repeater task os150bb_lastensq starting 3.4 seconds late 2017-06-02 01:33:41,165 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-02 01:33:41,165 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 01:33:26 2017-06-02 01:33:41,166 INFO repeater task os150nb_lastensq starting 5.2 seconds late 2017-06-02 01:33:42,375 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 01:33:42,376 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 01:33:26 2017-06-02 01:36:47,756 INFO repeater task os150bb_10pings starting 2.0 seconds late 2017-06-02 01:36:47,900 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 01:36:47,901 INFO repeater task os150nb_10pings starting 2.1 seconds late 2017-06-02 01:36:47,996 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 01:38:28,268 INFO repeater task os150bb_quick starting 2.5 seconds late 2017-06-02 01:38:31,461 INFO repeater task os150bb_quick ended after 3.2 seconds 2017-06-02 01:38:31,462 INFO repeater task os150bb_3dayplots starting 5.7 seconds late 2017-06-02 01:38:38,602 INFO repeater task os150bb_3dayplots ended after 7.1 seconds 2017-06-02 01:38:38,602 INFO repeater task os150bb_tsstats starting 12.8 seconds late 2017-06-02 01:38:39,150 INFO repeater task os150bb_tsstats ended after 0.5 seconds 2017-06-02 01:38:39,150 INFO repeater task os150nb_lastensq starting 3.2 seconds late 2017-06-02 01:38:40,388 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 01:38:40,388 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 01:38:26 2017-06-02 01:38:40,389 INFO repeater task os150nb_quick starting 14.6 seconds late 2017-06-02 01:38:43,503 INFO repeater task os150nb_quick ended after 3.1 seconds 2017-06-02 01:38:43,504 INFO repeater task os150nb_3dayplots starting 17.7 seconds late 2017-06-02 01:38:50,295 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-02 01:38:50,295 INFO repeater task os150nb_3dayplots ended after 6.8 seconds 2017-06-02 01:38:50,296 INFO repeater task os150nb_tsstats starting 24.5 seconds late 2017-06-02 01:38:50,875 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-02 01:38:50,875 INFO repeater task os150bb_beamplots starting 25.1 seconds late 2017-06-02 01:38:51,811 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 01:38:51,811 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-02 01:38:51,811 INFO repeater task os150bb_beamplots starting 26.0 seconds late 2017-06-02 01:38:52,784 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 01:38:52,784 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-02 01:38:52,785 INFO repeater task os150nb_beamplots starting 27.0 seconds late 2017-06-02 01:38:53,760 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 01:38:53,760 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-02 01:38:53,760 INFO repeater task os150nb_beamplots starting 28.0 seconds late 2017-06-02 01:38:54,737 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 01:38:54,737 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-02 01:38:54,737 INFO repeater task posmv_hcorrstats starting 29.0 seconds late 2017-06-02 01:38:55,613 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 01:38:55,613 INFO repeater task posmv_plotposmv starting 29.8 seconds late 2017-06-02 01:38:56,286 INFO repeater task posmv_plotposmv ended after 0.7 seconds 2017-06-02 01:39:01,291 INFO repeater task os150bb_lastensq starting 25.3 seconds late 2017-06-02 01:39:03,040 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-02 01:39:03,041 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 01:38:26 2017-06-02 01:42:08,566 INFO repeater task os150bb_10pings starting 2.8 seconds late 2017-06-02 01:42:08,739 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 01:42:08,740 INFO repeater task os150nb_10pings starting 3.0 seconds late 2017-06-02 01:42:08,837 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 01:43:28,998 INFO repeater task posmv_hcorrstats starting 3.2 seconds late 2017-06-02 01:43:29,946 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 01:43:29,946 INFO repeater task posmv_plotposmv starting 4.2 seconds late 2017-06-02 01:43:30,560 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 01:43:40,575 INFO repeater task os150bb_lastensq starting 4.6 seconds late 2017-06-02 01:43:42,335 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 01:43:42,336 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 01:43:26 2017-06-02 01:43:42,336 INFO repeater task os150nb_lastensq starting 6.3 seconds late 2017-06-02 01:43:43,540 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 01:43:43,540 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 01:43:26 2017-06-02 01:47:28,987 INFO repeater task os150bb_10pings starting 3.2 seconds late 2017-06-02 01:47:29,152 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 01:47:29,153 INFO repeater task os150nb_10pings starting 3.4 seconds late 2017-06-02 01:47:29,246 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 01:48:29,422 INFO repeater task posmv_hcorrstats starting 3.6 seconds late 2017-06-02 01:48:30,416 INFO repeater task posmv_hcorrstats ended after 1.0 seconds 2017-06-02 01:48:30,416 INFO repeater task posmv_plotposmv starting 4.6 seconds late 2017-06-02 01:48:31,044 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 01:48:36,050 INFO repeater task os150bb_lastensq starting 0.1 seconds late 2017-06-02 01:48:37,916 INFO repeater task os150bb_lastensq ended after 1.9 seconds 2017-06-02 01:48:37,916 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 01:48:26 2017-06-02 01:48:37,917 INFO repeater task os150nb_lastensq starting 1.9 seconds late 2017-06-02 01:48:39,228 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-02 01:48:39,228 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 01:48:26 2017-06-02 01:52:49,792 INFO repeater task os150bb_10pings starting 4.0 seconds late 2017-06-02 01:52:49,933 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 01:52:49,934 INFO repeater task os150nb_10pings starting 4.2 seconds late 2017-06-02 01:52:50,027 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 01:53:30,113 INFO repeater task os150bb_quick starting 4.3 seconds late 2017-06-02 01:53:33,440 INFO repeater task os150bb_quick ended after 3.3 seconds 2017-06-02 01:53:33,440 INFO repeater task os150bb_3dayplots starting 7.7 seconds late 2017-06-02 01:53:40,751 INFO repeater task os150bb_3dayplots ended after 7.3 seconds 2017-06-02 01:53:40,751 INFO repeater task os150bb_tsstats starting 15.0 seconds late 2017-06-02 01:53:41,348 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-02 01:53:41,349 INFO repeater task os150nb_lastensq starting 5.3 seconds late 2017-06-02 01:53:42,640 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-02 01:53:42,640 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 01:53:26 2017-06-02 01:53:42,641 INFO repeater task os150nb_quick starting 16.9 seconds late 2017-06-02 01:53:45,828 INFO repeater task os150nb_quick ended after 3.2 seconds 2017-06-02 01:53:45,828 INFO repeater task os150nb_3dayplots starting 20.0 seconds late 2017-06-02 01:53:52,877 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-02 01:53:52,878 INFO repeater task os150nb_3dayplots ended after 7.0 seconds 2017-06-02 01:53:52,878 INFO repeater task os150nb_tsstats starting 27.1 seconds late 2017-06-02 01:53:53,445 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-02 01:53:53,445 INFO repeater task os150bb_beamplots starting 27.7 seconds late 2017-06-02 01:53:54,380 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 01:53:54,380 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-02 01:53:54,380 INFO repeater task os150bb_beamplots starting 28.6 seconds late 2017-06-02 01:53:55,389 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 01:53:55,390 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-02 01:53:55,390 INFO repeater task os150nb_beamplots starting 29.6 seconds late 2017-06-02 01:53:56,309 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 01:53:56,309 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 01:53:56,310 INFO repeater task os150nb_beamplots starting 30.5 seconds late 2017-06-02 01:53:57,318 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 01:53:57,318 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-02 01:53:57,319 INFO repeater task posmv_hcorrstats starting 31.5 seconds late 2017-06-02 01:53:58,218 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 01:53:58,218 INFO repeater task posmv_plotposmv starting 32.4 seconds late 2017-06-02 01:53:58,865 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 01:54:03,869 INFO repeater task os150bb_lastensq starting 27.9 seconds late 2017-06-02 01:54:05,692 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 01:54:05,692 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 01:53:26 2017-06-02 01:58:06,181 INFO repeater task os150bb_10pings starting 0.4 seconds late 2017-06-02 01:58:06,318 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 01:58:06,318 INFO repeater task os150nb_10pings starting 0.5 seconds late 2017-06-02 01:58:06,414 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 01:58:26,448 INFO repeater task posmv_hcorrstats starting 0.7 seconds late 2017-06-02 01:58:27,303 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 01:58:27,303 INFO repeater task posmv_plotposmv starting 1.5 seconds late 2017-06-02 01:58:27,934 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 01:58:37,953 INFO repeater task os150bb_lastensq starting 2.0 seconds late 2017-06-02 01:58:39,591 INFO repeater task os150bb_lastensq ended after 1.6 seconds 2017-06-02 01:58:39,592 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 01:58:26 2017-06-02 01:58:39,592 INFO repeater task os150nb_lastensq starting 3.6 seconds late 2017-06-02 01:58:40,772 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 01:58:40,772 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 01:58:26 2017-06-02 02:03:26,373 INFO repeater task os150bb_10pings starting 0.6 seconds late 2017-06-02 02:03:26,543 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 02:03:26,544 INFO repeater task os150nb_10pings starting 0.8 seconds late 2017-06-02 02:03:26,638 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 02:03:26,640 INFO repeater task posmv_hcorrstats starting 0.9 seconds late 2017-06-02 02:03:27,486 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 02:03:27,486 INFO repeater task posmv_plotposmv starting 1.7 seconds late 2017-06-02 02:03:28,089 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 02:03:38,104 INFO repeater task os150bb_lastensq starting 2.1 seconds late 2017-06-02 02:03:39,810 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-02 02:03:39,810 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 02:03:26 2017-06-02 02:03:39,811 INFO repeater task os150nb_lastensq starting 3.8 seconds late 2017-06-02 02:03:41,000 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 02:03:41,000 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 02:03:26 2017-06-02 02:08:26,596 INFO repeater task os150bb_quick starting 0.8 seconds late 2017-06-02 02:08:29,811 INFO repeater task os150bb_quick ended after 3.2 seconds 2017-06-02 02:08:29,811 INFO repeater task os150bb_3dayplots starting 4.0 seconds late 2017-06-02 02:08:37,147 INFO repeater task os150bb_3dayplots ended after 7.3 seconds 2017-06-02 02:08:37,147 INFO repeater task os150bb_tsstats starting 11.4 seconds late 2017-06-02 02:08:37,704 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-02 02:08:37,704 INFO repeater task os150nb_lastensq starting 1.7 seconds late 2017-06-02 02:08:38,930 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 02:08:38,930 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 02:08:26 2017-06-02 02:08:38,930 INFO repeater task os150nb_quick starting 13.1 seconds late 2017-06-02 02:08:42,155 INFO repeater task os150nb_quick ended after 3.2 seconds 2017-06-02 02:08:42,155 INFO repeater task os150nb_3dayplots starting 16.4 seconds late 2017-06-02 02:08:49,552 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-02 02:08:49,552 INFO repeater task os150nb_3dayplots ended after 7.4 seconds 2017-06-02 02:08:49,552 INFO repeater task os150nb_tsstats starting 23.8 seconds late 2017-06-02 02:08:50,160 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-02 02:08:50,160 INFO repeater task os150bb_beamplots starting 24.4 seconds late 2017-06-02 02:08:51,151 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 02:08:51,151 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-02 02:08:51,152 INFO repeater task os150bb_beamplots starting 25.4 seconds late 2017-06-02 02:08:52,142 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 02:08:52,142 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-02 02:08:52,142 INFO repeater task os150nb_beamplots starting 26.4 seconds late 2017-06-02 02:08:53,073 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 02:08:53,073 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 02:08:53,073 INFO repeater task os150nb_beamplots starting 27.3 seconds late 2017-06-02 02:08:54,045 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 02:08:54,045 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-02 02:08:54,045 INFO repeater task posmv_hcorrstats starting 28.3 seconds late 2017-06-02 02:08:54,891 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 02:08:54,891 INFO repeater task posmv_plotposmv starting 29.1 seconds late 2017-06-02 02:08:55,514 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 02:09:00,520 INFO repeater task os150bb_lastensq starting 24.5 seconds late 2017-06-02 02:09:02,432 INFO repeater task os150bb_lastensq ended after 1.9 seconds 2017-06-02 02:09:02,432 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 02:08:26 2017-06-02 02:09:02,432 INFO repeater task os150bb_10pings starting 16.7 seconds late 2017-06-02 02:09:02,534 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 02:09:02,535 INFO repeater task os150nb_10pings starting 16.8 seconds late 2017-06-02 02:09:02,631 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 02:13:28,202 INFO repeater task posmv_hcorrstats starting 2.4 seconds late 2017-06-02 02:13:29,136 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 02:13:29,136 INFO repeater task posmv_plotposmv starting 3.4 seconds late 2017-06-02 02:13:29,745 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 02:13:39,765 INFO repeater task os150bb_lastensq starting 3.8 seconds late 2017-06-02 02:13:41,539 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 02:13:41,539 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 02:13:26 2017-06-02 02:13:41,540 INFO repeater task os150nb_lastensq starting 5.5 seconds late 2017-06-02 02:13:42,681 INFO repeater task os150nb_lastensq ended after 1.1 seconds 2017-06-02 02:13:42,681 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 02:13:26 2017-06-02 02:14:07,728 INFO repeater task os150bb_10pings starting 1.9 seconds late 2017-06-02 02:14:07,852 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 02:14:07,854 INFO repeater task os150nb_10pings starting 2.1 seconds late 2017-06-02 02:14:07,943 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 02:18:28,543 INFO repeater task posmv_hcorrstats starting 2.8 seconds late 2017-06-02 02:18:29,454 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 02:18:29,454 INFO repeater task posmv_plotposmv starting 3.7 seconds late 2017-06-02 02:18:30,080 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 02:18:40,095 INFO repeater task os150bb_lastensq starting 4.1 seconds late 2017-06-02 02:18:41,746 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-02 02:18:41,746 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 02:18:26 2017-06-02 02:18:41,747 INFO repeater task os150nb_lastensq starting 5.7 seconds late 2017-06-02 02:18:42,932 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 02:18:42,932 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 02:18:26 2017-06-02 02:19:28,019 INFO repeater task os150bb_10pings starting 2.2 seconds late 2017-06-02 02:19:28,122 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 02:19:28,123 INFO repeater task os150nb_10pings starting 2.3 seconds late 2017-06-02 02:19:28,216 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 02:23:28,703 INFO repeater task os150bb_quick starting 2.9 seconds late 2017-06-02 02:23:32,033 INFO repeater task os150bb_quick ended after 3.3 seconds 2017-06-02 02:23:32,033 INFO repeater task os150bb_3dayplots starting 6.3 seconds late 2017-06-02 02:23:39,131 INFO repeater task os150bb_3dayplots ended after 7.1 seconds 2017-06-02 02:23:39,131 INFO repeater task os150bb_tsstats starting 13.4 seconds late 2017-06-02 02:23:39,681 INFO repeater task os150bb_tsstats ended after 0.5 seconds 2017-06-02 02:23:39,682 INFO repeater task os150nb_lastensq starting 3.7 seconds late 2017-06-02 02:23:40,944 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-02 02:23:40,945 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 02:23:26 2017-06-02 02:23:40,945 INFO repeater task os150nb_quick starting 15.2 seconds late 2017-06-02 02:23:44,080 INFO repeater task os150nb_quick ended after 3.1 seconds 2017-06-02 02:23:44,080 INFO repeater task os150nb_3dayplots starting 18.3 seconds late 2017-06-02 02:23:50,911 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-02 02:23:50,911 INFO repeater task os150nb_3dayplots ended after 6.8 seconds 2017-06-02 02:23:50,911 INFO repeater task os150nb_tsstats starting 25.1 seconds late 2017-06-02 02:23:51,503 INFO repeater task os150nb_tsstats output: /usr/lib/python2.7/dist-packages/numpy/ma/core.py:1073: RuntimeWarning: overflow encountered in true_divide result = self.f(da, db, *args, **kwargs) 2017-06-02 02:23:51,503 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-02 02:23:51,503 INFO repeater task os150bb_beamplots starting 25.7 seconds late 2017-06-02 02:23:52,406 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 02:23:52,406 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-02 02:23:52,406 INFO repeater task os150bb_beamplots starting 26.6 seconds late 2017-06-02 02:23:53,375 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 02:23:53,375 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-02 02:23:53,376 INFO repeater task os150nb_beamplots starting 27.6 seconds late 2017-06-02 02:23:54,329 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 02:23:54,330 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-02 02:23:54,330 INFO repeater task os150nb_beamplots starting 28.5 seconds late 2017-06-02 02:23:55,243 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 02:23:55,243 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 02:23:55,243 INFO repeater task posmv_hcorrstats starting 29.5 seconds late 2017-06-02 02:23:56,098 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 02:23:56,098 INFO repeater task posmv_plotposmv starting 30.3 seconds late 2017-06-02 02:23:56,711 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 02:24:01,717 INFO repeater task os150bb_lastensq starting 25.7 seconds late 2017-06-02 02:24:03,675 INFO repeater task os150bb_lastensq ended after 2.0 seconds 2017-06-02 02:24:03,675 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 02:23:26 2017-06-02 02:24:48,760 INFO repeater task os150bb_10pings starting 3.0 seconds late 2017-06-02 02:24:48,905 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 02:24:48,906 INFO repeater task os150nb_10pings starting 3.1 seconds late 2017-06-02 02:24:49,004 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 02:28:29,454 INFO repeater task posmv_hcorrstats starting 3.7 seconds late 2017-06-02 02:28:30,431 INFO repeater task posmv_hcorrstats ended after 1.0 seconds 2017-06-02 02:28:30,431 INFO repeater task posmv_plotposmv starting 4.6 seconds late 2017-06-02 02:28:31,070 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 02:28:36,076 INFO repeater task os150bb_lastensq starting 0.1 seconds late 2017-06-02 02:28:37,955 INFO repeater task os150bb_lastensq ended after 1.9 seconds 2017-06-02 02:28:37,955 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 02:28:26 2017-06-02 02:28:37,955 INFO repeater task os150nb_lastensq starting 2.0 seconds late 2017-06-02 02:28:39,226 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-02 02:28:39,226 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 02:28:26 2017-06-02 02:30:09,416 INFO repeater task os150bb_10pings starting 3.6 seconds late 2017-06-02 02:30:09,591 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 02:30:09,592 INFO repeater task os150nb_10pings starting 3.8 seconds late 2017-06-02 02:30:09,687 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 02:33:30,107 INFO repeater task posmv_hcorrstats starting 4.3 seconds late 2017-06-02 02:33:31,046 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 02:33:31,046 INFO repeater task posmv_plotposmv starting 5.3 seconds late 2017-06-02 02:33:31,690 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 02:33:36,696 INFO repeater task os150bb_lastensq starting 0.7 seconds late 2017-06-02 02:33:38,604 INFO repeater task os150bb_lastensq ended after 1.9 seconds 2017-06-02 02:33:38,604 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 02:33:26 2017-06-02 02:33:38,605 INFO repeater task os150nb_lastensq starting 2.6 seconds late 2017-06-02 02:33:39,839 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 02:33:39,840 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 02:33:26 2017-06-02 02:35:30,061 INFO repeater task os150bb_10pings starting 4.3 seconds late 2017-06-02 02:35:30,193 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 02:35:30,193 INFO repeater task os150nb_10pings starting 4.4 seconds late 2017-06-02 02:35:30,286 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 02:38:30,663 INFO repeater task os150bb_quick starting 4.9 seconds late 2017-06-02 02:38:33,824 INFO repeater task os150bb_quick ended after 3.2 seconds 2017-06-02 02:38:33,824 INFO repeater task os150bb_3dayplots starting 8.0 seconds late 2017-06-02 02:38:40,546 INFO repeater task os150bb_3dayplots ended after 6.7 seconds 2017-06-02 02:38:40,547 INFO repeater task os150bb_tsstats starting 14.8 seconds late 2017-06-02 02:38:41,086 INFO repeater task os150bb_tsstats ended after 0.5 seconds 2017-06-02 02:38:41,086 INFO repeater task os150nb_lastensq starting 5.1 seconds late 2017-06-02 02:38:42,353 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-02 02:38:42,354 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 02:38:26 2017-06-02 02:38:42,354 INFO repeater task os150nb_quick starting 16.6 seconds late 2017-06-02 02:38:45,469 INFO repeater task os150nb_quick ended after 3.1 seconds 2017-06-02 02:38:45,470 INFO repeater task os150nb_3dayplots starting 19.7 seconds late 2017-06-02 02:38:52,223 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-02 02:38:52,223 INFO repeater task os150nb_3dayplots ended after 6.8 seconds 2017-06-02 02:38:52,223 INFO repeater task os150nb_tsstats starting 26.4 seconds late 2017-06-02 02:38:52,762 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2017-06-02 02:38:52,762 INFO repeater task os150bb_beamplots starting 27.0 seconds late 2017-06-02 02:38:53,705 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 02:38:53,705 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-02 02:38:53,705 INFO repeater task os150bb_beamplots starting 27.9 seconds late 2017-06-02 02:38:54,628 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 02:38:54,628 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-02 02:38:54,628 INFO repeater task os150nb_beamplots starting 28.8 seconds late 2017-06-02 02:38:55,551 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 02:38:55,552 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 02:38:55,552 INFO repeater task os150nb_beamplots starting 29.8 seconds late 2017-06-02 02:38:56,456 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 02:38:56,456 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 02:38:56,456 INFO repeater task posmv_hcorrstats starting 30.7 seconds late 2017-06-02 02:38:57,265 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 02:38:57,265 INFO repeater task posmv_plotposmv starting 31.5 seconds late 2017-06-02 02:38:57,879 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 02:39:02,885 INFO repeater task os150bb_lastensq starting 26.9 seconds late 2017-06-02 02:39:04,668 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 02:39:04,668 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 02:38:26 2017-06-02 02:40:49,889 INFO repeater task os150bb_10pings starting 4.1 seconds late 2017-06-02 02:40:50,016 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 02:40:50,016 INFO repeater task os150nb_10pings starting 4.2 seconds late 2017-06-02 02:40:50,110 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 02:43:30,433 INFO repeater task posmv_hcorrstats starting 4.7 seconds late 2017-06-02 02:43:31,358 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 02:43:31,358 INFO repeater task posmv_plotposmv starting 5.6 seconds late 2017-06-02 02:43:31,970 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 02:43:36,976 INFO repeater task os150bb_lastensq starting 1.0 seconds late 2017-06-02 02:43:38,750 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 02:43:38,750 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 02:43:26 2017-06-02 02:43:38,751 INFO repeater task os150nb_lastensq starting 2.8 seconds late 2017-06-02 02:43:39,932 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 02:43:39,932 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 02:43:26 2017-06-02 02:46:10,153 INFO repeater task os150bb_10pings starting 4.4 seconds late 2017-06-02 02:46:10,256 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 02:46:10,256 INFO repeater task os150nb_10pings starting 4.5 seconds late 2017-06-02 02:46:10,349 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 02:48:30,596 INFO repeater task posmv_hcorrstats starting 4.8 seconds late 2017-06-02 02:48:31,433 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 02:48:31,434 INFO repeater task posmv_plotposmv starting 5.7 seconds late 2017-06-02 02:48:32,047 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 02:48:37,052 INFO repeater task os150bb_lastensq starting 1.1 seconds late 2017-06-02 02:48:38,771 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-02 02:48:38,772 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 02:48:26 2017-06-02 02:48:38,772 INFO repeater task os150nb_lastensq starting 2.8 seconds late 2017-06-02 02:48:39,945 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 02:48:39,945 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 02:48:26 2017-06-02 02:51:30,280 INFO repeater task os150bb_10pings starting 4.5 seconds late 2017-06-02 02:51:30,390 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 02:51:30,391 INFO repeater task os150nb_10pings starting 4.6 seconds late 2017-06-02 02:51:30,484 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 02:53:30,727 INFO repeater task os150bb_quick starting 4.9 seconds late 2017-06-02 02:53:34,029 INFO repeater task os150bb_quick ended after 3.3 seconds 2017-06-02 02:53:34,030 INFO repeater task os150bb_3dayplots starting 8.2 seconds late 2017-06-02 02:53:40,986 INFO repeater task os150bb_3dayplots ended after 7.0 seconds 2017-06-02 02:53:40,987 INFO repeater task os150bb_tsstats starting 15.2 seconds late 2017-06-02 02:53:41,588 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-02 02:53:41,589 INFO repeater task os150nb_lastensq starting 5.6 seconds late 2017-06-02 02:53:42,811 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 02:53:42,812 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 02:53:26 2017-06-02 02:53:42,812 INFO repeater task os150nb_quick starting 17.0 seconds late 2017-06-02 02:53:46,050 INFO repeater task os150nb_quick ended after 3.2 seconds 2017-06-02 02:53:46,051 INFO repeater task os150nb_3dayplots starting 20.3 seconds late 2017-06-02 02:53:53,359 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-02 02:53:53,360 INFO repeater task os150nb_3dayplots ended after 7.3 seconds 2017-06-02 02:53:53,360 INFO repeater task os150nb_tsstats starting 27.6 seconds late 2017-06-02 02:53:53,966 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-02 02:53:53,967 INFO repeater task os150bb_beamplots starting 28.2 seconds late 2017-06-02 02:53:54,922 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 02:53:54,922 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-02 02:53:54,923 INFO repeater task os150bb_beamplots starting 29.1 seconds late 2017-06-02 02:53:55,906 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 02:53:55,906 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-02 02:53:55,907 INFO repeater task os150nb_beamplots starting 30.1 seconds late 2017-06-02 02:53:56,808 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 02:53:56,809 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 02:53:56,809 INFO repeater task os150nb_beamplots starting 31.0 seconds late 2017-06-02 02:53:57,761 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 02:53:57,762 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-02 02:53:57,762 INFO repeater task posmv_hcorrstats starting 32.0 seconds late 2017-06-02 02:53:58,652 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 02:53:58,653 INFO repeater task posmv_plotposmv starting 32.9 seconds late 2017-06-02 02:53:59,273 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 02:54:04,278 INFO repeater task os150bb_lastensq starting 28.3 seconds late 2017-06-02 02:54:06,163 INFO repeater task os150bb_lastensq ended after 1.9 seconds 2017-06-02 02:54:06,163 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 02:53:26 2017-06-02 02:56:46,562 INFO repeater task os150bb_10pings starting 0.8 seconds late 2017-06-02 02:56:46,707 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 02:56:46,708 INFO repeater task os150nb_10pings starting 0.9 seconds late 2017-06-02 02:56:46,801 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 02:58:27,008 INFO repeater task posmv_hcorrstats starting 1.2 seconds late 2017-06-02 02:58:27,940 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 02:58:27,940 INFO repeater task posmv_plotposmv starting 2.2 seconds late 2017-06-02 02:58:28,549 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 02:58:38,565 INFO repeater task os150bb_lastensq starting 2.6 seconds late 2017-06-02 02:58:40,212 INFO repeater task os150bb_lastensq ended after 1.6 seconds 2017-06-02 02:58:40,212 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 02:58:26 2017-06-02 02:58:40,213 INFO repeater task os150nb_lastensq starting 4.2 seconds late 2017-06-02 02:58:41,364 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 02:58:41,364 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 02:58:26 2017-06-02 03:02:06,808 INFO repeater task os150bb_10pings starting 1.0 seconds late 2017-06-02 03:02:06,982 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 03:02:06,983 INFO repeater task os150nb_10pings starting 1.2 seconds late 2017-06-02 03:02:07,078 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 03:03:27,244 INFO repeater task posmv_hcorrstats starting 1.5 seconds late 2017-06-02 03:03:28,178 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 03:03:28,179 INFO repeater task posmv_plotposmv starting 2.4 seconds late 2017-06-02 03:03:28,788 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 03:03:38,805 INFO repeater task os150bb_lastensq starting 2.8 seconds late 2017-06-02 03:03:40,546 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-02 03:03:40,546 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 03:03:26 2017-06-02 03:03:40,547 INFO repeater task os150nb_lastensq starting 4.5 seconds late 2017-06-02 03:03:41,741 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 03:03:41,742 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 03:03:26 2017-06-02 03:07:27,204 INFO repeater task os150bb_10pings starting 1.4 seconds late 2017-06-02 03:07:27,358 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 03:07:27,359 INFO repeater task os150nb_10pings starting 1.6 seconds late 2017-06-02 03:07:27,456 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 03:08:27,565 INFO repeater task os150bb_quick starting 1.8 seconds late 2017-06-02 03:08:30,686 INFO repeater task os150bb_quick ended after 3.1 seconds 2017-06-02 03:08:30,686 INFO repeater task os150bb_3dayplots starting 4.9 seconds late 2017-06-02 03:08:38,066 INFO repeater task os150bb_3dayplots ended after 7.4 seconds 2017-06-02 03:08:38,066 INFO repeater task os150bb_tsstats starting 12.3 seconds late 2017-06-02 03:08:38,619 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-02 03:08:38,619 INFO repeater task os150nb_lastensq starting 2.6 seconds late 2017-06-02 03:08:39,853 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 03:08:39,853 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 03:08:26