2017-06-02 10:08:52,521 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 10:08:52,522 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 10:08:52,522 INFO repeater task posmv_hcorrstats starting 26.7 seconds late 2017-06-02 10:08:53,345 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 10:08:53,345 INFO repeater task posmv_plotposmv starting 27.6 seconds late 2017-06-02 10:08:53,956 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 10:08:58,962 INFO repeater task os150bb_lastensq starting 23.0 seconds late 2017-06-02 10:09:00,776 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 10:09:00,776 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 10:08:26 2017-06-02 10:09:00,776 INFO repeater task os150bb_10pings starting 15.0 seconds late 2017-06-02 10:09:00,871 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 10:09:00,872 INFO repeater task os150nb_10pings starting 15.1 seconds late 2017-06-02 10:09:00,965 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 10:13:26,467 INFO repeater task posmv_hcorrstats starting 0.7 seconds late 2017-06-02 10:13:27,332 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 10:13:27,333 INFO repeater task posmv_plotposmv starting 1.6 seconds late 2017-06-02 10:13:27,946 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 10:13:37,964 INFO repeater task os150bb_lastensq starting 2.0 seconds late 2017-06-02 10:13:39,629 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-02 10:13:39,629 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 10:13:26 2017-06-02 10:13:39,630 INFO repeater task os150nb_lastensq starting 3.6 seconds late 2017-06-02 10:13:40,796 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 10:13:40,796 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 10:13:26 2017-06-02 10:14:05,847 INFO repeater task os150bb_10pings starting 0.1 seconds late 2017-06-02 10:14:06,012 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 10:14:06,013 INFO repeater task os150nb_10pings starting 0.2 seconds late 2017-06-02 10:14:06,107 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 10:18:26,691 INFO repeater task posmv_hcorrstats starting 0.9 seconds late 2017-06-02 10:18:27,588 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 10:18:27,588 INFO repeater task posmv_plotposmv starting 1.8 seconds late 2017-06-02 10:18:28,210 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 10:18:38,223 INFO repeater task os150bb_lastensq starting 2.2 seconds late 2017-06-02 10:18:39,884 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-02 10:18:39,884 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 10:18:26 2017-06-02 10:18:39,885 INFO repeater task os150nb_lastensq starting 3.9 seconds late 2017-06-02 10:18:41,034 INFO repeater task os150nb_lastensq ended after 1.1 seconds 2017-06-02 10:18:41,034 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 10:18:26 2017-06-02 10:19:26,120 INFO repeater task os150bb_10pings starting 0.3 seconds late 2017-06-02 10:19:26,266 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 10:19:26,266 INFO repeater task os150nb_10pings starting 0.5 seconds late 2017-06-02 10:19:26,358 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 10:23:26,875 INFO repeater task os150bb_quick starting 1.1 seconds late 2017-06-02 10:23:30,151 INFO repeater task os150bb_quick ended after 3.3 seconds 2017-06-02 10:23:30,152 INFO repeater task os150bb_3dayplots starting 4.4 seconds late 2017-06-02 10:23:37,241 INFO repeater task os150bb_3dayplots ended after 7.1 seconds 2017-06-02 10:23:37,242 INFO repeater task os150bb_tsstats starting 11.5 seconds late 2017-06-02 10:23:37,812 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-02 10:23:37,812 INFO repeater task os150nb_lastensq starting 1.8 seconds late 2017-06-02 10:23:39,069 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-02 10:23:39,069 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 10:23:26 2017-06-02 10:23:39,070 INFO repeater task os150nb_quick starting 13.3 seconds late 2017-06-02 10:23:42,254 INFO repeater task os150nb_quick ended after 3.2 seconds 2017-06-02 10:23:42,255 INFO repeater task os150nb_3dayplots starting 16.5 seconds late 2017-06-02 10:23:49,009 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-02 10:23:49,009 INFO repeater task os150nb_3dayplots ended after 6.8 seconds 2017-06-02 10:23:49,009 INFO repeater task os150nb_tsstats starting 23.2 seconds late 2017-06-02 10:23:49,563 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-02 10:23:49,563 INFO repeater task os150bb_beamplots starting 23.8 seconds late 2017-06-02 10:23:50,544 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 10:23:50,544 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-02 10:23:50,545 INFO repeater task os150bb_beamplots starting 24.8 seconds late 2017-06-02 10:23:51,463 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 10:23:51,464 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-02 10:23:51,464 INFO repeater task os150nb_beamplots starting 25.7 seconds late 2017-06-02 10:23:52,405 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 10:23:52,405 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 10:23:52,405 INFO repeater task os150nb_beamplots starting 26.6 seconds late 2017-06-02 10:23:53,346 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 10:23:53,346 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 10:23:53,346 INFO repeater task posmv_hcorrstats starting 27.6 seconds late 2017-06-02 10:23:54,227 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 10:23:54,227 INFO repeater task posmv_plotposmv starting 28.4 seconds late 2017-06-02 10:23:54,857 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 10:23:59,863 INFO repeater task os150bb_lastensq starting 23.9 seconds late 2017-06-02 10:24:01,718 INFO repeater task os150bb_lastensq ended after 1.9 seconds 2017-06-02 10:24:01,718 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 10:23:26 2017-06-02 10:24:46,814 INFO repeater task os150bb_10pings starting 1.0 seconds late 2017-06-02 10:24:47,000 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 10:24:47,001 INFO repeater task os150nb_10pings starting 1.2 seconds late 2017-06-02 10:24:47,106 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 10:28:27,570 INFO repeater task posmv_hcorrstats starting 1.8 seconds late 2017-06-02 10:28:28,401 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 10:28:28,401 INFO repeater task posmv_plotposmv starting 2.6 seconds late 2017-06-02 10:28:29,018 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 10:28:39,035 INFO repeater task os150bb_lastensq starting 3.0 seconds late 2017-06-02 10:28:40,811 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 10:28:40,812 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 10:28:26 2017-06-02 10:28:40,812 INFO repeater task os150nb_lastensq starting 4.8 seconds late 2017-06-02 10:28:41,994 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 10:28:41,994 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 10:28:26 2017-06-02 10:30:07,143 INFO repeater task os150bb_10pings starting 1.4 seconds late 2017-06-02 10:30:07,240 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 10:30:07,240 INFO repeater task os150nb_10pings starting 1.5 seconds late 2017-06-02 10:30:07,332 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 10:33:27,660 INFO repeater task posmv_hcorrstats starting 1.9 seconds late 2017-06-02 10:33:28,543 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 10:33:28,543 INFO repeater task posmv_plotposmv starting 2.8 seconds late 2017-06-02 10:33:29,137 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 10:33:39,154 INFO repeater task os150bb_lastensq starting 3.2 seconds late 2017-06-02 10:33:40,797 INFO repeater task os150bb_lastensq ended after 1.6 seconds 2017-06-02 10:33:40,798 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 10:33:26 2017-06-02 10:33:40,798 INFO repeater task os150nb_lastensq starting 4.8 seconds late 2017-06-02 10:33:41,956 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 10:33:41,956 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 10:33:26 2017-06-02 10:35:27,157 INFO repeater task os150bb_10pings starting 1.4 seconds late 2017-06-02 10:35:27,303 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 10:35:27,303 INFO repeater task os150nb_10pings starting 1.5 seconds late 2017-06-02 10:35:27,404 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 10:38:27,749 INFO repeater task os150bb_quick starting 2.0 seconds late 2017-06-02 10:38:31,027 INFO repeater task os150bb_quick ended after 3.3 seconds 2017-06-02 10:38:31,027 INFO repeater task os150bb_3dayplots starting 5.2 seconds late 2017-06-02 10:38:37,828 INFO repeater task os150bb_3dayplots ended after 6.8 seconds 2017-06-02 10:38:37,828 INFO repeater task os150bb_tsstats starting 12.0 seconds late 2017-06-02 10:38:38,456 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-02 10:38:38,457 INFO repeater task os150nb_lastensq starting 2.5 seconds late 2017-06-02 10:38:39,646 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 10:38:39,647 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 10:38:26 2017-06-02 10:38:39,647 INFO repeater task os150nb_quick starting 13.9 seconds late 2017-06-02 10:38:42,925 INFO repeater task os150nb_quick ended after 3.3 seconds 2017-06-02 10:38:42,925 INFO repeater task os150nb_3dayplots starting 17.1 seconds late 2017-06-02 10:38:50,315 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-02 10:38:50,315 INFO repeater task os150nb_3dayplots ended after 7.4 seconds 2017-06-02 10:38:50,316 INFO repeater task os150nb_tsstats starting 24.5 seconds late 2017-06-02 10:38:50,884 INFO repeater task os150nb_tsstats output: /usr/lib/python2.7/dist-packages/numpy/ma/core.py:1073: RuntimeWarning: overflow encountered in true_divide result = self.f(da, db, *args, **kwargs) 2017-06-02 10:38:50,884 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-02 10:38:50,884 INFO repeater task os150bb_beamplots starting 25.1 seconds late 2017-06-02 10:38:51,870 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 10:38:51,870 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-02 10:38:51,871 INFO repeater task os150bb_beamplots starting 26.1 seconds late 2017-06-02 10:38:52,820 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 10:38:52,820 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-02 10:38:52,820 INFO repeater task os150nb_beamplots starting 27.0 seconds late 2017-06-02 10:38:53,797 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 10:38:53,797 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-02 10:38:53,798 INFO repeater task os150nb_beamplots starting 28.0 seconds late 2017-06-02 10:38:54,797 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 10:38:54,798 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-02 10:38:54,798 INFO repeater task posmv_hcorrstats starting 29.0 seconds late 2017-06-02 10:38:55,623 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 10:38:55,623 INFO repeater task posmv_plotposmv starting 29.8 seconds late 2017-06-02 10:38:56,249 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 10:39:01,254 INFO repeater task os150bb_lastensq starting 25.3 seconds late 2017-06-02 10:39:02,999 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-02 10:39:02,999 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 10:38:26 2017-06-02 10:40:48,198 INFO repeater task os150bb_10pings starting 2.4 seconds late 2017-06-02 10:40:48,363 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 10:40:48,364 INFO repeater task os150nb_10pings starting 2.6 seconds late 2017-06-02 10:40:48,457 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 10:43:28,783 INFO repeater task posmv_hcorrstats starting 3.0 seconds late 2017-06-02 10:43:29,609 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 10:43:29,610 INFO repeater task posmv_plotposmv starting 3.8 seconds late 2017-06-02 10:43:30,219 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 10:43:40,234 INFO repeater task os150bb_lastensq starting 4.2 seconds late 2017-06-02 10:43:41,937 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-02 10:43:41,937 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 10:43:26 2017-06-02 10:43:41,938 INFO repeater task os150nb_lastensq starting 5.9 seconds late 2017-06-02 10:43:43,104 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 10:43:43,104 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 10:43:26 2017-06-02 10:46:08,441 INFO repeater task os150bb_10pings starting 2.7 seconds late 2017-06-02 10:46:08,605 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 10:46:08,606 INFO repeater task os150nb_10pings starting 2.8 seconds late 2017-06-02 10:46:08,699 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 10:48:28,983 INFO repeater task posmv_hcorrstats starting 3.2 seconds late 2017-06-02 10:48:29,885 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 10:48:29,885 INFO repeater task posmv_plotposmv starting 4.1 seconds late 2017-06-02 10:48:30,493 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 10:48:40,510 INFO repeater task os150bb_lastensq starting 4.5 seconds late 2017-06-02 10:48:42,158 INFO repeater task os150bb_lastensq ended after 1.6 seconds 2017-06-02 10:48:42,158 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 10:48:26 2017-06-02 10:48:42,158 INFO repeater task os150nb_lastensq starting 6.2 seconds late 2017-06-02 10:48:43,330 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 10:48:43,330 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 10:48:26 2017-06-02 10:51:28,637 INFO repeater task os150bb_10pings starting 2.9 seconds late 2017-06-02 10:51:28,754 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 10:51:28,755 INFO repeater task os150nb_10pings starting 3.0 seconds late 2017-06-02 10:51:28,846 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 10:53:29,050 INFO repeater task os150bb_quick starting 3.3 seconds late 2017-06-02 10:53:32,154 INFO repeater task os150bb_quick ended after 3.1 seconds 2017-06-02 10:53:32,154 INFO repeater task os150bb_3dayplots starting 6.4 seconds late 2017-06-02 10:53:38,785 INFO repeater task os150bb_3dayplots ended after 6.6 seconds 2017-06-02 10:53:38,785 INFO repeater task os150bb_tsstats starting 13.0 seconds late 2017-06-02 10:53:39,348 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-02 10:53:39,348 INFO repeater task os150nb_lastensq starting 3.3 seconds late 2017-06-02 10:53:40,549 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 10:53:40,549 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 10:53:26 2017-06-02 10:53:40,550 INFO repeater task os150nb_quick starting 14.8 seconds late 2017-06-02 10:53:43,598 INFO repeater task os150nb_quick ended after 3.0 seconds 2017-06-02 10:53:43,599 INFO repeater task os150nb_3dayplots starting 17.8 seconds late 2017-06-02 10:53:50,313 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-02 10:53:50,313 INFO repeater task os150nb_3dayplots ended after 6.7 seconds 2017-06-02 10:53:50,313 INFO repeater task os150nb_tsstats starting 24.5 seconds late 2017-06-02 10:53:50,867 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-02 10:53:50,867 INFO repeater task os150bb_beamplots starting 25.1 seconds late 2017-06-02 10:53:51,763 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 10:53:51,763 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-02 10:53:51,764 INFO repeater task os150bb_beamplots starting 26.0 seconds late 2017-06-02 10:53:52,673 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 10:53:52,673 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-02 10:53:52,674 INFO repeater task os150nb_beamplots starting 26.9 seconds late 2017-06-02 10:53:53,626 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 10:53:53,626 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-02 10:53:53,627 INFO repeater task os150nb_beamplots starting 27.8 seconds late 2017-06-02 10:53:54,563 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 10:53:54,563 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 10:53:54,564 INFO repeater task posmv_hcorrstats starting 28.8 seconds late 2017-06-02 10:53:55,399 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 10:53:55,400 INFO repeater task posmv_plotposmv starting 29.6 seconds late 2017-06-02 10:53:56,007 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 10:54:01,012 INFO repeater task os150bb_lastensq starting 25.0 seconds late 2017-06-02 10:54:02,877 INFO repeater task os150bb_lastensq ended after 1.9 seconds 2017-06-02 10:54:02,878 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 10:53:26 2017-06-02 10:56:48,224 INFO repeater task os150bb_10pings starting 2.4 seconds late 2017-06-02 10:56:48,396 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 10:56:48,397 INFO repeater task os150nb_10pings starting 2.6 seconds late 2017-06-02 10:56:48,487 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 10:58:28,691 INFO repeater task posmv_hcorrstats starting 2.9 seconds late 2017-06-02 10:58:29,616 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 10:58:29,617 INFO repeater task posmv_plotposmv starting 3.8 seconds late 2017-06-02 10:58:30,233 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 10:58:40,246 INFO repeater task os150bb_lastensq starting 4.2 seconds late 2017-06-02 10:58:41,900 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-02 10:58:41,900 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 10:58:26 2017-06-02 10:58:41,901 INFO repeater task os150nb_lastensq starting 5.9 seconds late 2017-06-02 10:58:43,078 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 10:58:43,078 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 10:58:26 2017-06-02 11:02:08,555 INFO repeater task os150bb_10pings starting 2.8 seconds late 2017-06-02 11:02:08,716 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 11:02:08,717 INFO repeater task os150nb_10pings starting 2.9 seconds late 2017-06-02 11:02:08,818 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 11:03:28,984 INFO repeater task posmv_hcorrstats starting 3.2 seconds late 2017-06-02 11:03:29,898 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 11:03:29,898 INFO repeater task posmv_plotposmv starting 4.1 seconds late 2017-06-02 11:03:30,543 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 11:03:40,557 INFO repeater task os150bb_lastensq starting 4.6 seconds late 2017-06-02 11:03:42,426 INFO repeater task os150bb_lastensq ended after 1.9 seconds 2017-06-02 11:03:42,426 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 11:03:26 2017-06-02 11:03:42,427 INFO repeater task os150nb_lastensq starting 6.4 seconds late 2017-06-02 11:03:43,790 INFO repeater task os150nb_lastensq ended after 1.4 seconds 2017-06-02 11:03:43,791 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 11:03:26 2017-06-02 11:07:29,285 INFO repeater task os150bb_10pings starting 3.5 seconds late 2017-06-02 11:07:29,449 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 11:07:29,450 INFO repeater task os150nb_10pings starting 3.7 seconds late 2017-06-02 11:07:29,539 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 11:08:29,656 INFO repeater task os150bb_quick starting 3.9 seconds late 2017-06-02 11:08:32,759 INFO repeater task os150bb_quick ended after 3.1 seconds 2017-06-02 11:08:32,759 INFO repeater task os150bb_3dayplots starting 7.0 seconds late 2017-06-02 11:08:39,441 INFO repeater task os150bb_3dayplots ended after 6.7 seconds 2017-06-02 11:08:39,441 INFO repeater task os150bb_tsstats starting 13.7 seconds late 2017-06-02 11:08:39,985 INFO repeater task os150bb_tsstats ended after 0.5 seconds 2017-06-02 11:08:39,986 INFO repeater task os150nb_lastensq starting 4.0 seconds late 2017-06-02 11:08:41,155 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 11:08:41,155 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 11:08:26 2017-06-02 11:08:41,156 INFO repeater task os150nb_quick starting 15.4 seconds late 2017-06-02 11:08:44,331 INFO repeater task os150nb_quick ended after 3.2 seconds 2017-06-02 11:08:44,331 INFO repeater task os150nb_3dayplots starting 18.6 seconds late 2017-06-02 11:08:51,004 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-02 11:08:51,004 INFO repeater task os150nb_3dayplots ended after 6.7 seconds 2017-06-02 11:08:51,005 INFO repeater task os150nb_tsstats starting 25.2 seconds late 2017-06-02 11:08:51,540 INFO repeater task os150nb_tsstats output: /usr/lib/python2.7/dist-packages/numpy/ma/core.py:1073: RuntimeWarning: overflow encountered in true_divide result = self.f(da, db, *args, **kwargs) 2017-06-02 11:08:51,540 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2017-06-02 11:08:51,540 INFO repeater task os150bb_beamplots starting 25.8 seconds late 2017-06-02 11:08:52,441 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 11:08:52,441 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-02 11:08:52,441 INFO repeater task os150bb_beamplots starting 26.7 seconds late 2017-06-02 11:08:53,384 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 11:08:53,384 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-02 11:08:53,384 INFO repeater task os150nb_beamplots starting 27.6 seconds late 2017-06-02 11:08:54,285 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 11:08:54,285 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 11:08:54,286 INFO repeater task os150nb_beamplots starting 28.5 seconds late 2017-06-02 11:08:55,202 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 11:08:55,202 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 11:08:55,202 INFO repeater task posmv_hcorrstats starting 29.4 seconds late 2017-06-02 11:08:56,002 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 11:08:56,002 INFO repeater task posmv_plotposmv starting 30.2 seconds late 2017-06-02 11:08:56,623 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 11:09:01,628 INFO repeater task os150bb_lastensq starting 25.6 seconds late 2017-06-02 11:09:03,417 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 11:09:03,417 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 11:08:26 2017-06-02 11:12:48,824 INFO repeater task os150bb_10pings starting 3.0 seconds late 2017-06-02 11:12:48,946 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 11:12:48,947 INFO repeater task os150nb_10pings starting 3.2 seconds late 2017-06-02 11:12:49,039 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 11:13:29,121 INFO repeater task posmv_hcorrstats starting 3.3 seconds late 2017-06-02 11:13:30,039 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 11:13:30,040 INFO repeater task posmv_plotposmv starting 4.3 seconds late 2017-06-02 11:13:30,655 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 11:13:40,672 INFO repeater task os150bb_lastensq starting 4.7 seconds late 2017-06-02 11:13:42,515 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 11:13:42,516 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 11:13:26 2017-06-02 11:13:42,516 INFO repeater task os150nb_lastensq starting 6.5 seconds late 2017-06-02 11:13:43,765 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 11:13:43,765 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 11:13:26 2017-06-02 11:18:09,322 INFO repeater task os150bb_10pings starting 3.5 seconds late 2017-06-02 11:18:09,418 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 11:18:09,419 INFO repeater task os150nb_10pings starting 3.6 seconds late 2017-06-02 11:18:09,512 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 11:18:29,543 INFO repeater task posmv_hcorrstats starting 3.8 seconds late 2017-06-02 11:18:30,491 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 11:18:30,491 INFO repeater task posmv_plotposmv starting 4.7 seconds late 2017-06-02 11:18:31,144 INFO repeater task posmv_plotposmv ended after 0.7 seconds 2017-06-02 11:18:36,149 INFO repeater task os150bb_lastensq starting 0.1 seconds late 2017-06-02 11:18:38,067 INFO repeater task os150bb_lastensq ended after 1.9 seconds 2017-06-02 11:18:38,067 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 11:18:26 2017-06-02 11:18:38,068 INFO repeater task os150nb_lastensq starting 2.1 seconds late 2017-06-02 11:18:39,337 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-02 11:18:39,338 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 11:18:26 2017-06-02 11:23:29,932 INFO repeater task os150bb_10pings starting 4.2 seconds late 2017-06-02 11:23:30,103 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 11:23:30,104 INFO repeater task os150bb_quick starting 4.3 seconds late 2017-06-02 11:23:33,239 INFO repeater task os150bb_quick ended after 3.1 seconds 2017-06-02 11:23:33,240 INFO repeater task os150bb_3dayplots starting 7.5 seconds late 2017-06-02 11:23:39,903 INFO repeater task os150bb_3dayplots ended after 6.7 seconds 2017-06-02 11:23:39,903 INFO repeater task os150bb_tsstats starting 14.1 seconds late 2017-06-02 11:23:40,439 INFO repeater task os150bb_tsstats ended after 0.5 seconds 2017-06-02 11:23:40,439 INFO repeater task os150nb_lastensq starting 4.4 seconds late 2017-06-02 11:23:41,622 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 11:23:41,622 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 11:23:26 2017-06-02 11:23:41,622 INFO repeater task os150nb_10pings starting 15.8 seconds late 2017-06-02 11:23:41,712 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 11:23:41,713 INFO repeater task os150nb_quick starting 15.9 seconds late 2017-06-02 11:23:44,809 INFO repeater task os150nb_quick ended after 3.1 seconds 2017-06-02 11:23:44,809 INFO repeater task os150nb_3dayplots starting 19.0 seconds late 2017-06-02 11:23:51,510 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-02 11:23:51,510 INFO repeater task os150nb_3dayplots ended after 6.7 seconds 2017-06-02 11:23:51,510 INFO repeater task os150nb_tsstats starting 25.7 seconds late 2017-06-02 11:23:52,047 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2017-06-02 11:23:52,048 INFO repeater task os150bb_beamplots starting 26.3 seconds late 2017-06-02 11:23:53,000 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 11:23:53,000 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-02 11:23:53,001 INFO repeater task os150bb_beamplots starting 27.2 seconds late 2017-06-02 11:23:53,975 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 11:23:53,975 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-02 11:23:53,975 INFO repeater task os150nb_beamplots starting 28.2 seconds late 2017-06-02 11:23:54,890 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 11:23:54,890 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 11:23:54,891 INFO repeater task os150nb_beamplots starting 29.1 seconds late 2017-06-02 11:23:55,792 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 11:23:55,792 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 11:23:55,792 INFO repeater task posmv_hcorrstats starting 30.0 seconds late 2017-06-02 11:23:56,609 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 11:23:56,609 INFO repeater task posmv_plotposmv starting 30.8 seconds late 2017-06-02 11:23:57,219 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 11:24:02,225 INFO repeater task os150bb_lastensq starting 26.2 seconds late 2017-06-02 11:24:03,950 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-02 11:24:03,950 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 11:23:26 2017-06-02 11:28:29,587 INFO repeater task posmv_hcorrstats starting 3.8 seconds late 2017-06-02 11:28:30,530 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 11:28:30,530 INFO repeater task posmv_plotposmv starting 4.7 seconds late 2017-06-02 11:28:31,131 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 11:28:36,137 INFO repeater task os150bb_lastensq starting 0.1 seconds late 2017-06-02 11:28:37,820 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-02 11:28:37,821 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 11:28:26 2017-06-02 11:28:37,821 INFO repeater task os150nb_lastensq starting 1.8 seconds late 2017-06-02 11:28:39,007 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 11:28:39,007 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 11:28:26 2017-06-02 11:28:49,021 INFO repeater task os150bb_10pings starting 3.2 seconds late 2017-06-02 11:28:49,143 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 11:28:49,144 INFO repeater task os150nb_10pings starting 3.4 seconds late 2017-06-02 11:28:49,233 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 11:33:29,790 INFO repeater task posmv_hcorrstats starting 4.0 seconds late 2017-06-02 11:33:30,704 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 11:33:30,704 INFO repeater task posmv_plotposmv starting 4.9 seconds late 2017-06-02 11:33:31,351 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 11:33:36,357 INFO repeater task os150bb_lastensq starting 0.4 seconds late 2017-06-02 11:33:38,136 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 11:33:38,137 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 11:33:26 2017-06-02 11:33:38,137 INFO repeater task os150nb_lastensq starting 2.1 seconds late 2017-06-02 11:33:39,436 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-02 11:33:39,437 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 11:33:26 2017-06-02 11:34:09,510 INFO repeater task os150bb_10pings starting 3.7 seconds late 2017-06-02 11:34:09,633 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 11:34:09,634 INFO repeater task os150nb_10pings starting 3.9 seconds late 2017-06-02 11:34:09,727 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 11:38:30,268 INFO repeater task os150bb_quick starting 4.5 seconds late 2017-06-02 11:38:33,514 INFO repeater task os150bb_quick ended after 3.2 seconds 2017-06-02 11:38:33,515 INFO repeater task os150bb_3dayplots starting 7.7 seconds late 2017-06-02 11:38:40,568 INFO repeater task os150bb_3dayplots ended after 7.1 seconds 2017-06-02 11:38:40,569 INFO repeater task os150bb_tsstats starting 14.8 seconds late 2017-06-02 11:38:41,125 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-02 11:38:41,126 INFO repeater task os150nb_lastensq starting 5.1 seconds late 2017-06-02 11:38:42,387 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-02 11:38:42,387 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 11:38:26 2017-06-02 11:38:42,387 INFO repeater task os150nb_quick starting 16.6 seconds late 2017-06-02 11:38:45,645 INFO repeater task os150nb_quick ended after 3.3 seconds 2017-06-02 11:38:45,645 INFO repeater task os150nb_3dayplots starting 19.9 seconds late 2017-06-02 11:38:52,530 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-02 11:38:52,530 INFO repeater task os150nb_3dayplots ended after 6.9 seconds 2017-06-02 11:38:52,530 INFO repeater task os150nb_tsstats starting 26.7 seconds late 2017-06-02 11:38:53,082 INFO repeater task os150nb_tsstats output: /usr/lib/python2.7/dist-packages/numpy/ma/core.py:1073: RuntimeWarning: overflow encountered in true_divide result = self.f(da, db, *args, **kwargs) 2017-06-02 11:38:53,082 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-02 11:38:53,082 INFO repeater task os150bb_beamplots starting 27.3 seconds late 2017-06-02 11:38:53,986 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 11:38:53,986 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-02 11:38:53,986 INFO repeater task os150bb_beamplots starting 28.2 seconds late 2017-06-02 11:38:54,974 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 11:38:54,974 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-02 11:38:54,975 INFO repeater task os150nb_beamplots starting 29.2 seconds late 2017-06-02 11:38:55,987 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 11:38:55,987 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-02 11:38:55,987 INFO repeater task os150nb_beamplots starting 30.2 seconds late 2017-06-02 11:38:56,983 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 11:38:56,983 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-02 11:38:56,984 INFO repeater task posmv_hcorrstats starting 31.2 seconds late 2017-06-02 11:38:57,831 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 11:38:57,832 INFO repeater task posmv_plotposmv starting 32.0 seconds late 2017-06-02 11:38:58,489 INFO repeater task posmv_plotposmv ended after 0.7 seconds 2017-06-02 11:39:03,493 INFO repeater task os150bb_lastensq starting 27.5 seconds late 2017-06-02 11:39:05,294 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 11:39:05,294 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 11:38:26 2017-06-02 11:39:30,341 INFO repeater task os150bb_10pings starting 4.6 seconds late 2017-06-02 11:39:30,516 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 11:39:30,517 INFO repeater task os150nb_10pings starting 4.7 seconds late 2017-06-02 11:39:30,609 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 11:43:26,083 INFO repeater task posmv_hcorrstats starting 0.3 seconds late 2017-06-02 11:43:27,031 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 11:43:27,031 INFO repeater task posmv_plotposmv starting 1.2 seconds late 2017-06-02 11:43:27,638 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 11:43:37,655 INFO repeater task os150bb_lastensq starting 1.7 seconds late 2017-06-02 11:43:39,451 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 11:43:39,451 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 11:43:26 2017-06-02 11:43:39,452 INFO repeater task os150nb_lastensq starting 3.5 seconds late 2017-06-02 11:43:40,635 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 11:43:40,635 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 11:43:26 2017-06-02 11:44:50,771 INFO repeater task os150bb_10pings starting 5.0 seconds late 2017-06-02 11:44:50,939 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 11:44:50,940 INFO repeater task os150nb_10pings starting 5.2 seconds late 2017-06-02 11:44:51,031 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 11:48:26,575 INFO repeater task posmv_hcorrstats starting 0.8 seconds late 2017-06-02 11:48:27,415 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 11:48:27,415 INFO repeater task posmv_plotposmv starting 1.6 seconds late 2017-06-02 11:48:28,020 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 11:48:38,036 INFO repeater task os150bb_lastensq starting 2.0 seconds late 2017-06-02 11:48:39,829 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 11:48:39,829 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 11:48:26 2017-06-02 11:48:39,830 INFO repeater task os150nb_lastensq starting 3.8 seconds late 2017-06-02 11:48:40,981 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 11:48:40,981 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 11:48:26 2017-06-02 11:50:06,138 INFO repeater task os150bb_10pings starting 0.4 seconds late 2017-06-02 11:50:06,272 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 11:50:06,273 INFO repeater task os150nb_10pings starting 0.5 seconds late 2017-06-02 11:50:06,378 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 11:53:26,829 INFO repeater task os150bb_quick starting 1.0 seconds late 2017-06-02 11:53:30,051 INFO repeater task os150bb_quick ended after 3.2 seconds 2017-06-02 11:53:30,052 INFO repeater task os150bb_3dayplots starting 4.3 seconds late 2017-06-02 11:53:36,706 INFO repeater task os150bb_3dayplots ended after 6.7 seconds 2017-06-02 11:53:36,706 INFO repeater task os150bb_tsstats starting 10.9 seconds late 2017-06-02 11:53:37,265 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-02 11:53:37,265 INFO repeater task os150nb_lastensq starting 1.3 seconds late 2017-06-02 11:53:38,453 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 11:53:38,453 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 11:53:26 2017-06-02 11:53:38,454 INFO repeater task os150nb_quick starting 12.7 seconds late 2017-06-02 11:53:41,593 INFO repeater task os150nb_quick ended after 3.1 seconds 2017-06-02 11:53:41,593 INFO repeater task os150nb_3dayplots starting 15.8 seconds late 2017-06-02 11:53:48,280 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-02 11:53:48,281 INFO repeater task os150nb_3dayplots ended after 6.7 seconds 2017-06-02 11:53:48,281 INFO repeater task os150nb_tsstats starting 22.5 seconds late 2017-06-02 11:53:48,828 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2017-06-02 11:53:48,828 INFO repeater task os150bb_beamplots starting 23.0 seconds late 2017-06-02 11:53:49,784 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 11:53:49,784 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-02 11:53:49,784 INFO repeater task os150bb_beamplots starting 24.0 seconds late 2017-06-02 11:53:50,705 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 11:53:50,706 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-02 11:53:50,706 INFO repeater task os150nb_beamplots starting 24.9 seconds late 2017-06-02 11:53:51,658 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 11:53:51,658 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-02 11:53:51,658 INFO repeater task os150nb_beamplots starting 25.9 seconds late 2017-06-02 11:53:52,573 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 11:53:52,573 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 11:53:52,573 INFO repeater task posmv_hcorrstats starting 26.8 seconds late 2017-06-02 11:53:53,383 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 11:53:53,384 INFO repeater task posmv_plotposmv starting 27.6 seconds late 2017-06-02 11:53:53,984 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 11:53:58,990 INFO repeater task os150bb_lastensq starting 23.0 seconds late 2017-06-02 11:54:00,871 INFO repeater task os150bb_lastensq ended after 1.9 seconds 2017-06-02 11:54:00,871 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 11:53:26 2017-06-02 11:55:26,042 INFO repeater task os150bb_10pings starting 0.3 seconds late 2017-06-02 11:55:26,208 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 11:55:26,209 INFO repeater task os150nb_10pings starting 0.4 seconds late 2017-06-02 11:55:26,302 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 11:58:26,768 INFO repeater task posmv_hcorrstats starting 1.0 seconds late 2017-06-02 11:58:27,602 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 11:58:27,603 INFO repeater task posmv_plotposmv starting 1.8 seconds late 2017-06-02 11:58:28,220 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 11:58:38,236 INFO repeater task os150bb_lastensq starting 2.2 seconds late 2017-06-02 11:58:39,951 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-02 11:58:39,951 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 11:58:26 2017-06-02 11:58:39,952 INFO repeater task os150nb_lastensq starting 4.0 seconds late 2017-06-02 11:58:41,105 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 11:58:41,105 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 11:58:26 2017-06-02 12:00:46,358 INFO repeater task os150bb_10pings starting 0.6 seconds late 2017-06-02 12:00:46,526 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 12:00:46,527 INFO repeater task os150nb_10pings starting 0.7 seconds late 2017-06-02 12:00:46,618 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 12:03:26,951 INFO repeater task posmv_hcorrstats starting 1.2 seconds late 2017-06-02 12:03:27,803 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 12:03:27,804 INFO repeater task posmv_plotposmv starting 2.0 seconds late 2017-06-02 12:03:28,424 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 12:03:38,442 INFO repeater task os150bb_lastensq starting 2.4 seconds late 2017-06-02 12:03:40,193 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 12:03:40,193 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 12:03:26 2017-06-02 12:03:40,193 INFO repeater task os150nb_lastensq starting 4.2 seconds late 2017-06-02 12:03:41,367 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 12:03:41,367 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 12:03:26 2017-06-02 12:06:06,659 INFO repeater task os150bb_10pings starting 0.9 seconds late 2017-06-02 12:06:06,809 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 12:06:06,810 INFO repeater task os150nb_10pings starting 1.0 seconds late 2017-06-02 12:06:06,903 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 12:08:27,195 INFO repeater task os150bb_quick starting 1.4 seconds late 2017-06-02 12:08:30,352 INFO repeater task os150bb_quick ended after 3.2 seconds 2017-06-02 12:08:30,352 INFO repeater task os150bb_3dayplots starting 4.6 seconds late 2017-06-02 12:08:36,955 INFO repeater task os150bb_3dayplots ended after 6.6 seconds 2017-06-02 12:08:36,955 INFO repeater task os150bb_tsstats starting 11.2 seconds late 2017-06-02 12:08:37,488 INFO repeater task os150bb_tsstats ended after 0.5 seconds 2017-06-02 12:08:37,488 INFO repeater task os150nb_lastensq starting 1.5 seconds late 2017-06-02 12:08:38,700 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 12:08:38,700 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 12:08:26 2017-06-02 12:08:38,700 INFO repeater task os150nb_quick starting 12.9 seconds late 2017-06-02 12:08:41,794 INFO repeater task os150nb_quick ended after 3.1 seconds 2017-06-02 12:08:41,794 INFO repeater task os150nb_3dayplots starting 16.0 seconds late 2017-06-02 12:08:48,534 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-02 12:08:48,535 INFO repeater task os150nb_3dayplots ended after 6.7 seconds 2017-06-02 12:08:48,535 INFO repeater task os150nb_tsstats starting 22.8 seconds late 2017-06-02 12:08:49,087 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-02 12:08:49,087 INFO repeater task os150bb_beamplots starting 23.3 seconds late 2017-06-02 12:08:50,066 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 12:08:50,066 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-02 12:08:50,066 INFO repeater task os150bb_beamplots starting 24.3 seconds late 2017-06-02 12:08:50,992 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 12:08:50,992 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-02 12:08:50,992 INFO repeater task os150nb_beamplots starting 25.2 seconds late 2017-06-02 12:08:51,942 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 12:08:51,942 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 12:08:51,942 INFO repeater task os150nb_beamplots starting 26.2 seconds late 2017-06-02 12:08:52,869 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 12:08:52,869 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 12:08:52,869 INFO repeater task posmv_hcorrstats starting 27.1 seconds late 2017-06-02 12:08:53,715 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 12:08:53,716 INFO repeater task posmv_plotposmv starting 27.9 seconds late 2017-06-02 12:08:54,371 INFO repeater task posmv_plotposmv ended after 0.7 seconds 2017-06-02 12:08:59,372 INFO repeater task os150bb_lastensq starting 23.4 seconds late 2017-06-02 12:09:01,290 INFO repeater task os150bb_lastensq ended after 1.9 seconds 2017-06-02 12:09:01,291 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 12:08:26 2017-06-02 12:11:26,594 INFO repeater task os150bb_10pings starting 0.8 seconds late 2017-06-02 12:11:26,763 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 12:11:26,763 INFO repeater task os150nb_10pings starting 1.0 seconds late 2017-06-02 12:11:26,856 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 12:13:27,106 INFO repeater task posmv_hcorrstats starting 1.3 seconds late 2017-06-02 12:13:28,028 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 12:13:28,028 INFO repeater task posmv_plotposmv starting 2.2 seconds late 2017-06-02 12:13:28,650 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 12:13:38,667 INFO repeater task os150bb_lastensq starting 2.7 seconds late 2017-06-02 12:13:40,475 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 12:13:40,475 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 12:13:26 2017-06-02 12:13:40,477 INFO repeater task os150nb_lastensq starting 4.5 seconds late 2017-06-02 12:13:41,683 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 12:13:41,683 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 12:13:26 2017-06-02 12:16:47,108 INFO repeater task os150bb_10pings starting 1.3 seconds late 2017-06-02 12:16:47,279 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 12:16:47,282 INFO repeater task os150nb_10pings starting 1.5 seconds late 2017-06-02 12:16:47,375 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 12:18:27,633 INFO repeater task posmv_hcorrstats starting 1.9 seconds late 2017-06-02 12:18:28,597 INFO repeater task posmv_hcorrstats ended after 1.0 seconds 2017-06-02 12:18:28,598 INFO repeater task posmv_plotposmv starting 2.8 seconds late 2017-06-02 12:18:29,267 INFO repeater task posmv_plotposmv ended after 0.7 seconds 2017-06-02 12:18:39,289 INFO repeater task os150bb_lastensq starting 3.3 seconds late 2017-06-02 12:18:41,081 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 12:18:41,082 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 12:18:26 2017-06-02 12:18:41,082 INFO repeater task os150nb_lastensq starting 5.1 seconds late 2017-06-02 12:18:42,448 INFO repeater task os150nb_lastensq ended after 1.4 seconds 2017-06-02 12:18:42,449 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 12:18:26 2017-06-02 12:22:07,906 INFO repeater task os150bb_10pings starting 2.1 seconds late 2017-06-02 12:22:08,027 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 12:22:08,028 INFO repeater task os150nb_10pings starting 2.2 seconds late 2017-06-02 12:22:08,126 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 12:23:28,293 INFO repeater task os150bb_quick starting 2.5 seconds late 2017-06-02 12:23:31,498 INFO repeater task os150bb_quick ended after 3.2 seconds 2017-06-02 12:23:31,499 INFO repeater task os150bb_3dayplots starting 5.7 seconds late 2017-06-02 12:23:38,771 INFO repeater task os150bb_3dayplots ended after 7.3 seconds 2017-06-02 12:23:38,772 INFO repeater task os150bb_tsstats starting 13.0 seconds late 2017-06-02 12:23:39,330 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-02 12:23:39,331 INFO repeater task os150nb_lastensq starting 3.3 seconds late 2017-06-02 12:23:40,618 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-02 12:23:40,618 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 12:23:26 2017-06-02 12:23:40,618 INFO repeater task os150nb_quick starting 14.8 seconds late 2017-06-02 12:23:43,754 INFO repeater task os150nb_quick ended after 3.1 seconds 2017-06-02 12:23:43,754 INFO repeater task os150nb_3dayplots starting 18.0 seconds late 2017-06-02 12:23:51,085 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-02 12:23:51,085 INFO repeater task os150nb_3dayplots ended after 7.3 seconds 2017-06-02 12:23:51,085 INFO repeater task os150nb_tsstats starting 25.3 seconds late 2017-06-02 12:23:51,668 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-02 12:23:51,669 INFO repeater task os150bb_beamplots starting 25.9 seconds late 2017-06-02 12:23:52,666 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 12:23:52,666 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-02 12:23:52,666 INFO repeater task os150bb_beamplots starting 26.9 seconds late 2017-06-02 12:23:53,600 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 12:23:53,600 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-02 12:23:53,600 INFO repeater task os150nb_beamplots starting 27.8 seconds late 2017-06-02 12:23:54,502 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 12:23:54,502 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 12:23:54,503 INFO repeater task os150nb_beamplots starting 28.7 seconds late 2017-06-02 12:23:55,420 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 12:23:55,420 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 12:23:55,420 INFO repeater task posmv_hcorrstats starting 29.6 seconds late 2017-06-02 12:23:56,314 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 12:23:56,315 INFO repeater task posmv_plotposmv starting 30.5 seconds late 2017-06-02 12:23:56,945 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 12:24:01,951 INFO repeater task os150bb_lastensq starting 26.0 seconds late 2017-06-02 12:24:03,839 INFO repeater task os150bb_lastensq ended after 1.9 seconds 2017-06-02 12:24:03,840 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 12:23:26 2017-06-02 12:27:29,280 INFO repeater task os150bb_10pings starting 3.5 seconds late 2017-06-02 12:27:29,440 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 12:27:29,441 INFO repeater task os150nb_10pings starting 3.7 seconds late 2017-06-02 12:27:29,532 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 12:28:29,666 INFO repeater task posmv_hcorrstats starting 3.9 seconds late 2017-06-02 12:28:30,580 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 12:28:30,580 INFO repeater task posmv_plotposmv starting 4.8 seconds late 2017-06-02 12:28:31,185 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 12:28:36,189 INFO repeater task os150bb_lastensq starting 0.2 seconds late 2017-06-02 12:28:37,791 INFO repeater task os150bb_lastensq ended after 1.6 seconds 2017-06-02 12:28:37,791 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 12:28:26 2017-06-02 12:28:37,792 INFO repeater task os150nb_lastensq starting 1.8 seconds late 2017-06-02 12:28:38,925 INFO repeater task os150nb_lastensq ended after 1.1 seconds 2017-06-02 12:28:38,925 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 12:28:26 2017-06-02 12:32:49,475 INFO repeater task os150bb_10pings starting 3.7 seconds late 2017-06-02 12:32:49,636 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 12:32:49,637 INFO repeater task os150nb_10pings starting 3.9 seconds late 2017-06-02 12:32:49,730 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 12:33:29,818 INFO repeater task posmv_hcorrstats starting 4.0 seconds late 2017-06-02 12:33:30,763 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 12:33:30,763 INFO repeater task posmv_plotposmv starting 5.0 seconds late 2017-06-02 12:33:31,384 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 12:33:36,390 INFO repeater task os150bb_lastensq starting 0.4 seconds late 2017-06-02 12:33:38,122 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-02 12:33:38,122 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 12:33:26 2017-06-02 12:33:38,123 INFO repeater task os150nb_lastensq starting 2.1 seconds late 2017-06-02 12:33:39,346 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 12:33:39,346 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 12:33:26 2017-06-02 12:38:09,899 INFO repeater task os150bb_10pings starting 4.1 seconds late 2017-06-02 12:38:09,992 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 12:38:09,993 INFO repeater task os150nb_10pings starting 4.2 seconds late 2017-06-02 12:38:10,084 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 12:38:30,120 INFO repeater task os150bb_quick starting 4.3 seconds late 2017-06-02 12:38:33,382 INFO repeater task os150bb_quick ended after 3.3 seconds 2017-06-02 12:38:33,382 INFO repeater task os150bb_3dayplots starting 7.6 seconds late 2017-06-02 12:38:40,045 INFO repeater task os150bb_3dayplots ended after 6.7 seconds 2017-06-02 12:38:40,046 INFO repeater task os150bb_tsstats starting 14.3 seconds late 2017-06-02 12:38:40,602 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-02 12:38:40,602 INFO repeater task os150nb_lastensq starting 4.6 seconds late 2017-06-02 12:38:41,892 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-02 12:38:41,893 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 12:38:26 2017-06-02 12:38:41,893 INFO repeater task os150nb_quick starting 16.1 seconds late 2017-06-02 12:38:45,055 INFO repeater task os150nb_quick ended after 3.2 seconds 2017-06-02 12:38:45,055 INFO repeater task os150nb_3dayplots starting 19.3 seconds late 2017-06-02 12:38:52,330 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-02 12:38:52,331 INFO repeater task os150nb_3dayplots ended after 7.3 seconds 2017-06-02 12:38:52,331 INFO repeater task os150nb_tsstats starting 26.6 seconds late 2017-06-02 12:38:52,909 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-02 12:38:52,909 INFO repeater task os150bb_beamplots starting 27.1 seconds late 2017-06-02 12:38:53,901 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 12:38:53,901 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-02 12:38:53,901 INFO repeater task os150bb_beamplots starting 28.1 seconds late 2017-06-02 12:38:54,841 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 12:38:54,841 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-02 12:38:54,842 INFO repeater task os150nb_beamplots starting 29.1 seconds late 2017-06-02 12:38:55,787 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 12:38:55,787 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 12:38:55,787 INFO repeater task os150nb_beamplots starting 30.0 seconds late 2017-06-02 12:38:56,719 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 12:38:56,719 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 12:38:56,720 INFO repeater task posmv_hcorrstats starting 30.9 seconds late 2017-06-02 12:38:57,601 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 12:38:57,602 INFO repeater task posmv_plotposmv starting 31.8 seconds late 2017-06-02 12:38:58,223 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 12:39:03,229 INFO repeater task os150bb_lastensq starting 27.2 seconds late 2017-06-02 12:39:05,051 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 12:39:05,051 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 12:38:26 2017-06-02 12:43:30,542 INFO repeater task os150bb_10pings starting 4.8 seconds late 2017-06-02 12:43:30,693 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 12:43:30,694 INFO repeater task os150nb_10pings starting 4.9 seconds late 2017-06-02 12:43:30,787 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 12:43:30,788 INFO repeater task posmv_hcorrstats starting 5.0 seconds late 2017-06-02 12:43:31,630 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 12:43:31,630 INFO repeater task posmv_plotposmv starting 5.8 seconds late 2017-06-02 12:43:32,243 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 12:43:37,249 INFO repeater task os150bb_lastensq starting 1.2 seconds late 2017-06-02 12:43:39,005 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 12:43:39,005 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 12:43:26 2017-06-02 12:43:39,006 INFO repeater task os150nb_lastensq starting 3.0 seconds late 2017-06-02 12:43:40,218 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 12:43:40,219 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 12:43:26 2017-06-02 12:48:25,898 INFO repeater task posmv_hcorrstats starting 0.1 seconds late 2017-06-02 12:48:26,815 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 12:48:26,815 INFO repeater task posmv_plotposmv starting 1.0 seconds late 2017-06-02 12:48:27,448 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 12:48:37,460 INFO repeater task os150bb_lastensq starting 1.5 seconds late 2017-06-02 12:48:39,277 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 12:48:39,278 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 12:48:26 2017-06-02 12:48:39,278 INFO repeater task os150nb_lastensq starting 3.3 seconds late 2017-06-02 12:48:40,517 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 12:48:40,518 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 12:48:26 2017-06-02 12:48:50,546 INFO repeater task os150bb_10pings starting 4.8 seconds late 2017-06-02 12:48:50,645 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 12:48:50,646 INFO repeater task os150nb_10pings starting 4.9 seconds late 2017-06-02 12:48:50,744 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 12:53:26,322 INFO repeater task os150bb_quick starting 0.5 seconds late 2017-06-02 12:53:29,614 INFO repeater task os150bb_quick ended after 3.3 seconds 2017-06-02 12:53:29,615 INFO repeater task os150bb_3dayplots starting 3.8 seconds late 2017-06-02 12:53:36,705 INFO repeater task os150bb_3dayplots ended after 7.1 seconds 2017-06-02 12:53:36,706 INFO repeater task os150bb_tsstats starting 10.9 seconds late 2017-06-02 12:53:37,255 INFO repeater task os150bb_tsstats ended after 0.5 seconds 2017-06-02 12:53:37,255 INFO repeater task os150nb_lastensq starting 1.3 seconds late 2017-06-02 12:53:38,480 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 12:53:38,480 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 12:53:26 2017-06-02 12:53:38,480 INFO repeater task os150nb_quick starting 12.7 seconds late 2017-06-02 12:53:41,693 INFO repeater task os150nb_quick ended after 3.2 seconds 2017-06-02 12:53:41,694 INFO repeater task os150nb_3dayplots starting 15.9 seconds late 2017-06-02 12:53:48,425 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-02 12:53:48,426 INFO repeater task os150nb_3dayplots ended after 6.7 seconds 2017-06-02 12:53:48,426 INFO repeater task os150nb_tsstats starting 22.6 seconds late 2017-06-02 12:53:48,992 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-02 12:53:48,992 INFO repeater task os150bb_beamplots starting 23.2 seconds late 2017-06-02 12:53:49,919 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 12:53:49,919 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-02 12:53:49,920 INFO repeater task os150bb_beamplots starting 24.1 seconds late 2017-06-02 12:53:50,896 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 12:53:50,896 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-02 12:53:50,896 INFO repeater task os150nb_beamplots starting 25.1 seconds late 2017-06-02 12:53:51,849 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 12:53:51,849 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-02 12:53:51,850 INFO repeater task os150nb_beamplots starting 26.1 seconds late 2017-06-02 12:53:52,847 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 12:53:52,847 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-02 12:53:52,848 INFO repeater task posmv_hcorrstats starting 27.1 seconds late 2017-06-02 12:53:53,669 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 12:53:53,670 INFO repeater task posmv_plotposmv starting 27.9 seconds late 2017-06-02 12:53:54,306 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 12:53:59,312 INFO repeater task os150bb_lastensq starting 23.3 seconds late 2017-06-02 12:54:01,014 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-02 12:54:01,014 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 12:53:26 2017-06-02 12:54:06,022 INFO repeater task os150bb_10pings starting 0.2 seconds late 2017-06-02 12:54:06,163 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 12:54:06,164 INFO repeater task os150nb_10pings starting 0.4 seconds late 2017-06-02 12:54:06,264 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 12:58:26,842 INFO repeater task posmv_hcorrstats starting 1.1 seconds late 2017-06-02 12:58:27,656 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 12:58:27,656 INFO repeater task posmv_plotposmv starting 1.9 seconds late 2017-06-02 12:58:28,267 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 12:58:38,276 INFO repeater task os150bb_lastensq starting 2.3 seconds late 2017-06-02 12:58:40,127 INFO repeater task os150bb_lastensq ended after 1.9 seconds 2017-06-02 12:58:40,127 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 12:58:26 2017-06-02 12:58:40,128 INFO repeater task os150nb_lastensq starting 4.1 seconds late 2017-06-02 12:58:41,418 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-02 12:58:41,419 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 12:58:26 2017-06-02 12:59:26,599 INFO repeater task os150bb_10pings starting 0.8 seconds late 2017-06-02 12:59:26,748 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 12:59:26,749 INFO repeater task os150nb_10pings starting 1.0 seconds late 2017-06-02 12:59:26,843 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 13:03:27,302 INFO repeater task posmv_hcorrstats starting 1.5 seconds late 2017-06-02 13:03:28,134 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 13:03:28,134 INFO repeater task posmv_plotposmv starting 2.4 seconds late 2017-06-02 13:03:28,804 INFO repeater task posmv_plotposmv ended after 0.7 seconds 2017-06-02 13:03:38,820 INFO repeater task os150bb_lastensq starting 2.8 seconds late 2017-06-02 13:03:40,648 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 13:03:40,648 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 13:03:26 2017-06-02 13:03:40,649 INFO repeater task os150nb_lastensq starting 4.6 seconds late 2017-06-02 13:03:41,877 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 13:03:41,877 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 13:03:26 2017-06-02 13:04:47,009 INFO repeater task os150bb_10pings starting 1.2 seconds late 2017-06-02 13:04:47,181 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 13:04:47,182 INFO repeater task os150nb_10pings starting 1.4 seconds late 2017-06-02 13:04:47,277 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 13:08:27,726 INFO repeater task os150bb_quick starting 1.9 seconds late 2017-06-02 13:08:31,072 INFO repeater task os150bb_quick ended after 3.3 seconds 2017-06-02 13:08:31,072 INFO repeater task os150bb_3dayplots starting 5.3 seconds late 2017-06-02 13:08:38,157 INFO repeater task os150bb_3dayplots ended after 7.1 seconds 2017-06-02 13:08:38,157 INFO repeater task os150bb_tsstats starting 12.4 seconds late 2017-06-02 13:08:38,741 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-02 13:08:38,742 INFO repeater task os150nb_lastensq starting 2.7 seconds late 2017-06-02 13:08:39,965 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 13:08:39,965 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 13:08:26 2017-06-02 13:08:39,966 INFO repeater task os150nb_quick starting 14.2 seconds late 2017-06-02 13:08:43,199 INFO repeater task os150nb_quick ended after 3.2 seconds 2017-06-02 13:08:43,199 INFO repeater task os150nb_3dayplots starting 17.4 seconds late 2017-06-02 13:08:50,122 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-02 13:08:50,122 INFO repeater task os150nb_3dayplots ended after 6.9 seconds 2017-06-02 13:08:50,122 INFO repeater task os150nb_tsstats starting 24.3 seconds late 2017-06-02 13:08:50,681 INFO repeater task os150nb_tsstats output: /usr/lib/python2.7/dist-packages/numpy/ma/core.py:1073: RuntimeWarning: overflow encountered in true_divide result = self.f(da, db, *args, **kwargs) 2017-06-02 13:08:50,681 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-02 13:08:50,681 INFO repeater task os150bb_beamplots starting 24.9 seconds late 2017-06-02 13:08:51,665 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 13:08:51,665 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-02 13:08:51,665 INFO repeater task os150bb_beamplots starting 25.9 seconds late 2017-06-02 13:08:52,582 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 13:08:52,582 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-02 13:08:52,582 INFO repeater task os150nb_beamplots starting 26.8 seconds late 2017-06-02 13:08:53,541 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 13:08:53,542 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-02 13:08:53,542 INFO repeater task os150nb_beamplots starting 27.8 seconds late 2017-06-02 13:08:54,474 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 13:08:54,475 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 13:08:54,475 INFO repeater task posmv_hcorrstats starting 28.7 seconds late 2017-06-02 13:08:55,305 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 13:08:55,305 INFO repeater task posmv_plotposmv starting 29.5 seconds late 2017-06-02 13:08:55,938 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 13:09:00,944 INFO repeater task os150bb_lastensq starting 24.9 seconds late 2017-06-02 13:09:02,747 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 13:09:02,747 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 13:08:26 2017-06-02 13:10:07,873 INFO repeater task os150bb_10pings starting 2.1 seconds late 2017-06-02 13:10:07,969 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 13:10:07,970 INFO repeater task os150nb_10pings starting 2.2 seconds late 2017-06-02 13:10:08,068 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 13:13:28,502 INFO repeater task posmv_hcorrstats starting 2.7 seconds late 2017-06-02 13:13:29,418 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 13:13:29,418 INFO repeater task posmv_plotposmv starting 3.6 seconds late 2017-06-02 13:13:30,042 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 13:13:40,055 INFO repeater task os150bb_lastensq starting 4.1 seconds late 2017-06-02 13:13:41,706 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-02 13:13:41,707 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 13:13:26 2017-06-02 13:13:41,707 INFO repeater task os150nb_lastensq starting 5.7 seconds late 2017-06-02 13:13:42,886 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 13:13:42,886 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 13:13:26 2017-06-02 13:15:28,104 INFO repeater task os150bb_10pings starting 2.3 seconds late 2017-06-02 13:15:28,263 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 13:15:28,264 INFO repeater task os150nb_10pings starting 2.5 seconds late 2017-06-02 13:15:28,355 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 13:18:28,800 INFO repeater task posmv_hcorrstats starting 3.0 seconds late 2017-06-02 13:18:29,705 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 13:18:29,705 INFO repeater task posmv_plotposmv starting 3.9 seconds late 2017-06-02 13:18:30,323 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 13:18:40,339 INFO repeater task os150bb_lastensq starting 4.3 seconds late 2017-06-02 13:18:42,028 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2017-06-02 13:18:42,028 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 13:18:26 2017-06-02 13:18:42,029 INFO repeater task os150nb_lastensq starting 6.0 seconds late 2017-06-02 13:18:43,193 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 13:18:43,193 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 13:18:26 2017-06-02 13:20:48,448 INFO repeater task os150bb_10pings starting 2.7 seconds late 2017-06-02 13:20:48,623 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 13:20:48,624 INFO repeater task os150nb_10pings starting 2.8 seconds late 2017-06-02 13:20:48,715 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 13:23:29,038 INFO repeater task os150bb_quick starting 3.3 seconds late 2017-06-02 13:23:32,316 INFO repeater task os150bb_quick ended after 3.3 seconds 2017-06-02 13:23:32,316 INFO repeater task os150bb_3dayplots starting 6.5 seconds late 2017-06-02 13:23:39,496 INFO repeater task os150bb_3dayplots ended after 7.2 seconds 2017-06-02 13:23:39,497 INFO repeater task os150bb_tsstats starting 13.7 seconds late 2017-06-02 13:23:40,102 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-02 13:23:40,102 INFO repeater task os150nb_lastensq starting 4.1 seconds late 2017-06-02 13:23:41,298 INFO repeater task os150nb_lastensq ended after 1.2 seconds 2017-06-02 13:23:41,298 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 13:23:26 2017-06-02 13:23:41,298 INFO repeater task os150nb_quick starting 15.5 seconds late 2017-06-02 13:23:44,556 INFO repeater task os150nb_quick ended after 3.3 seconds 2017-06-02 13:23:44,556 INFO repeater task os150nb_3dayplots starting 18.8 seconds late 2017-06-02 13:23:51,884 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-02 13:23:51,885 INFO repeater task os150nb_3dayplots ended after 7.3 seconds 2017-06-02 13:23:51,885 INFO repeater task os150nb_tsstats starting 26.1 seconds late 2017-06-02 13:23:52,444 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-02 13:23:52,444 INFO repeater task os150bb_beamplots starting 26.7 seconds late 2017-06-02 13:23:53,433 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 13:23:53,434 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-02 13:23:53,434 INFO repeater task os150bb_beamplots starting 27.7 seconds late 2017-06-02 13:23:54,436 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 13:23:54,437 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-02 13:23:54,437 INFO repeater task os150nb_beamplots starting 28.7 seconds late 2017-06-02 13:23:55,402 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 13:23:55,402 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-02 13:23:55,403 INFO repeater task os150nb_beamplots starting 29.6 seconds late 2017-06-02 13:23:56,362 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 13:23:56,362 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-02 13:23:56,362 INFO repeater task posmv_hcorrstats starting 30.6 seconds late 2017-06-02 13:23:57,176 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 13:23:57,176 INFO repeater task posmv_plotposmv starting 31.4 seconds late 2017-06-02 13:23:57,857 INFO repeater task posmv_plotposmv ended after 0.7 seconds 2017-06-02 13:24:02,862 INFO repeater task os150bb_lastensq starting 26.9 seconds late 2017-06-02 13:24:04,728 INFO repeater task os150bb_lastensq ended after 1.9 seconds 2017-06-02 13:24:04,728 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 13:23:26 2017-06-02 13:26:09,984 INFO repeater task os150bb_10pings starting 4.2 seconds late 2017-06-02 13:26:10,144 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 13:26:10,145 INFO repeater task os150nb_10pings starting 4.4 seconds late 2017-06-02 13:26:10,240 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 13:28:30,540 INFO repeater task posmv_hcorrstats starting 4.8 seconds late 2017-06-02 13:28:31,397 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 13:28:31,397 INFO repeater task posmv_plotposmv starting 5.6 seconds late 2017-06-02 13:28:32,030 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 13:28:37,035 INFO repeater task os150bb_lastensq starting 1.0 seconds late 2017-06-02 13:28:38,849 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2017-06-02 13:28:38,849 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 13:28:26 2017-06-02 13:28:38,850 INFO repeater task os150nb_lastensq starting 2.9 seconds late 2017-06-02 13:28:40,166 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-02 13:28:40,166 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 13:28:26 2017-06-02 13:31:30,453 INFO repeater task os150bb_10pings starting 4.7 seconds late 2017-06-02 13:31:30,572 INFO repeater task os150bb_10pings ended after 0.1 seconds 2017-06-02 13:31:30,572 INFO repeater task os150nb_10pings starting 4.8 seconds late 2017-06-02 13:31:30,668 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 13:33:25,862 INFO repeater task posmv_hcorrstats starting 0.1 seconds late 2017-06-02 13:33:26,797 INFO repeater task posmv_hcorrstats ended after 0.9 seconds 2017-06-02 13:33:26,798 INFO repeater task posmv_plotposmv starting 1.0 seconds late 2017-06-02 13:33:27,416 INFO repeater task posmv_plotposmv ended after 0.6 seconds 2017-06-02 13:33:37,434 INFO repeater task os150bb_lastensq starting 1.4 seconds late 2017-06-02 13:33:39,369 INFO repeater task os150bb_lastensq ended after 1.9 seconds 2017-06-02 13:33:39,369 INFO repeater task os150bb_lastensq: Time from /home/data/NF1704_B/proc/os150bb/load/wrote_ens_time: 2017/06/02 13:33:26 2017-06-02 13:33:39,370 INFO repeater task os150nb_lastensq starting 3.4 seconds late 2017-06-02 13:33:40,636 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-02 13:33:40,636 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 13:33:26 2017-06-02 13:36:45,994 INFO repeater task os150bb_10pings starting 0.2 seconds late 2017-06-02 13:36:46,169 INFO repeater task os150bb_10pings ended after 0.2 seconds 2017-06-02 13:36:46,170 INFO repeater task os150nb_10pings starting 0.4 seconds late 2017-06-02 13:36:46,267 INFO repeater task os150nb_10pings ended after 0.1 seconds 2017-06-02 13:38:26,475 INFO repeater task os150bb_quick starting 0.7 seconds late 2017-06-02 13:38:29,783 INFO repeater task os150bb_quick ended after 3.3 seconds 2017-06-02 13:38:29,783 INFO repeater task os150bb_3dayplots starting 4.0 seconds late 2017-06-02 13:38:36,766 INFO repeater task os150bb_3dayplots ended after 7.0 seconds 2017-06-02 13:38:36,767 INFO repeater task os150bb_tsstats starting 11.0 seconds late 2017-06-02 13:38:37,341 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2017-06-02 13:38:37,342 INFO repeater task os150nb_lastensq starting 1.3 seconds late 2017-06-02 13:38:38,621 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2017-06-02 13:38:38,622 INFO repeater task os150nb_lastensq: Time from /home/data/NF1704_B/proc/os150nb/load/wrote_ens_time: 2017/06/02 13:38:26 2017-06-02 13:38:38,622 INFO repeater task os150nb_quick starting 12.8 seconds late 2017-06-02 13:38:41,880 INFO repeater task os150nb_quick ended after 3.3 seconds 2017-06-02 13:38:41,881 INFO repeater task os150nb_3dayplots starting 16.1 seconds late 2017-06-02 13:38:49,284 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-06-02 13:38:49,284 INFO repeater task os150nb_3dayplots ended after 7.4 seconds 2017-06-02 13:38:49,284 INFO repeater task os150nb_tsstats starting 23.5 seconds late 2017-06-02 13:38:49,855 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2017-06-02 13:38:49,855 INFO repeater task os150bb_beamplots starting 24.1 seconds late 2017-06-02 13:38:50,854 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 13:38:50,854 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2017-06-02 13:38:50,854 INFO repeater task os150bb_beamplots starting 25.1 seconds late 2017-06-02 13:38:51,770 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 13:38:51,771 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2017-06-02 13:38:51,771 INFO repeater task os150nb_beamplots starting 26.0 seconds late 2017-06-02 13:38:52,721 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 13:38:52,721 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2017-06-02 13:38:52,722 INFO repeater task os150nb_beamplots starting 26.9 seconds late 2017-06-02 13:38:53,658 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-06-02 13:38:53,658 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2017-06-02 13:38:53,658 INFO repeater task posmv_hcorrstats starting 27.9 seconds late 2017-06-02 13:38:54,504 INFO repeater task posmv_hcorrstats ended after 0.8 seconds 2017-06-02 13:38:54,504 INFO repeater task posmv_plotposmv starting 28.7 seconds late 2017-06-02 13:38:55,154 INFO repeater task posmv_plotposmv ended after 0.7 seconds 2017-06-02 13:39:00,160 INFO repeater task os150bb_lastensq starting 24.2 seconds late 2017-06-02 13:39:02,011 INFO repeater task os150bb_lastensq ended after 1.9 seconds