read_autosal_dat_stc02 - Read the stc02 *n.txt files CTD Calibration toolbox INPUT: fileName: path and name of the *n.txt file to be translated. OUTPUT: salts: sample structure containing the averaged autosal analysis. DESCRIPTION: There are multiple ways to approach the salinity sample data provided by the autosal system on the Ron Brown. The RB system uses an automatic logging system which logs ten sample points for each analysis in an ascii file with a *.raw filenaming convention. Typically, the analyst performs three analyses per sample bottle for a total of 30 data points per sample bottle. The computer logging system averages each ten element ensemble into a mean value which is written to the *.xls Excel spreadsheet. The standard deviation of the ten element ensemble is also recorded in the spreadsheet. This procedure is repeated for every sample bottle in the current session. A The beginning and ending points of a session a determined by the user/analyst. Each session must begin and end with a (minimum) three sample standardization using standard seawater of known salinity. The before/after analyses are used to determine and correct for the instrumental drift during the session. If the analyst terminates the session properly, with a post standardization, each sample reading recorded in the *.xls spreadsheets are then corrected automatically by the logging software. There is some indication that the automatic correction procedure performed by the logging software is not perfect. However, this program assumes that the values in the *xls files have been adequately corrected for instrumental drift and are without outlying values. This is partly for pragmmatic concerns to facilitate the software development. Determining the adequacy of the logging software's drift correction is a subject of future work. Please advise me of your experiences so that better procedures can be developed.
0001 function salts = read_autosal_dat_stc02(this_file) 0002 % read_autosal_dat_stc02 - Read the stc02 *n.txt files 0003 % 0004 % CTD Calibration toolbox 0005 % 0006 % INPUT: 0007 % fileName: path and name of the *n.txt file to be translated. 0008 % 0009 % OUTPUT: 0010 % salts: sample structure containing the averaged autosal analysis. 0011 % 0012 % DESCRIPTION: 0013 % There are multiple ways to approach the salinity sample data provided 0014 % by the autosal system on the Ron Brown. The RB system uses an automatic 0015 % logging system which logs ten sample points for each analysis in an ascii file 0016 % with a *.raw filenaming convention. Typically, the analyst performs three 0017 % analyses per sample bottle for a total of 30 data points per sample bottle. 0018 % The computer logging system averages each ten element ensemble into 0019 % a mean value which is written to the *.xls Excel spreadsheet. The standard 0020 % deviation of the ten element ensemble is also recorded in the spreadsheet. 0021 % This procedure is repeated for every sample bottle in the current session. A 0022 % The beginning and ending points of a session a determined by the user/analyst. 0023 % Each session must begin and end with a (minimum) three sample standardization 0024 % using standard seawater of known salinity. The before/after analyses are used 0025 % to determine and correct for the instrumental drift during the session. If the 0026 % analyst terminates the session properly, with a post standardization, each sample 0027 % reading recorded in the *.xls spreadsheets are then corrected automatically by 0028 % the logging software. 0029 % 0030 % There is some indication that the automatic correction procedure performed 0031 % by the logging software is not perfect. However, this program assumes that the 0032 % values in the *xls files have been adequately corrected for instrumental drift 0033 % and are without outlying values. This is partly for pragmmatic concerns to facilitate 0034 % the software development. Determining the adequacy of the logging 0035 % software's drift correction is a subject of future work. Please advise me of your 0036 % experiences so that better procedures can be developed. 0037 0038 % 0039 % CHANGELOG: 0040 % 08-Jul-2004, Version 1.0 0041 % * Initial version. 0042 % 0043 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 0044 0045 error(nargchk(1,1,nargin)) 0046 % This meta-data parsing is specific to these files only.... 0047 year = 2002; 0048 std_water_batch_label = 'P140'; 0049 [pathstr,this_name,ext,versn] = fileparts(this_file); 0050 [mnth,day,startHr] = strread(this_name,'%02d%02d%02dn'); 0051 0052 0053 % First count the number of lines and pre allocated the output cell arrays. 0054 % This is only temporary since we will separate standards from salts later. 0055 nLines = count_lines(this_file); 0056 nLines = nLines - 1; % skip the header line. 0057 if isempty(nLines) 0058 disp(sprintf('Unable to open %s',this_file)) 0059 disp('Passing control to the keyboard for troubleshooting.') 0060 keyboard 0061 else 0062 % Preallocate the output variables. 0063 station_id = NaN.*ones(nLines,1); 0064 fire_order = NaN.*ones(nLines,1); 0065 samp_nbr = station_id; 0066 tank_temp = fire_order; 0067 autosal_ratio = fire_order; % two times conductivity ratio 0068 autosal_ratio_corr = fire_order; % correction due to drift. 0069 cond_ratio = fire_order; 0070 salt = fire_order; 0071 salt_stddev = fire_order; 0072 hours = fire_order; 0073 minutes = hours; 0074 seconds = hours; 0075 niskin_btl_pos = fire_order; 0076 qc_flag = salt; 0077 % standards 0078 stds_station_id = NaN.*ones(nLines,1); 0079 stds_fire_order = NaN.*ones(nLines,1); 0080 stds_samp_nbr = station_id; 0081 stds_tank_temp = fire_order; 0082 stds_autosal_ratio = fire_order; % two times conductivity ratio 0083 stds_autosal_ratio_corr = fire_order; % correction due to drift. 0084 stds_cond_ratio = fire_order; 0085 stds_salt = fire_order; 0086 stds_salt_stddev = fire_order; 0087 stds_hours = fire_order; 0088 stds_minutes = hours; 0089 stds_seconds = hours; 0090 stds_niskin_btl_pos = fire_order; 0091 stds_qc_flag = salt; 0092 end 0093 0094 % Open the file for reading. 0095 fid = fopen(this_file,'r'); 0096 if fid < 0 0097 disp(sprintf('Unable to open %s',this_file)) 0098 disp('Passing control to the keyboard for troubleshooting.') 0099 keyboard 0100 end 0101 0102 % Skip the first header line 0103 theLine = fgetl(fid); 0104 cnt_lines = 0; 0105 cnt_standards = 0; 0106 cnt_salts = 0; 0107 while 1 0108 theLine = fgetl(fid); 0109 cnt_lines = cnt_lines+1; % 0110 if theLine < 0 0111 fclose(fid); % end of file. 0112 break 0113 end 0114 theLine = splitjoin(theLine); 0115 % Now start saving the data.. 0116 % First remove the colons from the time string foor easier parsing 0117 theLine = strsplit(sprintf(' '),rstrrep(theLine,':',' ')); 0118 0119 % This effectively catches the lines with the words standards, disregard, possible error, 0120 % in the comments columns and removes them. 0121 if any(findstr(lower(strjoin(' ',theLine{:})),'standard')) 0122 % remove this 0123 theLine(strmatch('standard',lower(theLine)):end) = []; 0124 end 0125 if any(findstr(lower(strjoin(' ',theLine{:})),'disregard')) 0126 qc = '4'; 0127 theLine(strmatch('disregard',lower(theLine)):end) = []; 0128 elseif any(findstr(lower(strjoin(' ',theLine{:})),'possible')) 0129 qc = '3'; 0130 theLine(strmatch('possible',lower(theLine)):end) = []; 0131 elseif any(findstr(lower(strjoin(' ',theLine{:})),'no h20')) 0132 qc = '3'; 0133 theLine(strmatch('no',lower(theLine)):end) = []; 0134 else 0135 qc = '1'; 0136 end 0137 % NOTE: I added a column to most of the lines here. 0138 theLine = {theLine{:} qc}; 0139 0140 % This traps standard water analysis. 0141 if any(findstr(strjoin(' ',theLine{:}),std_water_batch_label)) 0142 % for this cruise, P140 was the batch label 0143 % the length of the lines are always 13 here except one line which has a comment in the 0144 % last (14th) column. It says Standards. Thanks.... 0145 cnt_standards = cnt_standards + 1; 0146 sample_id = '1000'; 0147 theLine{1} = sample_id; 0148 isstandards = true; 0149 else 0150 cnt_salts = cnt_salts + 1; 0151 isstandards = false; 0152 end 0153 if length(theLine)==14 0154 sample_id = theLine{1}; 0155 end 0156 if length(theLine)==13 0157 theLine = {sample_id theLine{:}}; 0158 end 0159 % At this point the entire file can be assembled with all rows having an equal # of cols 0160 station_id(cnt_lines) = str2num(theLine{1}); 0161 fire_order(cnt_lines) = str2num(theLine{2}); 0162 % NOTE: This requires that all samplebtlnbrs are numeric not alphanumeric. 0163 % There is one on the stc02 cruise 0164 if strcmp(theLine{3},'MG2') 0165 samp_nbr(cnt_lines) = 777; 0166 else 0167 samp_nbr(cnt_lines) = str2num(theLine{3}); 0168 end 0169 %samp_nbr{cnt_lines} = theLine{3}; 0170 tank_temp(cnt_lines) = str2num(theLine{4}); 0171 autosal_ratio(cnt_lines) = str2num(theLine{5}); 0172 autosal_ratio_corr(cnt_lines) = str2num(theLine{6}); % correction due to drift. 0173 cond_ratio(cnt_lines) = str2num(theLine{7}); 0174 salt(cnt_lines) = str2num(theLine{8}); 0175 salt_stddev(cnt_lines) = str2num(theLine{9}); 0176 hours(cnt_lines) = str2num(theLine{10}); 0177 minutes(cnt_lines) = str2num(theLine{11}); 0178 seconds(cnt_lines) = str2num(theLine{12}); 0179 niskin_btl_pos(cnt_lines) = str2num(theLine{13}); 0180 qc_flag(cnt_lines) = str2num(theLine{14}); 0181 % Write it to an ascii file, then reload it and save it as a structure. 0182 0183 end % while looping through this file 0184 % replacing the second standardization bottle number 0185 std_idx=find(samp_nbr==1000); 0186 end_idx=find(diff(std_idx)>1); 0187 end_idx=end_idx+1; 0188 samp_nbr(std_idx(end_idx):length(samp_nbr))=1001; 0189 % 0190 0191 salts.station_id = station_id; 0192 salts.fire_order = fire_order; 0193 salts.samp_nbr = samp_nbr; 0194 salts.tank_temp = tank_temp ; 0195 salts.autosal_ratio = autosal_ratio; 0196 salts.autosal_ratio_corr = autosal_ratio_corr; 0197 salts.cond_ratio = cond_ratio; 0198 salts.salt =salt ; 0199 salts.salt_stddev =salt_stddev ; 0200 salts.hours = hours; 0201 salts.minutes =minutes ; 0202 salts.seconds = seconds; 0203 salts.niskin_btl_pos = niskin_btl_pos; 0204 salts.qc_flag = qc_flag; 0205 salts = trimstruct(salts); 0206 return 0207 0208 0209 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 0210 function out = splitjoin(line) 0211 % splitjoin: Clean up a line of text for easier parsing. 0212 % 0213 0214 line = ddewhite(line); % rm leading/trailing whitespace 0215 noTab = untabify(line); % The stc02 files are tab separated columns. 0216 if ~isempty(noTab) % ...hopefully 0217 %noTab = del_dup_spaces(noTab); 0218 noTab = rstrrep(noTab,' ',' '); 0219 end 0220 line = ddewhite(noTab); % clean ends again. 0221 line = strsplit(sprintf(' '),line); % this is overkill. 0222 line = strjoin(' ',line{:}); % but just in case... 0223 out = line; 0224 return 0225 0226 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 0227 function out = trimstruct(in) 0228 % trimstruct: Trim unused elements from preallocated arrays. 0229 % 0230 0231 theNames = fieldnames(in); 0232 for idx = 1:length(theNames); 0233 theData = in.(theNames{idx}); 0234 if iscell(theData) 0235 outData = {}; 0236 for idxCell = 1:length(theData) 0237 if ~(isnan(theData{idxCell}) | isempty(theData{idxCell})) 0238 outData{idxCell} = theData{idxCell}; 0239 end 0240 end 0241 theData = outData; 0242 else 0243 theData(isnan(theData)) = []; 0244 end 0245 out.(theNames{idx}) = theData; 0246 end 0247 return 0248 0249