Extract file

  Extract file types

There are two types of Extract file, the Standard Extract file and the CSV Extract file:

• A Geoida Standard Extract file is a formatted text file and if necessary can be edited to rearrange, modify, delete, insert or combine observations, before final computation and adjustment by the Process Extracted Observations option or for importing into the Traverse or Least Squares Network Adjustment option. The file may be edited within the Geoida View File option, in any external text editor or word processor (such as Windows WordPAD) that can save in text format, or simply by clicking the Edit button in the Process Extracted Observations option after the Extract file required for processing has been selected
  
  
• A Geoida CSV Extract file is a free-format file of comma-separated values (CSV) and may be created by exporting observation data from the Traverse option. A CSV Extract file is intended for export only and not for further use within Geoida. Assuming the observation data has not been changed, an exported CSV Extract file will contain the same observation data as might have been previously imported as a Standard Extract file into the Traverse option

Apart from the difference in format there is otherwise no difference between the structure and content of the Standard and CSV Extract formats - refer to Sample Extract file for further details and examples. Since the CSV format is intended only for export and has no other purpose within Geoida, the discussion that follows below applies specifically to a Standard Extract file only.

NOTE - Initially, an Extract file records a survey's measurements made by the instrument - definition of location, orientation and scale are obtained from other sources. However, orientation may be defined in terms of observed bearings, and if required (for least-squares adjustment) additional 'azimuth' records may be inserted into the Extract file.

While all survey reduction options (except Radiations) are capable of computing and adjusting final point coordinates themselves, the Extract file provides a means of collecting, compiling, editing and further processing traverse and network observations, thus giving the user much greater control over computation and adjustment of survey data.

Return to Section Titles

  Creation of a Standard Extract File

The general steps necessary to produce and process a Standard Extract file are these ...

1. Select a Survey Menu option to process a data file or collection of observations (e-Data or Traverse options) - a Standard Extract file is created (if control survey, traverse or network observations are present)
   
   
2. Edit the Extract file if necessary
   
   
3. Process the Extract file with the Process Extracted Observations option and examine results
   
   
4. Repeat steps 2 and 3 if necessary

Note - an Extract file is NOT created if the data does not contain control survey, traverse or network observations, and the Radiations option does not produce an Extract file.

The following flowchart shows the steps involved in creating and processing Extract files

• Very often, data collected by digital means may not necessarily have been observed in the correct sequence of observations to process and adjust correctly as a traverse - they must be rearranged into a logical and continuous sequence for computation, closure and adjustment
  
  
• Unlike simple single-shot radiation observations, control traverses and networks may consist of multiple or repeated observations that must be meaned
  
  
• Bad or superfluous observations may need to be removed and additional observations may need to be inserted
  
  
• Data for large traverses or networks may need to be combined from different sources for adjustment as a single complete system
  
  
• Radiation observations from control survey stations cannot be computed until the control survey is completed and adjusted to requirements.

Standard Extract files provide the means of achieving these requirements and are processed with the option Process Extracted Observations.

Return to Section Titles

  Naming Convention

Standard Extract files are named according to the job name, and have a filename extension beginning with the '&' character followed by a number, as follows ...

1. Traverse option - a number directly related to the traverse number - for example, processing traverse data set 2, the Extract filename will be jobname.&02
   
   
2. Other options - the numbering begins at 1 but the user will be prompted to either over-write any existing file of the same name or to increment the number to the next available - for example, jobname.&03 is created if jobname.&01 and jobname.&02 already exist and the user wishes to retain them and create an additional new file.

All CSV Extract files are named with the .CSV filename extension.

Return to Section Titles

  'Continue processing' prompt

When processing a data file containing control survey observations with the e-Data option, the following prompt appears after the Extract file has been written, (eg) …

If the observations require editing (for example, a control traverse has not been surveyed in correct sequence and the observations will need to be rearranged to permit normal closure and adjustment), the processing may be halted at this point so that the Extract file may be edited and reorganised as necessary. However, if the data is already correct and there is no editing of the observations required, or to simply achieve a result with the data as-is, answer Yes to the prompt which will continue the current option to its normal conclusion.

Note - The Continue processing ? prompt does not appear within the Traverse option - it is assumed, in this case, that the traverse observations will have been entered in the correct sequence in the first instance; however, the Extract file is still created and may be edited if necessary and processed separately using the Process Extracted Observations option.

It can reasonably be expected that some prior editing of an Extract file will be necessary in most cases, particularly if provisional coordinates are to be computed by a traverse method (eg, the None or Bowditch adjustment modes) prior to least-squares network adjustment (LS_Network mode). If processing is continued in response to the prompt and the results are erroneous or nonsensical, then it is evident that some editing of the Extract file is first necessary.

Once an Extract file has been created or edited, it is then processed using the Process Extracted Observations option.

Return to Section Titles

  Meaning, sorting and correction of control, traverse and network observations

CDS Stadia and Civilcad Neutral files

Only meaned and/or corrected sets of traverse or network observations will be extracted where angle sets can be readily identified (eg Sokkia SDR2x/33 data 'MC' records or TDS Carlson SurvCE BD/BR and FD/FR pairs), and it is assumed that the only observations likely to be entered through the manual Traverse option will be those that can be similarly treated as traverse or network observations collected for the purpose of survey control.

But other data formats such as CDS or Civilcad data may not only be a mixture of general pickup as well as both raw and meaned/corrected control observations, but the control observations may not always be adequately differentiated from the general pickup observations, or a simple single-face traversing procedure may have been used during detail pickup.

Therefore to correctly process CDS and Civilcad data, it is first necessary that

1. If there are any sets of reduced control observations present, they are correctly distinguished from their associated raw observations and other pickup observations by the insertion of appropriate note records (i.e., in the CDS stadia or Neutral file)
2. All related observations in pairs or set of rounds are grouped together so that they will be correctly matched
3. Repeated raw observations, made at the same setup as those processed in a set but to different points, must precede the set observations or will be omitted
4. Multiple observations to the same point but on opposite faces must have matching observations also on opposite faces to the reference object. Observations will be omitted if a matching observation to the RO on the same face cannot be found, or if points are observed on Face 2 but the face for a proposed matching observation to the RO cannot be determined (i.e. no vertical angle was recorded - Face 1 is assumed)
5. Multiple observations to the same point and at the same setup must have identical height of instrument and height of target values or will be treated as distinctly separate observations and will not be meaned

Hence, in order to separate pickup from control, and raw from reduced observations, CDS Stadia and Civilcad Neutral data undergo an additional initial process which is not necessary for other options, with the following process ...

1. Points observed once-only are assumed to be for detail pickup only and are eliminated if:
   • they are not included in a meaned/corrected set of observations
   • they are not already defined as 'fixed' points
   • they occur nowhere else in the data file as either observed, setup or backsight reference points
   • they are not the last observed point in a traverse sequence before another traverse section starts or the data ends, or
   • they are determined NOT to be control points because other control observations have been marked in the data by the insertion of appropriate note records but these have not been so marked (if no other observations were marked as 'sets' then there would be no way of knowing if observations were for pickup or control so they must be retained unless eliminated by some other criterion)
2. If meaned and/or corrected sets of control observations occur at a setup, any raw observations to the same points are eliminated
3. Any remaining raw observations to other points are retained; those that are repeat observations on the SAME face are (corrected to Face 1 and) meaned; those on OPPOSITE faces to the same point (approximately 180 degrees different) are combined in pairs and are converted to a single Face 1 observation. Face 2 vertical angles are converted to Face 1.

For further specific details refer to CDS Stadia or Civilcad Neutral.

General

If the Mean Control Observations check-box (e-Data or Traverse options) is UNcrossed, none of the observations are meaned EXCEPT at each setup where observations have been made to the same point on opposite faces.

However, if the Mean Control Observations check-box is CROSSED, meaning of observations takes place by the following process.

Stage 1 - At each single setup, any multiple or repeated raw observations or Face1/Face2 pointings to the same point, are meaned until only one (meaned) observation to each point remains and are corrected for a zero reading to the backsight; Face 2 vertical angles are converted to Face 1. Set observations are NOT meaned.

Stage 2 - Distance observations are reduced into horizontal and vertical components. For each observation that has not already been used as a reciprocal observation to balance another distance or VC, Geoida searches forward until it finds a suitable matching reverse distance or VC observation that may be used as a reciprocal to eliminate atmospheric and curvature effects. The matching reverse observation is also reduced and the available components are meaned with those of the current forward observation. The reverse observation's horizontal and vertical distance components are changed to the meaned values; it is flagged so that it will not be re-used again later and is marked in the Extract file with '+'. During a preliminary traverse run (None or Bowditch adjustment options) the distance or VC values of an observation marked in this way may be used if necessary to complete a traverse section to enable provisional coordinates to be computed, but will be ignored when reading Extract data for a Least Squares network adjustment. Distance observations for which NO matching reverse observation can be found are marked '*' and curvature/refraction correction will be applied to these observations when the traverse or network is computed. A warning is given (when processing of observations is finished) if paired VCs or vertical angles observed in opposite directions were found to have the same sign and magnitude greater than the curvature/refraction effect.

Note - Observations may occur in a CDS Stadia file as two types, (i) those obtained by the instrument's set-collection program and recorded as a distinct set of corrected and meaned observations (1 record per point), and (ii) those collected as single, Face1/Face2 or repeated raw observations. Geoida does not mix these observation types when meaning. However there may be no distinct provision in a Civilcad Neutral file for indicating that a group of observations constitute a 'set', and it is possible that observations collected in a set may be meaned with repeated single-face observations unless steps are taken to prevent it or the sets can be identified by appropriate notes recorded within the data. Some instrument types will record notes to indicate the extent of a set which Geoida may recognise, or the user may manually edit the neutral file and insert note records at the required positions - refer to Civilcad Neutral for more details. Otherwise, if a Civilcad neutral file has been derived from a total station raw data file containing observations collected by different methods, the first processing of the neutral file should be done with the Mean Control Observations check-box UN-crossed so that the Extract file may be edited to delete the raw observations preceding a 'set' or to otherwise prevent combination of different observation types - refer to Editing an Extract file, and to the section Processing CDS Stadia and Civilcad Neutral control survey data under Total Station derived data in the Survey Options - General Notes topic for more information.

Stage 3 - Mean observations to eliminate duplicated angles and matching distances where there are consecutive setups with the same instrument station and RO. If there have been multiple setups at the same station with the same points observed but on different occasions (hence with different atmospheric conditions and different instrument and prism heights) and Mean Observations is set ON, all these observations will be combined and meaned with all other matching observations over the same lines regardless of occasion or relative location of each setup within the data set. Hence if this is undesirable, Mean Observations should be set OFF - all Face 1/Face 2 and repeated observations made at each single setup are still meaned together regardless of the setting for Mean Observations.

At this stage reduced distance and/or VC observations may be either (i) 1-way observations for which no matching reverse observations were found with which to mean them, or (ii) balanced 2-way "reciprocal" observations. When meaning distance/VC observations over the same line, Geoida does not mean 1-way and 2-way observations together, only those of the same kind are meaned because one-way distances and Ht Diffs have had CnR applied, whereas reciprocal/two-way have not, therefore it would not be strictly correct to mean them; if both 1-way and 2-way observations are found, the 2-way observations are used and the 1-way observations are discarded.

Note: Distances are reduced and meaned only at Stage 2.

Refer to Editing an Extract file for more details.

Stage 4 - If either of the None or Bowditch traverse modes is selected, any observations consisting only of distance, VC or height measurements are merged where possible into angle or bearing observations which do not already have these components, so that continuous traverse routes may be computed progressively using complete observations - any observations remaining 'incomplete' are discarded for traverse computation. This process is not necessary for a least-squares adjustment.

Stage 5 - Observations are output to the Extract file.

Process Extracted Observations option - specifics

Whether or not a Standard Extract file created by another of the Survey menu options is modified, it is separately processed using the Process Extracted Observations option. Notice that this option also has the Mean Control Observations check-box which is used as follows ...

• Mean Control Observations crossed - subjects the observations to further meaning and optimisation, and another new Standard Extract file will be written (Stages 3 to 5 above); processing will be continued to completion. Subsequent re-meaning of this or any other Extract file of previously meaned observations has NO further effect upon them UNLESS they have been re-organised into a different setup sequence as discussed at Stage 3 above. If None or Bowditch traverse mode is selected, distance-, VC- or RL-only observations are merged where possible into angle or bearing observations not having these components, or are discarded.
  
  
• Mean Control Observations NOT crossed - the selected Extract file will be processed as-is to completion and NO new Extract file will be created.

Packing observations

Whether or not meaning is applied, the observations are packed prior to computation and any invalid observations (eg, point number 0 for At or To points) or additional 'observations' created temporarily by Geoida for the purpose of internal sorting and processing, are eliminated.

Traverse continuity check

If the adjustment option is set to None or Bowditch a normal traverse would be computed if processing was continued at the Continue processing with the current option? prompt (see the Continue processing prompt section above).

Because it is necessary for traverse continuity that the last observation at a setup will be to the next instrument station, the observations at each setup are checked and their relative positions may be rearranged. Occasionally there may be more than one control point observed at a setup, so if the next instrument station is one of the points observed at a current setup but it is not the last observation immediately prior to the new setup, the observations will be rearranged so that it is. This will permit any single traverse section to be continued without interruption as far as possible and hopefully to its full extent.

Note - the LS-Network setting computes a least-squares network adjustment which is a completely separate process from traversing and does not require traverse observations to be in a sequential order; therefore with this setting no continuity check is made.

Return to Section Titles