Ross McAtamney

Combining SX10 and X7 data - Process Tear-down

Blog Post created by Ross McAtamney on Oct 1, 2020

Think about it, the capabilities of positioning a job and creating some reference scans with an SX10, then filling in the detail with the speed and efficiency of an X7. Both instruments working in unison with one another so when the data comes into TBC it is quickly combined and off you go.

 

Lets look at two cases here which should be fairly standard workflows where this could be used:

- RURAL: Road project where cracks in the bitumen are filled with bitumen which aren't adequately picked up with the SX10 but can be with an X7. Along with this due to the terrain and ease not all scan stations will leveled during pickup against the internal compensater for the X7.

The road has not been closed off during scanning meaning each site where the bitumen has been used to fill cannot be identified as pickup with the SX10, time constraints mean that the SX10 can't used a sole scanner, and positioning requirements mean the X7 can't be used as the sole scanner.

A project like this is used as an example at the end of this post.

 

- URBAN: Urban pickup including an entire multi-level car park, internals and externals, along with the surrounding area.

Again the area has not been closed off during scanning, however there was some field pickup done with the SX10 this time, time constraints mean that the SX10 can't used a sole scanner due to the number of setups needed, and positioning requirements mean the X7 can't be used as the sole scanner.

 

On paper this process works and is quite elegant, however in reality there are some large loopholes that we can get caught in when it comes to processing the data in TBC which should be considered so a successful outcome can be achieved. Along with this there are some issues with some of the processes in TBC meaning the full solution can be a little broken at times.

 

While as a whole the overall process is important, it should be noted the steps in bold below as there are some parts that can potentially have issues in this process:

Project Setup

Import Control (SX10)

Import Additional Data (X7)

Registration

Classify Regions

Process data

Export


PROJECT SETUP:

Set up project.

  • For projects on datum scale factors will be used, as a result needs to be considered here.
  • For Scale only projects, use a Blank project where a factor of 1 is set as the default.
    • This can be where custom scale factors can be set up too if needed.

 

This step is going to be important when considering what happens to the data on import especially with regards to scale factors.

 

 

IMPORT CONTROL (SX10):

Import control / GPS data first if it exists.

  • Make sure that control is set first, if there is no additional control specifications used this step can be ignored.

Import the SX10 and merge with existing control if needed.

  • This is associated with the control import in that we are concerned with the network coming in to set the correct location in TBC.

Perform adjustments and ensure the control is correct if needed.

 

TIP: Wherever possible use the same point name for different measurements to the same point from different data sources, and use the merge or rename point data on import. This will make it easier to control where data is placed on import.

 

This is a standard recommended process for all TBC projects, not just scan data.

 

--- This next part of the process has some particular points that need to be noted to ensure a successful time handling the data in TBC ---

 

IMPORT ADDITIONAL DATA (X7):

Import X7 / other scan data.

  • This process should be taken for imported scan data too from third party imports, especially Gridded / structured data.

NOTE: For importing data from external sources without TBC performing any changes to the data, particularly Scale, it is recommended to use the the Advanced option in the Import menu when importing the file into TBC and select Point Cloud from Unknown source. Otherwise a scale factor may be applied in the background on import.

 

  • TBC will import the data into the centre of the imported data.
  • This will also go great lengths to help make sure no unwanted scale factors will be applied through the shift.

 

BUG: Changing scan colour cannot be undone, however all other changes can be undone.

ENHANCEMENT REQUEST: Allow X7 data to be brought in at the central coordinate system to any other imported data, especially SX10 stations. Currently the X7 data only imports central to the data if GPS points are used, otherwise data is imported at an arbitrary datum location.

 

This is a recommended workflow for data import in general however due to the way TBC handles scales and projections, even in blank projects with a large shift, sometimes changes to the registration can occur which is undesirable. Using this workflow will ensure that the data holds a single centroid location making shifts minimal in the process.

 

REGISTRATION:

Pairwise registration of the X7 data to SX10 data.

  • Preferably use Pairwise Registration of the data, don’t use plane based, and where possible use more than 1 point to define the registration of the data.

 

NOTE: If there are leveled scans in the X7 data, be it one or many, this will force the level of the X7 data to be held constant in Plane-based and Pairwise registration.

This can be addressed by selecting all X7 scans in the Project Explorer and in the properties changing them to unleveled.

Once this is done the registration process will work. 

 

ENHANCEMENT REQUEST: Add a unlock leveling during pairwise and plane based registration option that is tuned on by default (meaning to keep the leveling in the X7 scans you physically have to turn it off), similar to the refinement process. 

Sometimes X7 data can be off and holding the level constant like this is generally pointless when registering data against SX10 data which will most likely be assumed to be correct due to the factors that have gone into registering the data in the field. This process currently doesn't make it immediately obvious that the level has been held as constant causing issues later on.

ENHANCEMENT REQUEST: When selecting scan stations, see if scans taken at that spot are leveled. Currently to turn off the levels of scans you need to select manually each individual scan after expanding each scan station. When using X7 data there can be 50+ scans quite easily and this is an extremely tedious and time consuming process to do it for each individual scan.

EDIT: Retrying this, this can be done already, for some reason I wasn't able to change this in the dateset state I was working with. It would be nice to see this in the project explorer, so stations with leveled scans have a slightly different symbol or something.

 

We need to do better at visualising data from different data sources so we can focus on one type. At the moment we handle everything as an individual scan OR as an entire point cloud. When merging datasets we need to have the ability to easily turn off subsets to be able to work with one data or another when problem solving. 

 

ENHANCEMENT REQUEST: Ability to create registration sets easily. Currently this can be done in the pairwise registration tool, however it is a bit tedious and time consuming. It would be good to just be able to select different scans and add them to a registration set.

ENHANCEMENT REQUEST: Ability to turn on and off registration sets in the View Filter manager. Sometimes there are several scans you want to isolate. Particularly with regards to merging datasets from different instruments. It would be good to be able to turn off the X7 scans or the SX10 scans as a blanket set, or have subsets of scan groups which can be turned off at once so you can quickly just get to an area you want to focus on.

ENHANCEMENT REQUEST: Colourise scans according to Registration sets. This would help identify better what data has come from where in instances where the data has come from different instruments.

 

TIP: When using Pairwise, the optimise pair registration step can be run multiple times to improve results to a point, too many times and they can start to degrade (if this happens hit undo to go back a step).

 

Run a refinement

  • To do a final tightening of the network and get a report out we now need to run a refinement.

 

BUG: Running the refinement step when there is SX10 and X7 data in the project breaks the registration of data apart for some reason returning horrible results. Running this process with just one instrument type seems to work ok, its only when we have a combination it seems to break.

BUG: Output report becomes rubbish as a result of the breaking up of the scans in this process.

BUG: The registration report is 50-50 on if the report produces percentages as a factor of 1000%, sometimes the report works correctly so XX.XX% others in the same dataset its X,XXX.XX%

 

ENHANCEMENT REQUEST: Ability to get a registration report out of pairwise or plane based steps, or some sort of report without needing to run the refinement step first. Not everyone wants to do a refinement, and as seen above this process can break up the registration of the data. In cases where we need to do this step to get the report out, if something goes wrong there is no way to check the data before running this step. Also users may want to check data quality along the way, and currently TBC doesn't have that ability.

 

ABOVE ALL CHECK THE REGISTRATION VISIBLY AGAINST MULTIPLE AREAS OF THE POINT CLOUD.

 

TIP: For road pickups create a surface on the road using the point cloud and ensure that it is flat, if it looks like swiss cheese there may be two levels of data being looked at and the registration process needs to be done again.

 

ENHANCEMENT REQUEST: In the properties of a STATION in TBC show the scale factor that has been applied to data, especially scan data. We show the rest of the data here, and it might not be editable here as this is a project setting, however the ability to see what the scale factor is on the data is important to see that a change has been made. At the moment it is hard to get this information out...if at all.

ENHANCEMENT REQUEST: Similar to SX10 data into blank projects, specify the scale factor for X7 data on import into TBC. 

ENHANCEMENT REQUEST: In the Georeference scans option handle it similar to a site calibration where users can enter specific parameters they want to see in the shift for the data, not just X,Y,Z. Also allowing users to define the centroid parameters, scale factor and location if one is desired.

 

Using the above workflow ensures that the same centroid is being worked off in both datasets and no scale factor will be applied. This process will also hold scale factors better in jobs where one is being used.

 

CLASSIFY:

Run the region classification process

  • It is important this is run after the above checks to ensure everything matches up to give TBC the best chance at correctly registering the data.

TIP: The classify regions tool can be run on regions that have already been classified like the Default region to help get more data classified. Once a subsequent classification is done the data is placed in a new region, this can be added to the main region using the Add to region tool which will also remove the empty region.

 

This step should only be run once the data has been checked and the data is registered correctly.

 

NOTE: If the classification process results are weird / poor then look for separation / ghosting in the point cloud between different scans. If this happens then registration needs to be run again to tighten the point cloud.

 

--- This next part of the step is back to being fairly straight forward. ---

 

PROCESS DATA:

There are many ways this can be done in TBC to a multitude of levels. Multiple bug and enhancement requests have been reported in the forums.

 

EXPORT:

Use one of the export processes to export the data. Multiple bug and enhancement requests have been reported in the forums. This is a specific one to the X7 workflow.

 

BUG: TDX format doesn't appear to export colourised scans that are colourised in TBC. If they are done in the tablet and imported colourised that way then it exports correctly. Its only if the colourisation step is done in TBC that while it views in TBC as true colour it doesn't appear to be exported as such.

 

 

SUMMARY:

The process works and CAN work well, however there are several considerations that need to be made.

  • Do standard processes where control needs to be set before importing additional data.
  • Leveled scans from the X7 have an impact on how the registration process works
  • Scan registration Refinement processes break the registration of scans when using different data sources.
  • Care needs to be taken when creating regions of the scan data, breaking the output report.

 

--//--

RURAL CASE STUDY:

Green SX10

Yellow X7 <- Some leveled stations

TBC cannot register the two data types just from the imported data, below is as close as it gets.

Another example of this the blue is the ground from the classify regions tool. The red is a second pass of the classify regions tool on the resulting original Default layer. A clear indication that something was wrong was the miss-classification of the ground region.

The gap here is 1.3m between the two datasets.

Resulting surface - highlights the issue in another way.

 

IF WE TURN OFF LEVELING:

Registration significantly improves across the project.

This is the result 

 

Running a refinement process to get a report:

Running a refinement on this breaks them apart again:

Outcomes