Trimble Business Center

In TBC 5.1 the point cloud import has seemed to get a new face lift in which I don't really care for.  I typically fly my sites using a P4 PRO and use Datumate to complete the processing.  In the past when I am on a site for example a borrow site in which I have a calibration completed ( like this one) I will use my rover to shoot in my ground targets then import the points from TBC to use to geo reference the site.  Then I kick out an .las and TBC prior to 5.0 0r 5.1 would import the point cloud data right into place.  Now it is trying to scale stuff and it is a little confusing.  It is my understanding that the .las files has the unit information so why can't TBC just read it and bring it in.  It looks like it is converting the coordinates to metric like the early versions of TBC did.  I tried importing the point clouds into 4.91 and they come in perfect.  So it looks like it is a metric conversion issue to me.  I attached one of the las files.  The project sits at approximately N:524800 E: 55780.  Any help would be great!

Ben

Ben

When you import LAS files into TBC we had to make some changes to accommodate a couple of things that were happening outside our control

1) The Industry Standard LAS file should always be defined in Meters, however different software products added a twist where they wrote a Scale Factor or a Units value into the header to say that it was not in Meters it was in Feet. This broke the standard, and we were in US getting a lot of US Feet version files. We were also getting a number of Unitless Files that we would assume that were in Meters as per the standard, however they were in feet. So if the software writes a Header now that includes a Scale Factor (to convert Metric to feet) or a Units Value (US Feet, International feet or Metric) we now read that and if being imported into a project of the same units we read it directly in, and if not then we convert the units of the file to the units of the project and import it. If the file is unitless we ask for the Units so that you can define them if you know what they are, or in some cases you may need to read it once, realize that the units are off and then reimport using the correct units conversion. This solves all the issues that we had reported, but places a step in the process where we dont know the units of the files. We cannot assume them because if wrong there is no way to fix the issue easily.

2) We also now have the ability to work with Point Clouds in Grid or Ground - Yes we see both. If we assume one or the other we will often be wrong. So we added the step to pick the Grid or Ground option on import. If you know which one you have to work with you are in good shape - just pick one of the two / three options presented -  If you are always using Daatummate, then you should know what it is outputting and correlate that to the correct import settings - and use those over and over.

On your file - I picked US Survey Foot and Advanced User / Option 3 / Unknown / Generic) and got this. I then compared that with v4.12 of BC-HCE and the Coords of points in the Top right Corner (used Measure Point command) match - so this is the same method of import that was in BC-HCE that you used to use.

If this is wrong let me know.

The enclosed Document from the Geospatial Team explains this in some additional detail as to why they did what they did here.

The reason you get asked for Units is because your file is Unitless so we don't know what units to utilize - that must be a Datummate Issue - it was the same in v4.12.

Alan

Alan,

thats good information about why they broke the system, but I still dont see a solution. I cant get typical USGS Lidar .las files to import, and we rely on them greatly. Prior to this update, i was able to generate a site plan w/ topo for any site in ohio in mere minutes... i'm 4 days and now and still can't get this to work. Since the update the typical lidar files are now all being placed somewhere off in the atlantic. Tried "non-georeferenced" no change, tried geo-reference ground scaled, no change and it gives no option to manually adjust scale. Tried geo-referenced grid scaled, but you can only "estimate" scale factors and it wont accept manual entry so is unhelpful. Tried unknown which gives no options so is unhelpful... no option anywhere that i can find allows selecting standard US Survey Foot.   http://gis3.oit.ohio.gov/ZIPARCHIVES/ELEVATION/LIDAR/Ashland/N2030400_LAS.zip 

Alan,

How did you select US Survey Foot and Advanced User / Option 3 / Unknown / Generic?  When I select option 3 with the advance user box checked it doesn't give me and option to tell it US Foot.

Ben

Alan,

can you please REOPEN this question. It should not have been marked ANSWERED, nor should the response above be listed as correct. There has been NO RESOLUTION. Questions shouldn't be marked as ANSWERED untill confirmed be the original submitter

Aaron

I mark questions as answered because I need to track when I / we have

responded to the question with an answer that I / we think answers the

question. To be honest when we answer questions - in probably more than 80

to 90% of cases we never get an acknowledgement that the answer covers the

question - people just tend to move on - so I use that function purely to

say that it has been answered. The answer may not be what the requestor

wanted to hear for sure (that is rare though) and the answer may still not

cover the angles, or provide a perfect workflow - but the question is

always open and you can always ask follow up questions, however the system

does not have anything other than it is answered or unanswered, so while I

kind of agree that in an ideal world you would tell us that the Question

covers the request or not, that very often never happens so I have to be

able to close these out so I don't have to wade my way through hundreds of

apparently unanswered questions on a daily basis.

I have been doing the forum personally for 20 months, and this is the first

time that anyone has said this process is not working, I have also not had

anyone in that 20 months say that we did not answer the question (I / we

try to give you a response / workflow). I very often talk to the requestors

on the phone if I have doubts that we did not cover what they wanted - and

if they tell me they are good then I close it out from my perspective (and

await a follow on question if one should arise). In this case I have talked

to Ben about this issue, and we have covered what he originally asked (I

spoke to him on this again just yesterday).

So please feel free to ask follow on questions or clarify points raised

etc. at any time, and please don't see the flag "Answered" as case closed -

it is not that - it is initial answer that we think covers the question has

been posted but case is still open if you want it to be.

Thanks for the feedback

Alan

On Thu, Oct 10, 2019 at 12:43 PM adearth@simonsonconstruction.com <

Alan,

thanks for the reply.

if the question has been resolved directly w/ Ben, presumably offline to this forum, would you mind copying the resolution into the thread so it can be found in the future. I've been held up for going on two weeks now on this issue, and have two other techs working on it as well. As of now, the only resolution that has been presented to us with is to downgrade back to 4.10 until the .las importing can be fixed back to its previously functional state.

The problem is this

1) the Industry definition of the LAS Format states that it should always

be written in Meters.

2) However software vendors have started to write LAS files in units other

than Meters. In the headers they place a Scale Factor (the conversion from

the reported values to get back to Meters or they state that the LAS file

has been written in US feet.

3) We updated our importer to read the Headers and use either the scale or

the Header Units and then read it into the project in the appropriate way

a) If your project is a metric project and the imported file has

         i) No Units defined - we assume Metric and import at 1:1

        ii) Meters defined in Header - we read as metric at 1:1

        iii) US Feet in Header - we import the data and scale by .3048 etc

to get the data into metric units

        iv) Scale in the Header - we use the scale info to get back to

Meters and import at 1:1

b) If your Project is in US Feet and the imported data file has

       i) No Units Defined - we assume Metric, we scale the data to get to

Feet and import at 1:1

      ii) Meters defined - We scale the data to get to US Feet and import

at 1:1

      iii) US Feet Defined - we import at 1:1

      iv) Scale in the Header - We scale the data to US Feet and import at

1:1 using the scale information

The problem that Bens file has is that it is actually written in Us Feet

Coordinate Values but has no header info to tell us a scale or that it is

in US Feet. So if we import into a Metric project - the coordinate values

are correct but then when we change to Feet it scales by 3.28 and the

coords are now 3x too big. If we import into feet, we assume metric and we

multiply the file coords by 3.28 and then they are also too big.

The answer to the problem is actually go back to the source software and

rewrite the file either in Metric Units or in US Feet Units but with the

header information - however if that is not possible, write the data in

Metric and then it will work just fine in TBC. Failing that - in BC-HCE

v4.12 we had the ability (which we added for this exact reason) to override

the Units during import and that will work - the Geospatial Team however

when they added all of the Grid and Ground Support that was causing their

users problems, took away the Units override function in TBC v5.0 and later

- so currently (v512) you have two options

1) Use BC-HCE v4.12 to import the LAS files and then save the project and

then open it in 5.12 and it will be OK

2) Ask the data provider to write it in Metric Units or in US Feet with the

correct header information so we can read it correctly

Option 3 is for Trimble to fix this of course - that request is in the

system and it has now been implemented for the next release due out in the

next month or so

Hope this answers the problem

Alan

On Thu, Oct 10, 2019 at 1:43 PM adearth@simonsonconstruction.com <

Alan, i understand, i think, the reasoning behind this change. I don't how ever feel it was effectively implemented, and I think i needs more attention before it is released outside of beta testing. I will try my best to address your comments directly

1) i have no access to the original standards for .las files so I can't confirm, nor argue this point. I'm simply trying to fix the issues this has caused.

2) You mention that "vendors have started to write" files in other units. However the files I mentioned, and linked, are in no way new. The vendor in fact is the US Government. and the files in question are 13 years old.

3) if the importer has been updated, can it be updated again in a way to gives us back the features we had even just  a few weeks ago

a) project nor file is in metric

b) project and files are both in US feet

   i) if no units are defined, please dont make an assumption, ask, TBC is a powerful software that asks us for everything with editable fields for every imaginable variable... except this new one?

going back to the source software is not a viable option. These are publicly available standard files that have been available and used by multiple agencies and softwares for over a decade. I personally have over 1000 files just for our county alone and I shutter to estimate the number for the rest of Ohio, let alone the US. It does not seem likely that we will have the clout to ask the government to update millions or more files that have worked for years, because one companies recent update now makes assumptions upon the files.

Just to confirm, the issue in question is not v5.0 related. I have been working in 5.0 for awhile now with no issue and .las import has worked without fail. Its only in the most recent update (i presume 5.10, but not sure) that the .las issue has come up. It brings up a popup on import but none of the options allow unit override or have any effect at all. Prior to that update the USER was trusted instead of ASSUMPTIONS upon the file and everything worked fine.

Thank you for your comments. I am doing my best to stay civil and keep my comments constructive. But my honest opinion here is that GeoSpatial Team dropped the ball on this one, and rushed out a change before it was fully vetted. I think the best recourse at this time would be a hotfix to return to the previous stable release while we wait for them to work the bugs out of this new importer. could this be possible?

The units option was put back in to the next release in yesterday’s build of the 5.2 release so I hope that will re solve the problem - I could do with a file from you that fails today that you can tell me the coord range that it is meant to be in so I can check it (Min Max coord range or coord of top right point in file after import so I can validate what you wanted

Alan

Sent from my iPhone

Next release is only 2 weeks out - so the time it takes to do a hotfix is not viable this close to a new release - key here is to make sure that the new release works as needed. If you need a Beta Version I need you in the Beta program (if not already) - let me know and then we can get you access to an update next week

Alan

Sent from my iPhone

Alan,

send me an email at adearth@simonsonconstruction.com  and ill send you the files. At this point I have located an install for 5.01 and downgraded us back to that release, which fixed the issue. Everything all works with stateplane coordinates as before.

I can see it trying to import it as meters not USFT. I know if you can find a way to convert it to xyz file, that will let you chose the old import option of what format it is.

Yes, I agree. I have used ESRI LAStools to convert from .las to .xyz. This has allowed me to bring in 3rd party .las files that are in iFT. 

I have had the same problem as was stated here. I have .las files in International Feet, and I was able to import them in previous versions of TBC; I only had to answer the unit question. Now, the .las files come in several hundred thousand feet away from where they should be, and none of the options allow me to fix this issue. 

I will try some of what Alan gave for solutions, but I wanted Trimble to know that this problem is affecting more than just one individual. 

Thanks for all the info from everyone. I'm going to give it all a try too. 

Thanks Sean,

FYI... I noticed my original 5.01 self extracting file left a download folder in C:\Trimble Downloaded Installations, I completely removed 5.10 and reinstalled 5.01, and everything is working wonderfully again. Stepping back to version 4 just wasn't an option for us as all our ribbons, templates, etc are built for 5.