Trimble Business Center

 View Only

 Is there a way to filter out points that were not stored today?

tyler masse's profile image
tyler masse posted 12-12-2022 13:14
Hello everyone.

Using Trimble Business Center I get the day's work from the field crews, and import it into the file I am working on, but it contains calculated points I have provided or points stored on a previous day that I have already brought into the job and do not want to include in a new .csv when I transfer them to CAD. Is there a way to filter out only the points from today's field work and not include anything else? Maybe a sort by date or only include certain types of points? Exclude imported points? I have looked through the selection options and am unable to figure it out. It looks like it has to be there but the solution eludes me.

I had one the other day with easily 100 bars set in a new subdivision as well as several house's laid out and needed to manually sort the bar's from the calculated points I sent them which were millimeters away. The entire time I sat there thinking there has to be an easier way but I can not find it.

Help
Marian Plucinski's profile image
Marian Plucinski
I would highly recommend to store every surveyed point as <reversed date><initials of the field operator><consecutive number>
I cannot stress enough that it will work wonders ONLY in YEAR-MONTH-DAY order. 

[YYMMDD][NNN][1234]

221213MGP1001...
221213MGP1002...
Ronny Schneider's profile image
Ronny Schneider
Are those Access or Siteworks jobs we're talking about? Can we see a sample?
And it also depends how your field crew collected the data, as simple measure or as stakeout and how they named the points.

But in general, if your field crew would just use the reverse date in the pickup points, you could just advanced filter for the point ID.
We always use YYMMDD Initial running point number, i.e. 221213RS0001.

If they are at least stakeout points you can still advanced select just for those.

Or use the exaggerated 3D view in order to separate the points a bit more.

Ronny Schneider's profile image
Ronny Schneider
Another option is to open the optical spreadsheet view.
Set a custom filter for the timestamp.

Select the observations and right click "select points by vector".
tyler masse's profile image
tyler masse
Thank you for the answers. I don't see renaming how we number our points as an option, I think it would take longer in CAD to type out that number every time I want to refer to a point then it would to do the one time task of cleaning up the job on import. Getting the field crews to actually do that would be a whole different topic as well.

It does look like I can select the points based on vector though which should be helpful. I also see a similar option for GPS points. When I do the points by vector selection it either gets all the gps or all the total station points but not both. Is there an option that would allow both. I could then Invert the selection and delete the old points. I am not seeing one in advanced select. If memory serves, the data collector (using access) stores imported points as "Keyed In" points but I am not seeing that distinction when looking through TBC.

I am looking at a job right now where a crew did a topo on a lot in town for new construction. They showed up, set a couple spikes with GPS then set up on those GPS spikes and completed the legal/topo work with their total station. They then brought that info to the office and the Licensed Surveyor gave them a csv with a couple points to set new survey bars. They created a new empty job, imported all the points from the previous job plus the new ones and went to set the bars. I want to remove all the points from this second site visit except the new bars so when I save the .vce and export the csv to CAD it only has the new info. This job is a fairly small example but we do 100 lot subdivisions and along with the houses is can be thousands of points.
ian bissonnette's profile image
ian bissonnette
i can tell you our workflow. we do the same thing into tbc for processing, export csv and into carlson cad.

-select all
-use advanced select to apply to current selection/drop down coordinates/apply
-project explorer/imported files/expand one of the files and if you did not click anywhere else then the 'coordinates' should still be highlighted 
-right click and delete
this will delete all coordinates that were imported by csv into data collector and the majority of calcs made in the field
note: the point numbers will still exist.
so now expand the points in project explorer. any point without a drop down will not have any raw data associated and can be deleted(as long as all points have been computed correctly in the project). or you can see them in the point list with no coordinate associated any more.

we also select all asstaked points and right click and convert and then delete the asstaked ones, verifying them in cad.

this works well, but there is then the question of control points going into cad again, depending on what happened with your project sometimes control points might have small shifts......
ian bissonnette's profile image
ian bissonnette
i just thought i should add a bit to my previous answer. if you are going to follow my suggestion then you need to be aware of how you are going to deal with your control points. 
-in the example you provided, we would assign an office entered coordinate in tbc for the base point. then the project will compute no problem. 
-depending on the situation, like if you only have gun work on a second trip to a site using existing control, then you might need to use office entered coordinates for these control points. if i know they are good, i will sometimes do this prior to the coordinate delete, because they are there and assigned to the points already. but another approach is to import a csv of the control points into tbc.
-and another thing that we struggle with is how many tbc files exist for a job. we like to have only one, but on big jobs this is not always possible. with one tbc file, usually control points have a starting rtk tie and so only the base needs to have an office entered coordinate. but if the control network expands on the project then sometimes there can be a small change in these coordinates. currently we export all the points and on import into cad check any differences.

and maybe this is the really obvious solution. you could just export everything and make sure the field crews are clear as to what point range was used during a day. then in excel you could parse out the points needed. 

i hope this makes sense.
Marian Plucinski's profile image
Marian Plucinski
Getting the field crews to actually do that would be a whole different topic as well

Well,
I guess that can be simply achieved by internal procedures and quality system.

@ian bissonnette
We keep ALL surveyed points in one project, but we do not import JOB files.

1) Every field crew starts new job everyday [YYMMDD][NNN]​[X].job e.g. 221214MGP1
2) Every point is numbered [YYMMDD][NNN]​[X]xxx e.g. 221214MGP1001... 221214MGP1999
3) When the day is done day job is imported into its own individual TBC project [YYMMDD][NNN]​[X].vce e.g. 221214MGP1.vce
4) Data is checked for QA, features are processed, lines are drafted etc.
5) Results are exported as VCL eg. 221214MGP1.vcl
6) This VCL is imported to the master project file

This way points and features are no longer dependent, and the master is heaps easier to manage
ian bissonnette's profile image
ian bissonnette
thanks marian. i notice a lot of your posts and you seem to be a more advanced tbc user than myself and very helpful.

i will look into vcl files. it has not been something we have tried. but i would imagine if a network grows over several days then these would need to be adjusted in one file. also lines that auto draw from one day might not connect to the next day(actually i don t think they do even in one tbc file). 

i can sympathize with marian and tyler on point numbering since we use both. on small jobs just start at 1. but on larger ones we do use date ranges almost exactly as mentioned. with date range a couple problems come up.
we often use a shorter number for control, sometimes an alpha numeric. 

some cad software will not accept a really long point number. 

and even in access i don t think you can add a really long point number to a line control code. or it did not work last time i tried with tsc5. 

sorry to push the limit on the main topic for this thread.
Scott Lehr's profile image
Scott Lehr
Hey Tyler, 

I might add another option that I don't see in the thread here.  I have had similar problems with TBC project files with numerous JXL and job files and  importing a ton of Calc points. 
My solution is more of a workflow but I think it is reasonable. 

1 - Every day's data should be processed in an individual TBC.vce file. Do your control checks and base height checks here as it is easier to deal with than with a merged combined TBC project file. 
2 - To delete Calc "Unconnected points" en masse do the following. 
  • in view filter manager deselect "Show Disconnected points"
  • Once the disconnected points are no longer in the display view, type CTRL+A This will select everything on the screen. 
  • Now Re-check Show disconnected points.
  • You should now see everything selected except the Calc points.  Next step is in the Home TAB select the Invert selection button. This will now reverse what is selected and you should only have the calc points selected. 

From here you can Deleted the Points. 

3- Now once you have cleaned up your individual days data, you should then merge it into you "master" or "Combined" file that contains all survey data collected. 
Use the "Merge Survey Projects" button in the Survey tab to do this. 

The end result of this work flow is better QC and cleaner more organized master TBC project files. 

Hope this helps. 

SL




Rob Koot's profile image
Rob Koot
I'm curious as to why you are importing CSV points and other jobs points into a new job. There is really no need to import them. You can just link the files instead. They will be available in the new job and you can do cogo with them (in which case the linked CSV points are automatically copied into the job at that point), or you can stake them (and the CSV point is not copied into the new job at all). 

This won't stop all 'calc' points coming into TBC, but should reduce them significantly.
https://help.trimblegeospatial.com/TrimbleAccess/latest/en/Layer-manager-point-files.htm?Highlight=linked%20CSV
ian bissonnette's profile image
ian bissonnette
rob. importing csv s is our current workflow as well. this might be a carry over from working with survey pro for so long. or it is in our case. i don t remember a way to link files or jobs in survey pro. i will look into linking files, this could be handy for us. 

can i mention to the trimble employees that this is the second recent post about having difficulties selecting certain entities in tbc. the other post was trying to select office entered coordinates. having two recent posts indicates to me that a little more functionality in the selection tools would be appreciated. in particular: advanced select office entered coordinates and advanced select unconnected points. there are likely a couple others.
tyler masse's profile image
tyler masse
I think I have managed to cobble together a way of getting it to work. Thank you everyone.

To reply to a couple of the comments. I unfortunately cant mandate that the field crews do things a certain way. They import which is probably a holdover from different field programs we used to use. My preferred method when I am in the field is to link files but (true or not) there is the impression amongst our field teams that this method causes problems. I know there used to be issues with being unable to do a field calc from a linked point in trimble access but i have not had that problem so I would assume it is fixed. As an Instrument man I once had a crew chief get annoyed with me when I mentioned I had linked the csv instead of importing it. He made me unlink it which wiped out our GPS base point and all co-ordinates. It required starting the whole job again, thankfully it was only 30 mins of work. Knowing more about the office part now I imagine that might be fixable inside but we couldn't get it working in the field.

To echo the above comment it would be really useful to search by type like that. I would also love to see a way to separate out field calculations so they can be kept, i don't want to indiscriminately get rid of most of them as our crews will occasionally hand measure in things like hard to see building corners and then enter them as a calculated point. When I encounter people with drafting experience that were not taught by the same people as I was I often ask them if they know of solutions to whatever problem is eluding me. I have yet to find someone who could answer the above filtering problems. Thank you again everyone. The problem might be more widespread then it seems. In my experience most people won't create a login and register to a forum to ask the question when it can be done the long way.
Rob Koot's profile image
Rob Koot
I don't believe any other survey field products let  you link files like Trimble Access, and I can see why you would still be doing it when coming from other products like Survey Pro. Linking a job so you can do a check on a point you measured yesterday is very useful! Trimble Access (formerly known as Survey Controller) has been linking files for 15 or more years, and the only issue people might have is when they have points of the same name (and diff coords) in both the current job and linked files. The point in the job will always take priority. By all means keep importing if that is the workflow you prefer, but IMHO there is little value to doing this, it just adds a lot of unnecessary clutter to your daily job files, and extra steps in the office to clean it up.
Robert Hoy's profile image
Robert Hoy
If I bring in a JOB file into TBC and save the TBC project, I will start a new JOB file in the field for new observations. Keep your data clean and don't import a JOB file that has already been imported into TBC and saved there.
Ryan gmail.com's profile image
Ryan gmail.com

We make a blank file from the TBC job; with the Date and Initials of the field crew.   We also make a all-pints file "master" or all points file from TBC. (can be up-dated daily or weekly depending on workload   Then we work in the field, we open the day file, and link the up-dated all-point file.

This is allows for simple day to day separation of the data collected.    Easier to review, process and look-back on as needed.

hope that can help ya'll, though this is an old post. :)

-Ryan