Project Setup
Overview
In order to make creating new projects easy Percept incorporates a project setup wizard, a software component which guides you through the different steps needed in order to set up your own Project.
These steps are:
-
Step 2 - Registration of multiple LiDAR devices (optional) if you are using Percept in a multi-LiDAR setup.
-
Step 4 - Zone Management in order to setup areas of interest for e.g. object detection.
-
Step 5 - Configure Advanced Settings and select algorithm parameters for fine tuning Percept’s behaviour.
-
Step 6 - Define Output Options for data generated by Percept.
-
Step 7 - Manage existing Projects for your generated project.
Step 1 - Create a Project
The first time Percept is accessed over the browser (see also access Percept web interface) it will show the project view with an empty project list. The project overview shows the list of your created projects.
In order to create a new project you will have to click on "New Project" on the lower right corner of the browser window. This will open the project setup wizard, as detailed in Create a New Project. As an alternative, it is possible to create a demo project by clicking on "Add Demo Project". These projects are completely setup and ready to be used, as explained in Create a Demo Project.
Create a New Project
Enter General Project Information
After clicking on the "New Project" button, the project wizard is started and it will bring you to the basic project information configuration screen. Here you need to define:
-
A unique name for the project (cannot be changed later on).
-
The LiDAR devices to use in your project.
-
A meaningful name, e.g. cube_front_entrance in order to easily distinguish between different devices and have an indication for the location of your LiDAR.
-
The IP-address/Hostname of the LiDAR connected to your network.
-
-
To get familiar with percept, we provide example recordings. Therefore choose as data source Example Point Cloud Recording and select an example instead of using a Connected Lidar. The example recordings will be played in an infinite loop.
the IP-address/Hostname of the LiDAR can be found within the Web GUI of the LiDAR device itself. Please refer to the user manual of the device in order to understand how to access the device’s Web GUI. |
After choosing a project name and adding the LiDAR devices to your project, click on "Create Project" to continue with the wizard. If you click on the "X" in the top left corner, you will abort the project setup and get redirected to the project overview page.
Make sure that the LiDAR devices you want to use are shown in the Input Sources To Use In This Project, you can remove devices from the list as well. |
3D Viewer Overview
The 3D Viewer is a shared visualization tool between the project wizard and the project visualization.
It will show you the point cloud(s) generated by the device(s) used.
Based on the project wizard step, the 3D viewer will have additional functionalities which will be explained in detail in the respective section. |
Based on the defined zones in your project, the 3D viewer will show the zones in the visualization and the corresponding messages (e.g. current object counter for zone A) see Visualize your Project for details. |
You can rotate and translate the camera in the 3D scene and also zoom in and out of the scene.
You can change the camera view on the 3D scene with the following mouse interaction:
|
There is a control window, which lets you adjust the visualization to your needs.
-
Objects
-
Hide Objects will disable the visualization of the detected objects
-
Object Options
-
Hide Bounding Boxes will disable the visualization of the object bounding boxes
-
Hide IDs will disable the visualization of the object ID
-
Hide Paths will disable the visualization of the object path
-
Hide Spheres will disable the visualization of the sphere marking a detected object
-
-
-
Zones
-
Hide Zones will disable the visualization of the zones
-
Hide Zone Arrows will disable the visualization of the direction indication with zone arrows
-
-
Grid
-
Grid Tile Count selects the amount of tiles for the grid
-
Grid Tile Size (m) selects the size of each grid cell in meter
-
-
Point Clouds
-
Point Cloud: Full will disable the visualization of the point clouds
-
Point Cloud: Filtered will enable the visualization of the foreground points generated by the background subtraction
-
Point Cloud: Full
-
Color Schemas selects the coloring method for the point cloud
-
Point Color Channels The channel to use for coloring, e.g. range
-
Minimum value The minimum value of the point color channel to start with the coloring, everything smaller will get mapped to the same color
-
Maximum value The maximum value of the point color channel to start with the coloring, everything above will get mapped to the same color
-
Point Size will adjust the point size of the point clouds displayed
-
-
-
Camera
-
Camera Position sets the camera position
-
Viewports selects a fixed camera viewpoint (Side, Top, Default)
-
Damping when active, if you move the camera around it will continue the movement and slowly stops
-
Not all of the options mentioned above will be always present. |
Create a Demo Project
By clicking on "Add Demo Project" in the lower right corner of the project list window, it is possible to create a project that is ready to be used and does not need any setup. The purpose of a demo project is to show some use-cases of Percept in different applications and give an example of how a complete project looks like without needing to go through the setup wizard first.
After clicking on the button, a pop-up is shown. Here, the only needed steps are:
-
Select a meaningful name for the project, e.g. example-project-security.
-
Choose one of the available projects based on the application of interest. Currently, these applications are available:
-
Security: a security zone is placed at a crossroads to check if people walk through it.
-
People Counting: two pass-through zones are used to count the number of people walking down two different streets. People are counted based on the direction they walk through the zone.
-
More information about these zones can be found in Step 4 - Zone Management.
From the pop-up, click "Cancel" to cancel the project creation or "Create" to create the selected demo project. In the second case, the new project can be now seen in the project list and directly started, as explained in Step 7 - Manage existing Projects.
a "_demo" suffix is added to the chosen project name to distinguish the demo projects from the ones created using the setup wizard. |
Step 2 - Registration of multiple LiDAR devices (optional)
If your setup consists of multiple LiDAR devices, the project wizard will bring up the registration step after the initial project information screen. Registration entails that we will combine multiple point clouds from different devices to one, by finding the position and orientation of the devices with respect to each other. This is required to combine multiple sensor measurements, allowing for utilizing their data just as they were generated by a single sensor with a larger field of view or a higher point cloud density.
The registration step is only required if you use more than one LiDAR device in your project. |
For registration of multiple LiDAR devices, you can choose between a marker based and a semi-automatic approach, which will be explained below.
In both methods the point clouds need to have an overlap in order to register them. |
Marker Based Registration
For marker based registration a spherical object is used as a marker and access to the area in field of view of the lidars is needed. During the registration process the marker needs to be placed in the overlapping area between the corresponding devices. At least three detections of the marker on different positions in the overlapping area are required to reliably register the point clouds. This method works best in a static environment, where only the marker and the person moving the marker to different positions are present.
Requirements for Marker Based Registration
|
The marker-based registration shows you two point clouds in parallel, you can select which point clouds you want to display using the drop down menu. Select two point clouds which have an overlap and then start placing the marker in the overlapping area.
Percept will try to find the marker in the scene. When there is a marker detected, Percept will show you a dark blue sphere at the place where it is detected.
If the sphere shows up in both point clouds, press ADD MATCHING DETECTIONS and it will be saved as a valid match. The sphere in the visualization will turn light blue and gets a number, indicating the matched marker.
After the match is saved, physically move the marker to another location in the overlapping area and repeat this step until you have at least three matched detections.
If you have enough (best 3 or more) matched detections between all overlapping areas, click on CALCULATE TRANSFORMATION, Percept will then calculate the transformation between all correspondences and display the result in the next step.
Check the result if it is correct, before you continue with the wizard. If the result is not correct, please go back and try it again. |
-
Make sure that the scene is empty
-
Start the marker based registration and wait around ten seconds until you move into the scene with the marker
-
Place the marker into the scene and make sure it is clearly visible in the corresponding point clouds
-
Make sure that the marker does not move
-
If there is a dark blue sphere in both point clouds, at the place where the marker is, click on ADD MATCHING DETECTION to save this match. The Number of Matched Detections between these two devices will increase by one.
-
Repeat steps (3) to (5) until you have at least three markers per overlapping area
-
Click on "Calculate Transformation"
-
Check the result in the next step, if it is not correct, go back and try it again
Semi-Automatic Registration
The semi-automatic registration is especially useful in crowded environments, or if you can’t use the marker based methods and in scenarios with not much overlap. If you are used to viewing 3D point clouds it might even be faster than the marker based method.
The semi-automatic method allows the user to transform every point cloud, with the goal of them matching roughly together. Then in the next step, Percept will refine the transformation so that it is accurately registered. In case it is needed, there is also the possibility of skipping the refinement and directly moving on to the next step. In this case, the point clouds will be kept in the manually set positions and no refinement will be applied.
Refining the transformations is the preferred way to register point clouds. Skipping the refinement is to be avoided, unless the refinement step is not able to improve the registration or in cases where the registration is actually not needed (for example, if multiple LiDAR devices are used but they do not have overlap with each other). |
Requirements for Semi-Automatic Registration
|
There are two ways to manipulate (transform) the point cloud, 1) using transform control window, 2) using keyboard shortcuts.
Figure 10. Transform Control Window
Some useful keyboard shortcuts to transform a point cloud (make sure that you selected the controls for this point cloud):
Figure 11. Controls to translate the point cloud
Figure 12. Controls to rotate the point cloud
Figure 13. Controls locked
|
One tip on how to speed up your workflow
|
-
Select the first point cloud coordinate system by clicking on it
-
Unlock the controls for it
-
Rotate and translate the point cloud, so that the overlapping area fits to the point cloud of the other LiDAR
-
Press "Refine Transformations" (preferred option). As an alternative, press "Save Current Transformations" to avoid applying refinement
-
Check the result in the next step, if it is not correct, go back and try it again
Have a look at the refinement result before continuing with the project creation. If you are not satisfied with the result you have the chance to go back and try it again. |
Step 3 - Alignment of the Point Cloud to the Virtual Grid
The goal of the alignment step is to align the ground area of the point cloud to the virtual grid. The alignment with the ground to the virtual grid will help you later on with creating zones, and it will make it easier to understand the 3D scene.
If you are happy with your alignment click on "Set Alignment", which will take you to the next step.
For tracking and object counting/monitoring as well as for volume measurement projects it is required to align the ground to the grid! |
You can use the auto-alignment function, to align the point cloud on the grid (if the result is not as expected, manually align the point cloud) (currently you may still need to adjust the height to match the ground with the grid) |
There are two ways to manipulate (transform) the point cloud, 1) using transform control window, 2) using keyboard shortcuts.
Figure 15. Transform Control Window
Some useful keyboard shortcuts to transform a point cloud (make sure that you selected the controls for this point cloud):
Figure 16. Controls to translate the point cloud
Figure 17. Controls to rotate the point cloud
Figure 18. Controls locked
|
Step 4 - Zone Management
Percept has different types of detection zones. Detection zones are used to apply certain algorithms to an area of interest in the point cloud, that could be a zone to count if something (e.g. an object) is inside the zone or not. The following section will introduce you to all available types and lay out corresponding example use case.
A good mounting position and the right amount of LiDAR devices is important to make the results reliable. For example, if you expect objects to occlude other objects, or there are some obstacles in the field of view, consider changing the mounting position or adding one or more LiDARs from different angles. Since occlusion can have a negative effect on the system performance. |
Pass Through Counter Zone
The Pass Through Counter Zone counts the number of objects that moves through the zone in a specific direction. In order for a counting event to trigger, the object needs to start outside the zone, move through the zone and fully exit the zone again. The Pass Through Counter Zone can either accumulate all counted objects or show only the currently counted objects. This can be configured by activating or deactivating the 'Accumulate the Counter Zone' option in the Pass Through Counter Zone definition, as can be seen in Example of the user interface to create a Pass Through Counter zone.
The Zone has a direction to specify what movement will be counted as entry and what movement will be counted as exit (see screenshot below). A movement in the direction of the arrow will be an counted as entry, while a movement in the opposite direction will be counted as an exit.
Objects moving into the zone and disappearing (have a look at the Appearance/Disappearance Counter Zone for this use case) or leaving the zone through the side it entered (seen from the middle of the zone) will not get counted. |
Please make sure that the LiDAR covers enough area around the zone. Since we need at least one detection before the object enters the zone and one detection after the object leaves the zone for the system to trigger an increase in the inbound or outbound direction |
Please adjust the width of the zone to allow at least one detection inside of the box (this will depend on the scan pattern and therefore on the framerate and on the object speed), to reliably detect inbound/outbound movements. |
Appearance/Disappearance Counter Zone
The Appearance/Disappearance Counter Zone will count if an object moves into a zone and then disappears (sink) or if an object appears in a zone and then leaves the zone (source). Use cases are e.g. a person entering an elevator or an escalator. Another use case would be the entrance into an area which is not covered by a LiDAR, e.g. a store where you want to count how many people are entering or exiting, but you do not have enough coverage to use a Pass Through Counter Zone. The Appearance/Disappearance Counter Zone can either accumulate all counted objects or show only the currently counted objects. This can be configured by activating or deactivating the 'Accumulate the Counter Zone' option in the Appearance/Disappearance Counter Zone definition. An example of this for the Pass Through Counter Zone is shown in figure Example of the user interface to create a Pass Through Counter zone and for the Appearance/Disappearance Counter Zone there is an equivalent option during the zone creation.
Objects appearing inside the zone and disappearing inside the zone, without leaving the zone will not get counted. Objects entering the zone and leaving the zone will also not get counted.
Please make sure that the LiDAR covers enough area around the zone. Since there needs to be at least one detection outside of the zone and one detection inside the zone. |
Please make the zone large enough to allow at least one detection inside the zone. |
Current Objects Counter Zone
The Current Objects Counter Zone counts the amount of detected objects inside a zone for every frame.
An object is counted if the center of the object is inside the zone. |
Security Zone
The Security Zone works with points rather than with objects. While an object can consist of e.g. 50 or 100 points, the security zone decides based on the number of points if an intrusion is happening or not. The threshold level defining the minimum number of points before an alarm is raised can be defined by the user.
The amount of points reflected from an object in the zone will depend on the scan-pattern used by the LiDAR device and therefore needs to be adjusted to prevent false or missed alerts. |
The amount of points reflected from an object is distance dependent. Think about splitting the zone in multiple ones with different threshold values or adding LiDARs to the scene for a better coverage. |
It does not matter where the points in the zone are, if the threshold level is exceeded an alarm is raised. |
Occupancy Zone
The Occupancy zone works similar to the Security Zone, with the difference that the output message will be different. This zone is for e.g. parking spot monitoring where you want to get a message that a specific spot is occupied rather than an alarm message.
Volume Measurement Zone
The Volume Measurement Zone provides a method to measure the volume inside the zone. It is important that the point cloud is aligned on the grid for the volume measurement zone to work reliably. You can specify the grid resolution used to calculate the volume. The grid resolution will influence how accurate the result is, a smaller grid resolution will therefore be more accurate. But note that a value smaller then the point to point distance generated in the point cloud will not increase the accuracy. If you have material which isn’t fine grained, a higher value for the grid is sufficient.
It is also possible to define the empty volume in the zone, this value will be subtracted from the measured volume. With this it is possible to remove objects standing around or walls which are not intended to get measured. Also it is possible to measure negative volumes.
The grid resolution per tile is defined in volume in meter and the empty volume in cubic meter. |
Zone Management Interface
In the project setup and in the project edit mode you can create, delete or change the zones.
If you don’t have a zone created you can either create one or if you don’t want to have a zone at all you can go to the next wizard step with the Save Zones button.
By adding a zone, Percept will open a dialog where you can select the different zone types. Select the zone type and enter a name for your zone. Press "Save" to create the zone.
Now since we created a Pass Through Counter zone make sure that its arrow shows into the direction you want to count as entry, have a look into the Pass Through Counter Zone description for details.
There are now two additional buttons to delete or edit the created zones.
In the transform control panel you will find a copy button for each zone, with which you can create a copy of a zone including the zone type, size and orientation. The copy of the zone will be positioned next to the original zone. A name will be suggested for the new zone but it can be modified to different name if needed.
Add all the zones you need to your project.
You can mix different zone types in your project. E.g. create one Pass Through Counter zone and one Appearance/Disappearance Counter. |
The Security Zone and Occupancy Zone will have an additional field where you can set the minimum points threshold.
Click on Save Zones to continue with the next wizard step or finish your project edit.
Some useful keyboard shortcuts to transform a zone:
|
Step 5 - Configure Advanced Settings
In the Advanced Settings you can change the algorithm parameters used by Percept. The default parameters are good for object tracking use cases with slow objects, like people.
If that does not meet your use case you can change the parameters here to improve your result.
The options will be discussed in this section in detail.
Data Source Options
Here you can select if you want to enable time synchronization of the devices or not. If it is enabled, Percept will synchronize the device/s with an ntp server. This ntp server has to be set in the WebGUI of each LiDAR device.
When to use time synchronization:
|
If time synchronization fails when starting a project, check in the device WebGUI if these conditions are met:
|
Background Subtraction Options
Background subtraction is used to separate the point cloud into foreground and background. Therefore you could also call it foreground detection. The background stores the non relevant features in a scene e.g. all the points generated by static elements in the environment, while the foreground has the relevant features e.g. the points generated by an object entering the scene. The foreground points are then used for clustering to detect objects.
Percept provides two main types of background subtraction algorithms.
-
Mixture of Gaussians (for dynamic background)
-
Mixture of Gaussians on device (will activate the on device version)(for dynamic background)
-
Octree for static background
Mixture of Gaussians
Mixture of Gaussians is the default background subtraction used. Its benefit is that it can adjust to changing environments. If a new object is placed into the scene and it is standing there for a certain amount of time it will get merged into the background and will not get recognized anymore.
Number of Frames for Initialization |
Number of frames used to build the initial background |
Exponential Decay |
Specifies how fast objects switch between foreground and background |
Minimum Weight Threshold |
Controls how much noise the background/foreground is expected to have |
Mixture of Gaussians (on Device)
This option enables the LiDAR to process the background on the device and only send the foreground to Percept. Since this will only send the foreground to Percept, the network traffic is reduced. The reduction highly depends on how many foreground objects are in the scene.
Number of Frames for Initialization |
Number of frames used to build the initial background |
Exponential Decay |
Specifies how fast objects switch between foreground and background |
If Network traffic is of concern, use this option to reduce the network traffic. |
Octree
The Octree option is for static scenes, where no change in the background is expected. It will generate the background when the project is started. The first frames are collected to calculate the background.
Number of Frames for Initialization |
Number of frames used to build the background |
Octree Resolution |
The Octree grid resolution in meters |
Point Cloud Filter Options
The point cloud filter can remove noise in the scene and therefore improving the object detection. The radius outlier filter will filter out all points that do not have a certain amount of neighbor points in a given radius around that point.
Minimum Number of Neighbor Points |
The minimum number of points in a given radius around that point to be an inlier (valid point) |
Neighbor Radius |
The radius to search for the neighbor points |
Clustering Options
Clustering identifies dense point cloud regions in the foreground point cloud. Each dense region is called a 'cluster' and represents an object for Percept. Percept provides several clustering methods.
Mean Shift
For people counting applications we recommend mean shift, since it is very good at separating close circular/spherical objects. But it also requires more processing power in comparison to DBScan.
Minimum Points for a Cluster |
The minimum amount of points to form a cluster |
Average Radius of Objects |
Specifies the average radius of an object in meters |
DBScan
DBScan is not so processing heavy as Mean Shift and works well for objects with arbitrary shape and size. DBScan performs well on mixed environments with pedestrians, cars and cyclists.
Minimum Points for a Cluster |
The minimum amount of points to form a cluster |
Distance to Next Cluster Point |
Controls how far points are allowed to be separated to belong to the same object-cluster in meters |
-
Set the distance value to the average distance between the LiDAR measurements of the same object. Thereby, the object should be located at the largest distance it still needs to get detected.
-
Set the
minimum points
to the average amount of points measured within theaverage point distance
identified in step (1).
If two different objects are detected as one while they are close to the sensor, increase (2) until objects get separated again. The trade-off is that now objects are getting detected only in shorter distances.
Tracking Options
The Tracking parameters are set to work with slow objects, like people. You can adjust the parameters to your needs. The following parameters are available.
Initial Position Uncertainty |
The expected position error of the first detection of an object |
Initial Velocity Uncertainty |
The expected velocity deviation from zero of newly detected object |
Linear Acceleration Noise |
The expected error with respect to the assumption of having zero acceleration |
Observation Noise |
The expected error of the observed center of an object to the real unobservable center |
-
Increase these values if objects enter the scene with a higher velocity
-
Start by increasing the
initial velocity uncertainty
Step 6 - Define Output Options
While you can view the result of the data processed by Percept on the web user interface of Percept, there are also several options to output the data.
Write to File
Percept can save the detected objects and/or the generated states (e.g. an alarm) to files. The following options are available.
Messages per file limit |
Choose how many messages to write into one file |
Generated State List |
Write the state list to file |
Generated Object List |
Write the object list to file |
Encoding |
Switch between Protobuf or Json as encoding |
The files can be found in your percept directory under /data/projects/project_uuid/file_stream/.
You can get the ProjectUUID of your project while hovering with your mouse over the icon next of your project in the overview, see image below. Figure 43. How to find the ProjectUUID of your Project
As an alternative you can also export the zone list, there you will also find the project name and UUID inside the downloaded file. |
MQTT Publisher
Percept can send the detected objects and/or the generated states to an MQTT Broker. The following options are available.
MQTT Broker Hostname/IP |
The hostname or IP of your MQTT Broker |
MQTT Port |
The MQTT Broker port |
Generated State List |
Write the state list to file |
Generated Object List |
Write the object list to file |
Encoding |
Switch between Protobuf or Json as encoding |
Make sure that the MQTT Broker is running before starting your project. Percept will not start an MQTT Broker. |
You will receive the MQTT messages on the following Topics, e.g. for Json encoding:
-
blickfeld/project_uuid/json/Statelist
-
blickfeld/project_uuid/json/Objectlist
You can get the ProjectUUID of your project while hovering with your mouse over the icon next of your project in the overview, see image below. Figure 45. How to find the ProjectUUID of your Project
As an alternative you can also export the zone list, there you will also find the project name and UUID inside the downloaded file. |
You cannot enter localhost as hostname for your MQTT Broker. Even if you are running your MQTT Broker on the same machine as Percept, localhost inside the docker container is something different than on your machine. Therefore you need to enter the IP address of your local machine to Percept. |
Step 7 - Manage existing Projects
If you are finished creating your project, you will be redirected to the project overview. There you will find your newly created project, which you can
-
Start
-
Edit
-
Download the zone configuration
-
Delete
Start your Project
If you click on the play button, the project will be started. It will generate the selected output and run until you stop it.
After starting your project a button will appear to visualize and stop the project. If you click on the visualization button, you will get redirected to a visualization page, where you can see the generated Point Cloud of the LiDAR device/s used and the status messages generated by the configured zones, see Visualize your Project for details.
Edit your Project
Percept allows you to change the following parts of your configured Project:
-
Edit Zone Configuration (to add/delete/change/rename the zones in your project)
-
Edit Advanced Settings (to change algorithm parameters for your project)
-
Edit Data Output Options (to enable/disable/change the data output options of your project)
The interface is the same as in the setup phase described above.
Download the Zone Configuration
You can download the zone configuration either in Protobuf or JSON format. See Output Encoding Description for a description of the output encoding.
The zone configuration might be helpful:
-
If you want to get the zone position into your application
Delete your Project
Removes your created project.
If you delete your Project it will get removed and cannot be restored! |
Visualize your Project
The project visualization will use the 3D viewer to show you the point cloud and the detection zones of a running project. This allows you to have a quick access to see what is going on and if the system is working as intended.
The visualization does not need to be active to run a project |
In Addition there is a Status display in the right corner which shows you e.g. if someone is walking through the Pass Through Counter Zone zone. On the top left next to the project visualization caption there is a back button which will lead you back to the project overview.