What type of deep learning algorithm does pointnet++ use

Submitted by reap-521 t3_10631l0 in deeplearning

Apologies correct me if my question makes no sense, I'm a medical doctor so completely out of my depth with a project I am writing. What frameworks does pointnet++ use apart from CNN? I read it was CNN but can see it uses point cloud spatial data which would require something else? Pls explain in simple terms if possible thank you!

9

Comments

You must log in or register to comment.

suflaj t1_j3eoh9u wrote

Depends on what model you mean. From a quick glance it seems to be a very generic convolutional network with some linear layers. Type of stuff you'd create in an introductory DL course.

−2

BalanceStandard4941 t1_j3frfqe wrote

Because the points are not like pixels in a continuous space, pointnet first sample a few anchors from the point set. Then every anchor point will find their k nearest neighbor(like CNN works on windows of pixels). Then with shared MLP layers, point will now have higher dimension of latent features. Last, to aggregate features of local points, max-pooling will used on every group of points that we clustered previously.

This is one layer they called Set Abstraction layer. Which repeat for 4 times. After SA layers, Feature Propagation Layers can be used if ur task is segmentation, which just upsampling the points.

5

dtjon1 t1_j3g0zhe wrote

The pointnet family of NN's use special mathematical functions called "symmetric" functions to leverage the unordered and unstructured nature of point clouds. For some set of points, no affine transformation (rotation, scale, translation) nor any reordering of points should have any effect on the output of the model. These symmetric functions enable pointnet to handle these cases.

It's hard to discuss pointnet in simple terms beyond this point, but the training process basically has two parts:

  1. We learn these symmetric functions for our dataset and use them to build a representation of the data that is meaningful to the model. This is called feature extraction, and gives us a feature vector. Think of this like an abbreviated form of the data that is able to be used by the model.

  2. We can then use this feature vector to perform whatever task we want. For classification we typically just throw the feature vector into a second neural network (usually an MLP) which outputs a probability distribution over our classes.

All of this happens together during training - the model learns to extract meaningful features from your data and also learns to perform whatever task you have in mind. I really recommend watching the authors' presentation for more info.

Pointnet is really easy to use with some programming experience and doesn't require massive compute requirements, yet is still really powerful. Pointnet++ can be a little trickier as the main implementation I've seen requires custom cuda kernels.

10

reap-521 OP t1_j3gp33l wrote

Thanks for all the responses really appreciate it and v helpful!

1