[JohnnyWaffles]

Hey everyone! So, I'm so glad to find this forum. I've been scraping the web for guidance on AI development and it sure has been a long journey. I look forward to learning more with all of you.

So I want to embark on a project and I’m not quite sure where to start. What I would like to do is train an AI using a specific set of 3d point Clouds, and once it’s training is optimized, create a 3D point cloud based on the ones it trained on.

Here is what my research as revealed so far: that there is a library called PointNet: https://github.com/charlesq34/pointnet
It’s for deep learning on point sets or 3D point clouds and it uses Tensorflow. I’ve used Tensorflow before, but not extensively.

So, I think I understand how to train a neural network using point clouds. What I don’t understand is, when you are done training the NN, how do you use the information to create something with it?
I have followed some of the basic Tensorflow tutorials, and they all stop at training and optimizing. They never go beyond by telling you what you can do with that information.

In my case, how do I use a properly trained NN to build a 3D point cloud? Will I need to use another library besides Tensorflow & PointNet to pick it up from here? Any help is very much appreciated.

[8pla.net]

What I don’t understand is, when you are done training the NN, how do you use the information to create something with it?

You can save what it successfully learned, which is faster than figuring it out again in the future.

[Freddy]

Just wanted to say hello and welcome

[Korrelan]

Welcome JohnnyWaffles

They never go beyond by telling you what you can do with that information.

What is going to be the purpose of the AI?, you must have a specific task in mind?

create a 3D point cloud based on the ones it trained on.

So do you just mean it has to classify a point cloud and then display the corresponding point cloud or a descriptor? Or is it some kind of morphing schema where it produces a point cloud of mixed/ composition of objects.

Perhaps an example usage or scenario would help? or are you looking for ideas of how to combine AI and point clouds?

 

[AgentSmith]

Hey everyone! So, I'm so glad to find this forum. I've been scraping the web for guidance on AI development and it sure has been a long journey. I look forward to learning more with all of you.

So I want to embark on a project and I’m not quite sure where to start. What I would like to do is train an AI using a specific set of 3d point Clouds, and once it’s training is optimized, create a 3D point cloud based on the ones it trained on.

Here is what my research as revealed so far: that there is a library called PointNet: https://github.com/charlesq34/pointnet
It’s for deep learning on point sets or 3D point clouds and it uses Tensorflow. I’ve used Tensorflow before, but not extensively.

So, I think I understand how to train a neural network using point clouds. What I don’t understand is, when you are done training the NN, how do you use the information to create something with it?
I have followed some of the basic Tensorflow tutorials, and they all stop at training and optimizing. They never go beyond by telling you what you can do with that information.

In my case, how do I use a properly trained NN to build a 3D point cloud? Will I need to use another library besides Tensorflow & PointNet to pick it up from here? Any help is very much appreciated.

Nice topic and nice thoughts. This is a good motivational example of what GANs are good for. You got it right. Training a single ANN on data will not suffice to generate similar data. It is quite impossible. You will need at least 2 different networks that interact with each other, in the context of GANs: a discriminator and a generator. Generally spoken the discriminator is trained on your data and tells whether a produced 3D point cloud fits this training data. The generator actually generates the desired data and is trained on the feedback given by the discriminator. GANs were used for generating 2D images but one should also be able to use it for 3D point clouds with a fixed number of points.

[8pla.net]

Johnny (in part) wondered, "when you are done training the NN" Like anything "alive", even artificially so, as a Neural Network... Training is never done. But a computer program that simulates being "alive" is what makes it worthwhile, I think.

[LOCKSUIT]

i had seen a GAN that was 3D....may be what you are looking for. I could find the video, I think it is on Two Minute Papers. It shows a chair or beds made out of 3D red dots on white background, maybe try looking for that.

[JohnnyWaffles]

Hello everyone! Thank you for the replies and the suggestions on using a GAN. I like to do game development as a fun hobby on the side. I had the idea of using AI to produce a 3D model based on a host of models. So let’s say something as simple as a boulder. A boulder or rocks in general have their own design language and distinct look, and while we might consider a rock simple, modeling a convincing rock can be harder than you think. Unless of course you use scanned data like at www.megascans.se (which I’m considering).

Anyway, all that to say, why can’t an AI reproduce a convincing rock if you’re training data is sufficient. Theoretically I’m hoping that with the push of a button, it can iterate a variety of original designs. Now I used a rock as an example, but with that idea you could reproduce anything if you have enough specific training data. Trees, logs, stumps etc.

Maybe even more complicated models like cars or buildings for designing video game environments in software like Unreal Engine. With this kind of set-up you could batch process 3D models. Which would save me a lot of time!

Anyway, I had a realization, that using a 3d point cloud might be more complicated than necessary. Why not just use the vertices of a 3d model and use their 3d coordinates as input for the GAN?

Thank you for your time.

[Korrelan]

https://kotaku.com/a-look-at-how-no-mans-skys-procedural-generation-works-1787928446

 

[AgentSmith]

Anyway, I had a realization, that using a 3d point cloud might be more complicated than necessary. Why not just use the vertices of a 3d model and use their 3d coordinates as input for the GAN?

A voxel representation of the point cloud is also possible and actually even way more practical. In that case you can drop the 3D position information and you just have to provide the color values (binary black/white or RGB,...) for each voxel. All models should still have the same voxel sizes for all dimensions. You need to do some preprocessing such as removing outliers, affine transformations and so on.