Using AI to Create 3D Point Clouds

  • 9 Replies
  • 2738 Views
*

JohnnyWaffles

  • Roomba
  • *
  • 14
Using AI to Create 3D Point Clouds
« on: October 17, 2018, 05:40:09 pm »
Hey everyone! So, I'm so glad to find this forum. I've been scraping the web for guidance on AI development and it sure has been a long journey. I look forward to learning more with all of you.

So I want to embark on a project and I’m not quite sure where to start. What I would like to do is train an AI using a specific set of 3d point Clouds, and once it’s training is optimized, create a 3D point cloud based on the ones it trained on.

Here is what my research as revealed so far: that there is a library called PointNet: https://github.com/charlesq34/pointnet
It’s for deep learning on point sets or 3D point clouds and it uses Tensorflow. I’ve used Tensorflow before, but not extensively.

So, I think I understand how to train a neural network using point clouds. What I don’t understand is, when you are done training the NN, how do you use the information to create something with it?
I have followed some of the basic Tensorflow tutorials, and they all stop at training and optimizing. They never go beyond by telling you what you can do with that information.

In my case, how do I use a properly trained NN to build a 3D point cloud? Will I need to use another library besides Tensorflow & PointNet to pick it up from here? Any help is very much appreciated.

*

8pla.net

  • Trusty Member
  • ***********
  • Eve
  • *
  • 1302
  • TV News. Pub. UAL (PhD). Robitron Mod. LPC Judge.
    • 8pla.net
Re: Using AI to Create 3D Point Clouds
« Reply #1 on: October 18, 2018, 10:53:15 pm »
What I don’t understand is, when you are done training the NN, how do you use the information to create something with it?

You can save what it successfully learned, which is faster than figuring it out again in the future.
My Very Enormous Monster Just Stopped Using Nine

*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6855
  • Mostly Harmless
Re: Using AI to Create 3D Point Clouds
« Reply #2 on: October 19, 2018, 12:33:10 am »
Just wanted to say hello and welcome :)

*

Korrelan

  • Trusty Member
  • ***********
  • Eve
  • *
  • 1454
  • Look into my eyes! WOAH!
    • YouTube
Re: Using AI to Create 3D Point Clouds
« Reply #3 on: October 19, 2018, 09:05:51 am »
Welcome JohnnyWaffles

Quote
They never go beyond by telling you what you can do with that information.

What is going to be the purpose of the AI?, you must have a specific task in mind?

Quote
create a 3D point cloud based on the ones it trained on.

So do you just mean it has to classify a point cloud and then display the corresponding point cloud or a descriptor? Or is it some kind of morphing schema where it produces a point cloud of mixed/ composition of objects.

Perhaps an example usage or scenario would help? or are you looking for ideas of how to combine AI and point clouds?

 :)
It thunk... therefore it is!...    /    Project Page    /    KorrTecx Website

*

AgentSmith

  • Bumblebee
  • **
  • 37
Re: Using AI to Create 3D Point Clouds
« Reply #4 on: October 19, 2018, 12:08:30 pm »
Hey everyone! So, I'm so glad to find this forum. I've been scraping the web for guidance on AI development and it sure has been a long journey. I look forward to learning more with all of you.

So I want to embark on a project and I’m not quite sure where to start. What I would like to do is train an AI using a specific set of 3d point Clouds, and once it’s training is optimized, create a 3D point cloud based on the ones it trained on.

Here is what my research as revealed so far: that there is a library called PointNet: https://github.com/charlesq34/pointnet
It’s for deep learning on point sets or 3D point clouds and it uses Tensorflow. I’ve used Tensorflow before, but not extensively.

So, I think I understand how to train a neural network using point clouds. What I don’t understand is, when you are done training the NN, how do you use the information to create something with it?
I have followed some of the basic Tensorflow tutorials, and they all stop at training and optimizing. They never go beyond by telling you what you can do with that information.

In my case, how do I use a properly trained NN to build a 3D point cloud? Will I need to use another library besides Tensorflow & PointNet to pick it up from here? Any help is very much appreciated.

Nice topic and nice thoughts. This is a good motivational example of what GANs are good for. You got it right. Training a single ANN on data will not suffice to generate similar data. It is quite impossible. You will need at least 2 different networks that interact with each other, in the context of GANs: a discriminator and a generator. Generally spoken the discriminator is trained on your data and tells whether a produced 3D point cloud fits this training data. The generator actually generates the desired data and is trained on the feedback given by the discriminator. GANs were used for generating 2D images but one should also be able to use it for 3D point clouds with a fixed number of points.

*

8pla.net

  • Trusty Member
  • ***********
  • Eve
  • *
  • 1302
  • TV News. Pub. UAL (PhD). Robitron Mod. LPC Judge.
    • 8pla.net
Re: Using AI to Create 3D Point Clouds
« Reply #5 on: October 20, 2018, 02:42:34 pm »
Johnny (in part) wondered, "when you are done training the NN" Like anything "alive", even artificially so, as a Neural Network... Training is never done. But a computer program that simulates being "alive" is what makes it worthwhile, I think.
My Very Enormous Monster Just Stopped Using Nine

*

LOCKSUIT

  • Emerged from nothing
  • Trusty Member
  • *******************
  • Prometheus
  • *
  • 4659
  • First it wiggles, then it is rewarded.
    • Main Project Thread
Re: Using AI to Create 3D Point Clouds
« Reply #6 on: October 20, 2018, 03:59:40 pm »
i had seen a GAN that was 3D....may be what you are looking for. I could find the video, I think it is on Two Minute Papers. It shows a chair or beds made out of 3D red dots on white background, maybe try looking for that.
Emergent          https://openai.com/blog/

*

JohnnyWaffles

  • Roomba
  • *
  • 14
Re: Using AI to Create 3D Point Clouds
« Reply #7 on: October 22, 2018, 01:09:05 pm »
Hello everyone! Thank you for the replies and the suggestions on using a GAN. I like to do game development as a fun hobby on the side. I had the idea of using AI to produce a 3D model based on a host of models. So let’s say something as simple as a boulder. A boulder or rocks in general have their own design language and distinct look, and while we might consider a rock simple, modeling a convincing rock can be harder than you think. Unless of course you use scanned data like at www.megascans.se (which I’m considering).

Anyway, all that to say, why can’t an AI reproduce a convincing rock if you’re training data is sufficient. Theoretically I’m hoping that with the push of a button, it can iterate a variety of original designs. Now I used a rock as an example, but with that idea you could reproduce anything if you have enough specific training data. Trees, logs, stumps etc.

Maybe even more complicated models like cars or buildings for designing video game environments in software like Unreal Engine. With this kind of set-up you could batch process 3D models. Which would save me a lot of time!

Anyway, I had a realization, that using a 3d point cloud might be more complicated than necessary. Why not just use the vertices of a 3d model and use their 3d coordinates as input for the GAN?

Thank you for your time.

*

Korrelan

  • Trusty Member
  • ***********
  • Eve
  • *
  • 1454
  • Look into my eyes! WOAH!
    • YouTube
It thunk... therefore it is!...    /    Project Page    /    KorrTecx Website

*

AgentSmith

  • Bumblebee
  • **
  • 37
Re: Using AI to Create 3D Point Clouds
« Reply #9 on: October 22, 2018, 03:54:41 pm »
Anyway, I had a realization, that using a 3d point cloud might be more complicated than necessary. Why not just use the vertices of a 3d model and use their 3d coordinates as input for the GAN?

A voxel representation of the point cloud is also possible and actually even way more practical. In that case you can drop the 3D position information and you just have to provide the color values (binary black/white or RGB,...) for each voxel. All models should still have the same voxel sizes for all dimensions. You need to do some preprocessing such as removing outliers, affine transformations and so on.

 


OpenAI Speech-to-Speech Reasoning Demo
by ivan.moony (AI News )
Today at 01:31:53 pm
Say good-bye to GPUs...
by MikeB (AI News )
March 23, 2024, 09:23:52 am
Google Bard report
by ivan.moony (AI News )
February 14, 2024, 04:42:23 pm
Elon Musk's xAI Grok Chatbot
by MikeB (AI News )
December 11, 2023, 06:26:33 am
Nvidia Hype
by 8pla.net (AI News )
December 06, 2023, 10:04:52 pm
How will the OpenAI CEO being Fired affect ChatGPT?
by 8pla.net (AI News )
December 06, 2023, 09:54:25 pm
Independent AI sovereignties
by WriterOfMinds (AI News )
November 08, 2023, 04:51:21 am
LLaMA2 Meta's chatbot released
by 8pla.net (AI News )
October 18, 2023, 11:41:21 pm

Users Online

287 Guests, 0 Users

Most Online Today: 343. Most Online Ever: 2369 (November 21, 2020, 04:08:13 pm)

Articles