[wormtrex]

Hello Everyone,

This is my first post here. I have been in software development for almost 20 years (if you count grad school projects). I have never worked on any AI projects and but am planning start one in about 2 years. Please excuse any terminology or incorrect AI assumptions I am making here.

The intent of this post is to get my brainstorming ideas down on paper and ask how other people have tackled a problem like this.

My company has an online platform that tracks a user’s progress through about 50 actions that occur on our online platform. A user may complete all of these actions in order, or they may end up skipping some actions. The entire process may take some customers only a day and others a few months. After going over all of the answers and documentation given in each step and an extensive onsite visit a human makes a decision on a customer passing or failing the process. 

The entire process is too heavily tied to human interaction to model but I believe the correlation between how long it takes a user to complete each step and what order they are completed can give me insight into the likely hood they will ultimately pass or fail the entire process.

The goal of our completed model is to display a percentage value of predicted success or failure to our internal customer liaisons during the time a customer is completing all of the steps of the entire process.

Our system is hosted in Azure and our data is in an Azure SQL database. Ideally our model would be integrated directly into our Azure SQL database and would run each time a process step is entered or changed by a given customer.

We will have a relatively small dataset of about 5 thousand customers/year to train and test our model on.

• What AI platform would you use for a project like this if you were starting from scratch as a first AI project?
• What training sites/materials would you recommend getting up to speed on before starting this project?
• Would you be worried that a dataset of only a few thousand records would be too small to predict meaningful outcomes?
• To start we wouldn’t assume weights of completing step and just assume they are all equal in importance leading to the ultimate outcome. Will it be possible to easily determine why the model decides an outcome percentage?
  

[MagnusWootton]

That looks good.  Seems like you want to "build a model" which is what Ai is about I think,  instead of manually building the model,  the computer works it out itself,  hence the "automation" thats happening for you.

So you want to predict the failure of the customers -  that means that u need to have it explicit the detection of these failures.  (you need a direct compute looking at the 50 or so actions, and u have to be able to **simply tell** which ones did better than the others,  thats one of the restrictions of doing Ai like this.)

Another thing you could do, which could be interesting,  is you make a computer "customer" and it will brute force strings of the 50 actions, and try to invent a successful customer itself.   all you need is the "way to measure success"  then its just a matter of waiting for it,     if you make the string too long, you dont have a chance,  it can only be so big a "customer key"  because to do linear search, is actually paying things exponentially, and u get diminished returns the more complex the creature is.

But if you keep the customer simple and small,  it does work like magic!   But if you go for too much, youll see the limits of machine learning straight away.


To answer a question about what programming language to use, or what hardware,   Personally I do everything with Direct Compute in C, and I get GPU performance,  only thing better than that I think is FPGA's,  but its only if you know how to simplify your algorythms alot, or your probably better off with a GPU.

I think if you do simplify every operation down beautifully an FPGA probably outperforms a GPU dollar for dollar,   but its close if its true or not.

[wormtrex]

Thanks for the reply Mangus,

Our model will need to run in the Azure cloud and ideally be executed each time a customer completes a step in our app.  That will probably end up being something like every 10 minutes or so during 9-5 business hours. 

I am currently thinking of the model as being very simple.  It would simply look at the time it took to complete each step and if steps were skipped.  From these inputs it would output a percentage value that the customer is likely to succeed or fail.  Our training data would have historical steps for thousands of customers and if they ultimately succeeded of failed.

When the model output doesn't align with what really happened we would then try to analyze why the model made its percentage decision and how we could tune the model further.  It is likely that time it takes to complete certain steps has much more impact on the outcome than others. 

[MagnusWootton]

I see,  this looks pretty doable, practicality wise, not like insane intelligent undetectable chat bots, then its going to take you decades.   If you havent got a gpu for it,  and uve only got an ordinary cpu  it should be enough processing if you know how to code well.   

But I dont quite understand the situation fully.
So what do you mean by  customer success?  is it if they baught your product or not?

But it is true, you can collect the activities of the customers - and relate it to something like that - or even other things, as long as they are simply detectable.

You predict B with A.    thats the idea,   then once u have a model, you can now simulate using it,  but thats a little trickier to get going, but its all theoretically possible.

[infurl]

Hi wormtrex and welcome to the forum.

I've been thinking about your project and it seems like it is the kind of problem that could be solved using the Non-axiomatic Reasoning System (NARS). You can find out more about NARS at the following websites. It is open source software and freely available. I haven't used it myself for anything yet but have been learning what I can about it. It seems like a very practical AI solution to many sorts of problems and doesn't require a huge investment in training.

https://www.opennars.org/

https://cis.temple.edu/~pwang/NARS-Intro.html

[wormtrex]

Thanks Infurl,

I will check that out.  I had tensor flow planted in my mind as the tool we were going to use on this but I wanted to see what others would recommend.

Mangus,

To simplify our product, we essentially ask for a customer to fill out a ton of information in an online system.  The data entry is grouped into  the steps I talked about.  Some of the steps are file uploads or things that our internal support specialists do.  Once all of the data is gathered and a customer has completed the entire process we determine if they pass or fail the the entire process.  This decision of pass or fail is made by a human, it requires a human to physically visit the customer and is far to complicated to try to fully automate.  I believe we can make some assumptions about if a customer is having problems during this process and has a high probability of passing or failing by the time it takes them to complete each step.  I also believe we may have a correlation between customers skipping certain steps and this leading to high probably of them passing or failing. 

The goal of the model is to show our internal support specialists a percentage value of probability that the model says they will ultimately pass or fail.  This would be used by the support specialists to reach out to customers that the model is saying they have a high probability of passing or failing. 

At the end of the day I really only want to evaluate the time difference between each step in this process and if certain steps are skipped to then let the model decide a correlation of passing or failing from those inputs.  This model would need to be run each time a customer completes a new step and then output the current probability of passing or failing for that customer given all of their completed steps. 

[ivan.moony]

Hi Wormtrex

There exist algorithm synthesis methods that are trained just like artificial neural networks, but they spit out exact algorithm that functionally pair each input with each output, providing a method to grock yet unfed inputs. Nevertheless, in your case, I think there are so much underlying unknown parameters that existence of such an algorithm is questionable, other than statistical methods provided by ANNs. So, yes, I think ANNs might bring some fair results in your case, better than universal algorithm synthesis.

But there are always a good old fashioned programming dirty tricks like inventing your own algorithm powered by statistics (or something), that would calculate results of your interest. In this case, it is not guaranteed that there will be no misconceptions or bugs, contrasting to ANNs that aim to be more precise if conveniently trained on bigger example range.

ANNs are really a wonder of modern days in a way that they magically find a right way to pair inputs to outputs, even though mysterious enough to crypt the ad hoc methods they occasionally invented to provide their output. There are some academic attempts to decipher a structure of what ANNs just learned, but I don't know to what extent they work as intended. It would be great if ANNs could explain themselves on their own, answering how they got the desired results. Maybe this would correspond to self-awareness issue?

[wormtrex]

Thanks Ivan,

We could completely come up with a custom algorithm to accomplish this.  For this project there are 2 main reasons that I wanted to avoid that approach.

We are a medium sized organization (200ish people).  To get internal consensus on how to do this would require an endless amount of meetings around how to evaluate each step and the entire process.  By essentially keeping the model in the blackbox of an ANN would allow us to avoid that process.  We would then just need to have a means to get consensus around validating the output. 

Many of the steps in this process cross different business teams inside of our company.  Those teams don't have a good understanding of the steps outside of their responsibility.  By using an ANN to evaluate all steps evenly without user bias I believe will yield a better overall model.

[MagnusWootton]

Oh I see now.    I think that all you would need to do is pair the report with the success as the output,  and K-nearest neighbours would work fine.

That would be the easiest way to do it,      Just get the query report, and find the closest database report, and that should decide the success of the query.

I could probably write that in a day,   A gpu however would improve the performance a great deal.
Its best to write it bitwise too,  makes it 32x faster if its a 32 bit machine.   

Actually is a good beginner project!!

[wormtrex]

Thanks Mangus,

I was hoping this would be a fairly easy first project.  As we evaluate the expected output against what really happened we may end up giving certain steps a higher weight in the process.

This is going to be run in Azure on a job so I won't have any control over the hardware.