I have an idea which can be solved with machine learning, but no idea where to start

Submitted by excooo t3_zbpnzy in deeplearning

Hi,

so there is a user problem where someone has to manually create B from A. This is not deterministic, so B is always different. The process is tedious and could be solved through machine learning.

Examples:

A: 111222333 -> B: 123, but could also be 1123, 1223, etc. .. 123 would just be the better solution. The human process would just be to look at A and to decide what to remove in order to get the best result.

Since I have not the time to dig sooo deep into everything in machine learning, I would like to ask you if you can give me some hints on where to start my journey. What type of neural network, algorithms, keywords to google, etc.

To my background: I am a software engineer/devops/architect, so coding and understanding concepts is not that difficult for me.

I would appreciate your help.

0

Comments

You must log in or register to comment.

cadoi t1_iysj94d wrote

I thought software engineers were "professional googlers"?

Your description of the problem makes it sounds very similar to autocorrect. Maybe learn how that is done and adapt it?

No one is actually going to be of any help unless you more precisely phrase your problem.

4

excooo OP t1_iysnxov wrote

Can you explain how I should phrase my problem more precisely? What information would you need in order to understand it better? In an abstract way my example explains precisely what it should do. If you change the numbers with vectors you have the exact case.

0

cadoi t1_iystahu wrote

Your example as you have literally explained it entails just deleting all repeats (ambiguous if 121 -> 12 or 121, or for that matter if 121 is possible as an input). Either way a simple for loop could solve it. But you are a self proclaimed smart person who eats for loops for breakfast, so it seems your have over simplified your description of the problem.

Machine learning is essentially the following:

  • For sets X and Y, take a collection of some functions F = {f: X -> Y}
  • Take a probability distribution D on X x Y. Note this could be of the form (x, g(x)) for fixed unknown function g where only x is random.
  • Take a loss function L: F -> R, eg L(f) = ExpectedValue( distance(f(x),y) )
  • Sample data from the distribution
  • ???
  • Have an f in F that is close to minimizing L

So tell us what F, X, Y, D, L are.

3

hayAbhay t1_iysq7vu wrote

So vectors to vectors are extremely abstract and it's hard to understand what those vectors are. ML/DL are functions at the end of the day that are learned from observing a lot of examples of vectors -> vector transformations (inductive learning).

If these transformations are fairly easy to model, you might just be able to solve the function directly. Or you might need simple ML algorithms. If it's very complex, you might need stronger models and/or more data.

For instance image -> vector of probabilities over possible categories requires some powerful models where as a vector of 1 feature (height) -> probability of not hitting the head on the roof requires a basic model.

1

captaintel t1_iywh9xr wrote

Try to get an answer using either GTP-3 chat or GPT-3 playground, at any rate it could help you refine your question.

1

excooo OP t1_iyskf0h wrote

And so I also googled professionally beforehand and thought I'd also get the opinion of the pros without bias.

I will take a look at autocorrect, thank you :)
What I thought would be best are diffusion models, since denoising, etc. seems like the same concept.

−2

hayAbhay t1_iysnyi1 wrote

Again, unless you describe your actual problem, it's very hard to help. ML is a vast field with a lot of different approaches that come with their tradeoffs depending on the specific problem.

Simply throwing diffusers without understanding the space is like wanting to cut a cake but using a random household object like a chair to do so.

3

excooo OP t1_iysojd1 wrote

Like mentioned in another answer, it should convert an Array of vectors into another Array of different vectors. The vectors represent a 3d model and so the general shape of the model should stay the same, just represent through less vectors.

−1

onkus t1_iyux24i wrote

First of all. If you have no idea where to start then you cannot be certain your problem can be solved with ML.

If you want a useful answer then please ask an actual question.

1