Htydjtk

Problem: I would like to build a machine learning model that can predict the best candidate from any given set. What could be a good architecture for such a model?

Given: I have several training examples, each of which consists of:

- a set of candidates.

- a descriptor for the set as a whole.

- a label that tells which one of those candidates is the best in that set.

Details:

- The number of candidates in every set may be different.

- Every set is unordered.

- The descriptor of each set is currently a fixed length one-hot vector. I'm open to add more features to it though.

- Each candidate is represented by a fixed length feature vector. (However in future, the number of features describing each candidate may also differ for every candidate).

What I tried but didn't work:

One approach I tried was a simple MLP that takes one candidate as input and outputs whether or not the candidate is the best. But since this MLP wouldn't know which set the candidate belongs to, it fails in situations where a candidate is the best in one set but the same candidate is not the best in another set.

To get into some more specifics, in my current problem, each candidate is a 2D polygon with a fixed number of line segments. Labelling on the training examples is being done manually to pick the most good looking polygon in a given set of polylines. Each polygon is described by an array of (x,y) coordinates.

One problem I face is that I don't have a natural starting point for a polygon to begin it's array of (x,y) coordinates from. Currently I'm choosing the starting point to be the one with the minimum value of x+y and going counterclockwise from there.

Currently each 2D polygon has the same number of segments. But I would soon need to support polygons with varying number of segments.

In future, I would like to extend this ML model to 3D polyhedrons too, but I don't know how to even build a feature vector to describe for 3D polyhedron yet. I guess that's a problem for another day.

I think it would make more sense to train a model to grade (regression) each candidate, them from candidates of a particular set you can use the best candidate from its "grade".

Also, you should try changing the information from raw cloud of points to more meaningful geometric form irfomation:

• Number of vertices/segments


• Segments length mean and variance


• Skewness


• Size and direction of major and minor axis


• Center position


• Moments of area (first,second,third...)

Update

To apply a regression model you will need to generate grades for the training set and that might be a challenge. For that I will propose a few heuristics to generate this:

• 
  

  Since you have about 10k samples, you could assign to every candidate the probability of been the best candidate in any set (for example, if he is the best candidate in 10 sets you can give him a grade $frac{10}{10,000}$.

  
  
  
  
  

  • You could try clustering the samples and assigning a grade to every cluster as the probability of a candidate in that cluster been the best candidate by $frac{N_{best}}{N_{cluster}}$, where $N_{best}$ is the number of best candidates in any set in that cluster and $N_{cluster}$ is the number of candidates in that cluster

  
  

  • You can assign to each best candidate a grade like $1$ if he is best candidate in every set it appears and $1-e^{-alpha N}$ for every $N$ times it appears in a set without been the best candidate. You will have to tune the decay rate $alpha$ like any hyperparameter.