Htydjtk

I have a dataset that has a large number of categorical features and a few numerical features, and I want to predict the probability that any given input is one of two types of a certain binary output feature. I don't want to solve a classifier problem for reasons that are well outlined in this link .

I have seen many tutorials about how to handle them independently but am less sure how to handle them together.

I don't want to solve a classifier problem for reasons that are well outlined in this link.

I doubt that the link wants to tell you to stop performing classification tasks - the problem you propose is a classical example for classification. How I understand your source, it does not want you to use scoring rules as a heuristic.

For the problem you described I would propose a simple Naive Bayes approach. To make your numerical values discrete, you can simply use the mean of two adjacent numeric values as a treshold. E.g. for a list [1, 2] of numeric values, just split them at a threshold of 1.5 and check above and below.

One could approach this in two general ways:

1) bottom up: thinking about unifying the data somehow to begin with

2) top down: deciding how the data needs to look based on the final model you wish to use

Do you already know which model you will use? If that is fixed (for whatever reason), you already know you need to get your data into the correct form, be it numerical or categorical.

As you pinned your question with the tag regression, I can tell you that you need to make your data all numerical, so regression can work.

An example of making numerical data categorical would be to put it into bins. Imagine we have values ranging from zero to ten: [0.173, 7.88, 3.91, ...]. You could simply say that values between 0.00 and 0.99 are category A, values between 1.00 and 1.99 are category B, and so on.

[Edit:]

A slightly more sophisticated way of defining the bins to use would be to define the bins based on some characteristic statistics of your dataset. For example, have a look at the possible ways possible implemented within python's Numpy. Of the available methods there, I have found the Doane method to work best - it will depend on your data though, so read the descriptions.

Making categorical values numerical in a meaningful way depends a little more on you data. It is easy to make them numberic, but you should focus on doing it in such a way as to retain as much of the information each variable contains as well as the relative relationships between each of the categories that you started with. E.g. converting colours into integers would allow you to perform regression, but if yellow becomes 1 and purple 10, the model needs to be able to learn that purple isn't necessarily 10 times bigger than yellow, and that is difficult in the context of regression!

Adding to the answers of above, there's one more better way to do the same i.e Target Encoding, which naively means you are e encoding your cats according to the target variable via using some aggregate (works out of the box)