Can an output class be defaulted?












0












$begingroup$


In my use-case of multi-class classification, my data distribution is like below:
enter image description here



It might be too silly to ask this (and possibly could be gravely wrong), but is there a provision to default an o/p class to a value which is safe to be defaulted than to be predicted a completely wrong outcome.



Ex. Suppose the case, where an incoming email meant to be for "hardware" department but is predicted as for "Company Leadership" department and hence routed to all senior members of the company. In such case, since the prediction accuracy of the entire output class is below say 20% accuracy, I would like to default that entire class to "service desk" group and let them manually sort it.



Hope I made my question clear (Might be confusing as well!). Please let me know if any clarifications required. I would be happy to amend the wordings.



Thanks. :)






classification predictive-modeling multilabel-classification







share|improve this question









$endgroup$

















    0












    $begingroup$


    In my use-case of multi-class classification, my data distribution is like below:
    enter image description here



    It might be too silly to ask this (and possibly could be gravely wrong), but is there a provision to default an o/p class to a value which is safe to be defaulted than to be predicted a completely wrong outcome.



    Ex. Suppose the case, where an incoming email meant to be for "hardware" department but is predicted as for "Company Leadership" department and hence routed to all senior members of the company. In such case, since the prediction accuracy of the entire output class is below say 20% accuracy, I would like to default that entire class to "service desk" group and let them manually sort it.



    Hope I made my question clear (Might be confusing as well!). Please let me know if any clarifications required. I would be happy to amend the wordings.



    Thanks. :)






    classification predictive-modeling multilabel-classification







    share|improve this question









    $endgroup$















      0












      0








      0





      $begingroup$


      In my use-case of multi-class classification, my data distribution is like below:
      enter image description here



      It might be too silly to ask this (and possibly could be gravely wrong), but is there a provision to default an o/p class to a value which is safe to be defaulted than to be predicted a completely wrong outcome.



      Ex. Suppose the case, where an incoming email meant to be for "hardware" department but is predicted as for "Company Leadership" department and hence routed to all senior members of the company. In such case, since the prediction accuracy of the entire output class is below say 20% accuracy, I would like to default that entire class to "service desk" group and let them manually sort it.



      Hope I made my question clear (Might be confusing as well!). Please let me know if any clarifications required. I would be happy to amend the wordings.



      Thanks. :)






      classification predictive-modeling multilabel-classification







      share|improve this question









      $endgroup$




      In my use-case of multi-class classification, my data distribution is like below:
      enter image description here



      It might be too silly to ask this (and possibly could be gravely wrong), but is there a provision to default an o/p class to a value which is safe to be defaulted than to be predicted a completely wrong outcome.



      Ex. Suppose the case, where an incoming email meant to be for "hardware" department but is predicted as for "Company Leadership" department and hence routed to all senior members of the company. In such case, since the prediction accuracy of the entire output class is below say 20% accuracy, I would like to default that entire class to "service desk" group and let them manually sort it.



      Hope I made my question clear (Might be confusing as well!). Please let me know if any clarifications required. I would be happy to amend the wordings.



      Thanks. :)






      classification predictive-modeling multilabel-classification




      classification predictive-modeling multilabel-classification






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked 12 hours ago









      ranit.branit.b

      807




      807






















          1 Answer
          1






          active

          oldest

          votes


















          0












          $begingroup$

          I don't think there is a standard method to do this. But if you use a probabilistic model you can use the predicted probability together with thresholds on each class to only allow classifications that you deem certain enough. Then if the class with the highest probability does not meet the threshold you can set it to the default class.



          I tested it out with sklearn and numpy, this could be an approach:



          # Train probabilistic classifier
          
clf.fit(X_train, y_train)
          

          
# Get probabilities
          
probas = clf.predict_proba(X_test)
          

          
# Get the class with highest probability
          
highest_proba_class = np.argmax(probas, axis=1)
          

          
# Set different thresholds per class
          
thresholds = np.array([0.9, 0.2, 0.5])
          

          
# Init our prediction array
          
predictions = np.zeros_like(highest_proba_class)
          

          
# Set a default class to set if we don't reach threshold
          
default_class = 2
          

          
# Loop over predictions
          
for idx, highest_class in enumerate(highest_proba_class):
          

          
    # Threshold check if threshold was met, otherwise set default
          
    if probas[idx][highest_class] >= thresholds[highest_class]:
          
        predictions[idx] = highest_class
          
    else:
          
        predictions[idx] = default_class





          share|improve this answer











          $endgroup$














            Your Answer





            StackExchange.ifUsing("editor", function () {
            return StackExchange.using("mathjaxEditing", function () {
            StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
            StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
            });
            });
            }, "mathjax-editing");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "557"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: false,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: null,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f48702%2fcan-an-output-class-be-defaulted%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            1 Answer
            1






            active

            oldest

            votes








            1 Answer
            1






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            0












            $begingroup$

            I don't think there is a standard method to do this. But if you use a probabilistic model you can use the predicted probability together with thresholds on each class to only allow classifications that you deem certain enough. Then if the class with the highest probability does not meet the threshold you can set it to the default class.



            I tested it out with sklearn and numpy, this could be an approach:



            # Train probabilistic classifier
            
clf.fit(X_train, y_train)
            

            
# Get probabilities
            
probas = clf.predict_proba(X_test)
            

            
# Get the class with highest probability
            
highest_proba_class = np.argmax(probas, axis=1)
            

            
# Set different thresholds per class
            
thresholds = np.array([0.9, 0.2, 0.5])
            

            
# Init our prediction array
            
predictions = np.zeros_like(highest_proba_class)
            

            
# Set a default class to set if we don't reach threshold
            
default_class = 2
            

            
# Loop over predictions
            
for idx, highest_class in enumerate(highest_proba_class):
            

            
    # Threshold check if threshold was met, otherwise set default
            
    if probas[idx][highest_class] >= thresholds[highest_class]:
            
        predictions[idx] = highest_class
            
    else:
            
        predictions[idx] = default_class





            share|improve this answer











            $endgroup$


















              0












              $begingroup$

              I don't think there is a standard method to do this. But if you use a probabilistic model you can use the predicted probability together with thresholds on each class to only allow classifications that you deem certain enough. Then if the class with the highest probability does not meet the threshold you can set it to the default class.



              I tested it out with sklearn and numpy, this could be an approach:



              # Train probabilistic classifier
              
clf.fit(X_train, y_train)
              

              
# Get probabilities
              
probas = clf.predict_proba(X_test)
              

              
# Get the class with highest probability
              
highest_proba_class = np.argmax(probas, axis=1)
              

              
# Set different thresholds per class
              
thresholds = np.array([0.9, 0.2, 0.5])
              

              
# Init our prediction array
              
predictions = np.zeros_like(highest_proba_class)
              

              
# Set a default class to set if we don't reach threshold
              
default_class = 2
              

              
# Loop over predictions
              
for idx, highest_class in enumerate(highest_proba_class):
              

              
    # Threshold check if threshold was met, otherwise set default
              
    if probas[idx][highest_class] >= thresholds[highest_class]:
              
        predictions[idx] = highest_class
              
    else:
              
        predictions[idx] = default_class





              share|improve this answer











              $endgroup$
















                0












                0








                0





                $begingroup$

                I don't think there is a standard method to do this. But if you use a probabilistic model you can use the predicted probability together with thresholds on each class to only allow classifications that you deem certain enough. Then if the class with the highest probability does not meet the threshold you can set it to the default class.



                I tested it out with sklearn and numpy, this could be an approach:



                # Train probabilistic classifier
                
clf.fit(X_train, y_train)
                

                
# Get probabilities
                
probas = clf.predict_proba(X_test)
                

                
# Get the class with highest probability
                
highest_proba_class = np.argmax(probas, axis=1)
                

                
# Set different thresholds per class
                
thresholds = np.array([0.9, 0.2, 0.5])
                

                
# Init our prediction array
                
predictions = np.zeros_like(highest_proba_class)
                

                
# Set a default class to set if we don't reach threshold
                
default_class = 2
                

                
# Loop over predictions
                
for idx, highest_class in enumerate(highest_proba_class):
                

                
    # Threshold check if threshold was met, otherwise set default
                
    if probas[idx][highest_class] >= thresholds[highest_class]:
                
        predictions[idx] = highest_class
                
    else:
                
        predictions[idx] = default_class





                share|improve this answer











                $endgroup$



                I don't think there is a standard method to do this. But if you use a probabilistic model you can use the predicted probability together with thresholds on each class to only allow classifications that you deem certain enough. Then if the class with the highest probability does not meet the threshold you can set it to the default class.



                I tested it out with sklearn and numpy, this could be an approach:



                # Train probabilistic classifier
                
clf.fit(X_train, y_train)
                

                
# Get probabilities
                
probas = clf.predict_proba(X_test)
                

                
# Get the class with highest probability
                
highest_proba_class = np.argmax(probas, axis=1)
                

                
# Set different thresholds per class
                
thresholds = np.array([0.9, 0.2, 0.5])
                

                
# Init our prediction array
                
predictions = np.zeros_like(highest_proba_class)
                

                
# Set a default class to set if we don't reach threshold
                
default_class = 2
                

                
# Loop over predictions
                
for idx, highest_class in enumerate(highest_proba_class):
                

                
    # Threshold check if threshold was met, otherwise set default
                
    if probas[idx][highest_class] >= thresholds[highest_class]:
                
        predictions[idx] = highest_class
                
    else:
                
        predictions[idx] = default_class






                share|improve this answer














                share|improve this answer



                share|improve this answer








                edited 8 hours ago

























                answered 11 hours ago









                Simon LarssonSimon Larsson

                673113




                673113






























                    draft saved

                    draft discarded




















































                    Thanks for contributing an answer to Data Science Stack Exchange!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid …



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    Use MathJax to format equations. MathJax reference.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f48702%2fcan-an-output-class-be-defaulted%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    -

                    Popular posts from this blog

                    How to label and detect the document text images

                    Image
                    1 $begingroup$ This is what I mean as document text image: I want to label the texts in image as separate blocks and my model should detect these labels as classes. NOTE: This is how the end result should be like: The labels like Block 1, Block 2, Block 3,.. should be Logo, Title, Date,.. Others, etc. Work done: First approach : I tried to implement this method via Object Detection, it didn't work. It didn't even detect any text. Second approach : Then I tried it using PixelLink. As this model is build for scene text detection, it detected each and every text in the image. But this method can detect multiple lines of text if the threshold values are increased. But I have no idea how do I add labels to the text blocks. PIXEL_CLS_WEIGHT_all_ones = 'PIXEL_CLS_WEIGHT_all_ones' PIXEL_C...
                    Read more

                    Vallis Paradisi

                    Image
                    Coordinata: .mw-parser-output .geo-default,.mw-parser-output .geo-dms,.mw-parser-output .geo-dec{display:inline}.mw-parser-output .geo-nondefault,.mw-parser-output .geo-multi-punct{display:none}.mw-parser-output .longitude,.mw-parser-output .latitude{white-space:nowrap} 33°03′S 71°37′W  /  33.05°S 71.617°W  / -33.05; -71.617 Vallis Paradisi Res apud Vicidata repertae: Civitas : Chilia Numerus incolarum : 296 655 Zona horaria : UTC-4 Situs interretialis Nomen officiale : Valparaíso Gubernium Consilium : municipal council of Valparaíso Geographia Superficies : 401 600 000 Metrum quadratum Coniunctiones urbium Urbes gemellae : Ovetum , Badalona , Santa Fe , Ora longa , Sciamhaevum , Quantunia , Callao , Bat Yam , Melaka , Novorossiysk , Busan , Corduba , Odessa , Rosario , Malacca , Barcino , Soteropolis , Franciscopolis (California) , Guanajuato, Silao Tabula aut despectus Despectus in portum sinumque et urbem Vallis Paradisi Vallis Para...
                    Read more

                    Tabula Rosettana

                    Image
                    Tabula multilinguis Rosettana in Museo Britannico ostenditur. Tabula Rosettana, [1] etiam titulo OGIS 90 agnita, est stela decreto de rebus sacris in Aegypto anno 196 a.C.n. lato inscripta. Tabula iuxta Rosettam Aegypti, urbem in delta Nili et ad oram maris Mediterranei iacentem, anno 1799 a milite Francico reperta est. Inventio stelae, linguis duabus scripturisque tribus inscriptae, eruditis Instituti Aegypti statim nuntiata est; ibi enim iussu imperatoris Napoleonis eruditi omnium scientiarum (sub aegide Commissionis Scientiarum et Artium) properaverant cum expeditione Francica. Qua a Britannis mox debellata, tabula Rosettana Londinium missa hodie apud Museum Britannicum iacet. Textus Graecus cito lectus interpretationi textuum Aegyptiorum (in formis hieroglyphica et demotica expressorum) gradatim adiuvit. Denique textum plene interpretatus est Ioannes Franciscus Champollion. Ab opere eruditorum cumulativo coepit hodiernus scripturae hieroglyphicae linguaeque Aegyptiae a...
                    Read more