Is It Possible To Find Uncertainties Of Spacy Pos Tags?

I am trying to build a non-English spell checker that relies on classification of sentences by spaCy, which allows my algorithm to then use the POS tags and the grammatical depende

Solution 1:

>>> nlp = spacy.load("en_core_web_sm")
>>> tagger = nlp.get_pipe("tagger")
>>> doc = nlp("Turn left")
>>> tagger.model.predict([doc])[0][1]
array([2.4706091e-07, 9.5889463e-06, 7.8214543e-07, 1.0063847e-06,
       1.4711081e-07, 8.9995199e-05, 1.3229882e-05, 1.7524673e-07,
       1.8464769e-05, 2.4248957e-06, 1.2176755e-06, 3.3774859e-07,
       1.3199920e-06, 1.2011193e-06, 9.4455345e-06, 2.1991875e-05,
       1.6732251e-02, 1.3964747e-07, 2.0764594e-07, 7.0467541e-07,
       1.4303426e-07, 3.7962508e-07, 1.2130551e-03, 3.1479198e-07,
       4.8646534e-08, 6.1310317e-07, 1.0607551e-05, 3.7493783e-06,
       2.7809198e-08, 1.2118652e-05, 9.9081490e-03, 1.8219554e-06,
       4.7322575e-07, 1.8754436e-05, 6.2416703e-08, 9.5453437e-08,
       1.8937490e-05, 6.3916352e-03, 3.7999314e-01, 1.5741379e-03,
       5.8360571e-01, 9.6441705e-05, 1.7456010e-04, 5.1820080e-06,
       1.2672864e-06, 9.7453121e-06, 2.4000105e-05, 5.1192428e-06,
       2.4821245e-05], dtype=float32)
>>> r = [*enumerate(tagger.model.predict([doc])[0][1])]
>>> r.sort(key=lambda x: x[1])
>>> r
[(28, 2.7809198e-08), (24, 4.8646534e-08), (34, 6.24167e-08), (35, 9.545344e-08), (17, 1.3964747e-07), (20, 1.4303426e-07), (4, 1.4711081e-07), (7, 1.7524673e-07), (18, 2.0764594e-07), (0, 2.470609e-07), (23, 3.1479198e-07), (11, 3.377486e-07), (21, 3.7962508e-07), (32, 4.7322575e-07), (25, 6.1310317e-07), (19, 7.046754e-07), (2, 7.8214543e-07), (3, 1.0063847e-06), (13, 1.2011193e-06), (10, 1.2176755e-06), (44, 1.2672864e-06), (12, 1.319992e-06), (31, 1.8219554e-06), (9, 2.4248957e-06), (27, 3.7493783e-06), (47, 5.119243e-06), (43, 5.182008e-06), (14, 9.4455345e-06), (1, 9.588946e-06), (45, 9.745312e-06), (26, 1.0607551e-05), (29, 1.2118652e-05), (6, 1.3229882e-05), (8, 1.8464769e-05), (33, 1.8754436e-05), (36, 1.893749e-05), (15, 2.1991875e-05), (46, 2.4000105e-05), (48, 2.4821245e-05), (5, 8.99952e-05), (41, 9.6441705e-05), (42, 0.0001745601), (22, 0.0012130551), (39, 0.001574138), (37, 0.006391635), (30, 0.009908149), (16, 0.016732251), (38, 0.37999314), (40, 0.5836057)]

You see here the top 2 matches (at the end of the list) (38, 0.37999314), (40, 0.5836057) don't have a high confidence (~50%) so you have some indication of the ambiguity.

>>> tagger.labels
('$', "''", ',', '-LRB-', '-RRB-', '.', ':', 'ADD', 'AFX', 'CC', 'CD', 'DT', 'EX', 'FW', 'HYPH', 'IN', 'JJ', 'JJR', 'JJS', 'LS', 'MD', 'NFP', 'NN', 'NNP', 'NNPS', 'NNS', 'PDT', 'POS', 'PRP', 'PRP$', 'RB', 'RBR', 'RBS', 'RP', 'SYM', 'TO', 'UH', 'VB', 'VBD', 'VBG', 'VBN', 'VBP', 'VBZ', 'WDT', 'WP', 'WP$', 'WRB', 'XX', '``')
>>> tagger.labels[40]
'VBN'>>> tagger.labels[38]
'VBD'

Looks like there is some language-specific tagging, and some mapping is required to get to universal POS tags.