lemmatize_doc ignores POS on pre-tokenized input.

[maayanorner]

trankit/trankit/pipeline.py

Line 980 in f6b916b

def _lemmatize_doc(self, in_doc, obmit_tag=False): # assuming input is a document

The issue is self-explanatory, this behavior is a bit unexpected.
I am not sure if it's a bug or a design decision.
It happens in _lemmatize_sent as well.

    def _lemmatize_doc(self, in_doc, obmit_tag=False):  # assuming input is a document
        if type(in_doc) == str:  # in_doc is a raw string in this case
            in_doc = self._tokenize_doc(in_doc)
            in_doc = self._posdep_doc(in_doc)

        lemmatized_doc = self._lemma_model[self._config.active_lang].predict(in_doc, obmit_tag)

        gc.collect()
        return lemmatized_doc

Possible fix (perhaps with some condition in cases where the data is already tagged):

    def _lemmatize_doc(self, in_doc, obmit_tag=False):  # assuming input is a document
        if type(in_doc) == str:  # in_doc is a raw string in this case
            in_doc = self._tokenize_doc(in_doc)
        in_doc = self._posdep_doc(in_doc)

        lemmatized_doc = self._lemma_model[self._config.active_lang].predict(in_doc, obmit_tag)

        gc.collect()
        return lemmatized_doc