pymystem3

A Python wrapper of the Yandex Mystem 3.1 morphological analyzer (http://api.yandex.ru/mystem). The original tool is shipped as a binary and this library makes it easy to integrate it in Python projects. Let us know in the issues if you would like to be involved into the developments or maintenance of this project. If you have any fix or suggestion, please make a pull request. We are very open to accepting any contributions.

#language#lemma#lemmatization#lemmatizer#morphological-analyser#morphological-analysis

View Repository on GitHub ↗

REPOSITORY OVERVIEW

Live repository statistics

★ 293Stars

⑂ 43Forks

◯ 14Open issues

◉ 293Watchers

64/100

OPENREPOHUB HEALTH SIGNAL

Mixed signals

A transparent discovery signal based on current public GitHub metadata.

Recent activity35% weight

Community adoption25% weight

Maintenance state20% weight

License clarity10% weight

Project information10% weight

100

This score does not audit code, security, maintainers, documentation quality, or suitability. Verify the repository and its current documentation before adoption.

README preview

================================================================== A Python wrapper of the Yandex Mystem 3.1 morphological analyzer

.. image:: https://travis-ci.org/nlpub/pymystem3.png?branch=master :target: http://travis-ci.org/nlpub/pymystem3 :alt: Build Status

Introduction

This module contains a wrapper for an excellent morphological analyzer for Russian language Yandex Mystem 3.1 <https://tech.yandex.ru/mystem/>_ released in June 2014. A morphological analyzer can perform lemmatization of text and derive a set of morphological attributes for each token. For more details about the algorithm see I. Segalovich «A fast morphological algorithm with unknown word guessing induced by a dictionary for a web search engine» <http://download.yandex.ru/company/iseg-las-vegas.pdf>_, MLMTA-2003, Las Vegas, Nevada, USA.

Python is the language of choice for many computational linguists, including those working with Russian language. The main motivation for this development was absence of any Python wrapper for the Mystem, a one of the most popular morphological analyzers for Russian language along with the PyMorphy2 <https://github.com/kmike/pymorphy2>, the TreeTagger <http://corpus.leeds.ac.uk/mocky/> and AOT <http://www.aot.ru/download.php>_.

The third version of Mystem introduces several importaint improvements, most importaintly part-of-speech disambiguation. Our wrapper runs the Mystem in the mode which performs POS disambiguation.

This wrapper is open sources under MIT license. However, please consider that the Yandex Mystem is not open source and licensed under conditions of the Yandex License <http://legal.yandex.ru/mystem/>_.

System Requrements

The wrapper works with CPython 2.6+/3.3+ and PyPy 1.9+.

The wrapper was tested on Ubuntu Linux 12.04+, Mac OSX 10.9+ and Windows 7+.

For 32bit architectures and freebsd platform support use ver. 0.1.10.

Installation

Stable version: https://pypi.python.org/pypi/pymystem3. You can install it using pip::

pip install pymystem3

.. * Documentation: http://pythonhosted.org/pymystem3

Latest version (recommended): ::

>>> from pymystem3 import Mystem
>>> text = "Красивая мама красиво мыла раму"
>>> m = Mystem()
>>> lemmas = m.lemmatize(text)
>>> print(''.join(lemmas))
красивый мама красиво мыть рама

import json
from pymystem3 import Mystem

text = "Красивая мама красиво мыла раму"
m = Mystem()
lemmas = m.lemmatize(text)

print ("lemmas:", ''.join(lemmas))
print ("full info:", json.dumps(m.analyze(text), ensure_ascii=False))

lemmas: красивый мама красиво мыть рама

full info: [{"text": "Красивая", "analysis": [{"lex": "красивый", "gr": "A=им,ед,полн,жен"}]}, {"text": " "}, {"text": "мама", "analysis": [{"lex": "мама", "gr": "S,жен,од=им,ед"}]}, {"text": " "}, {"text": "красиво", "analysis": [{"lex": "красиво", "gr": "ADV="}]}, {"text": " "}, {"text": "мыла", "analysis": [{"lex": "мыть", "gr": "V,несов,пе=прош,ед,изъяв,жен"}]}, {"text": " "}, {"text": "раму", "analysis": [{"lex": "рама", "gr": "S,жен,неод=вин,ед"}]}, {"text": "\n"}]