Who should use quanteda and what are the requirements?

quanteda is an R package for managing and analyzing text, and it is aimed at R users who need to apply natural language processing to texts, from documents through to final analysis. The main requirement is R programming knowledge; within that constraint, the package is designed so that powerful, efficient analysis takes a minimum of steps, and its consistent design is intended to lower the barriers to learning NLP and quantitative text analysis even for proficient R programmers. It suits researchers, students, and other analysts — particularly those with fewer financial resources — because its capabilities match or exceed those of many end-user software applications, many of which are expensive and not open source.

Who it is for

• R users who need to process text end to end. The package covers the path "from documents to final analysis," so you do not have to assemble a separate toolchain for each stage.
• Researchers and students. The project was created and maintained by Kenneth Benoit and Kohei Watanabe, funded by European Research Council grant ERC-2011-StG 283794-QUANTESS, with continued development supported by the Quanteda Initiative CIC.
• Analysts with limited budgets. The stated motivation is that comparable end-user software is often expensive and not open source, so quanteda is of "great benefit" to those with fewer financial resources.
• Proficient R programmers new to NLP. The API emphasizes consistent design specifically to lower the learning barrier for this group.

What you need before starting

The one explicit prerequisite in the project's own description is R programming knowledge. There is no claim that the package removes the need for R — only that it reduces the number of steps and the learning curve once you are in R.

Beyond that, what you need depends on which parts of the family you use:

The modular split happened progressively: quanteda.textmodels was split from the main package with the v2 release, and quanteda.textstats and quanteda.textplots were split with v3. The project notes that additional package releases are in progress.

What the core package covers

According to the package description, quanteda provides functionality for:

• corpus management
• creating and manipulating tokens and n-grams
• exploring keywords in context
• forming and manipulating sparse matrices of documents by features and feature co-occurrences
• analyzing keywords
• computing feature similarities and distances
• applying content dictionaries
• applying supervised and unsupervised machine learning
• visually representing text and text analyses

If your task falls outside this list, check whether it belongs to one of the companion packages before assuming it is unsupported.

Version 4 considerations

If you are deciding whether to adopt or upgrade, note that quanteda 4.0 is a major release that improves functionality and performance and further improves function consistency by removing previously deprecated functions. It also adds significant new tokeniser rules: new Unicode and ICU-compliant rules make the default tokeniser work more consistently across more languages. The project documents these changes in an article on the new external pointer tokens objects, an article with performance benchmarks for those objects and other v4 tokeniser improvements, and the v4 changelog listing all changes, improvements, and deprecations.

The practical implication: code that relied on functions deprecated before v4 will need updating, and multilingual tokenisation should behave more consistently than before.

How to decide

Use quanteda if you already work in R and want corpus management, tokenisation, and quantitative analysis in one consistent framework, especially if licensing cost is a constraint. Look elsewhere first if you do not know R and are not prepared to learn it, since R programming knowledge is the stated requirement. If you only need text models, text statistics, or text plots, install the corresponding companion package rather than expecting those functions in the core package after the v2 and v3 splits.