How to Use Esprima to Tokenize or Parse JavaScript Code

Esprima exposes two entry points for analyzing JavaScript source: esprima.tokenize(program) for lexical analysis and esprima.parse(program) for syntactic analysis. Both are available after loading the library in a Node.js environment with require('esprima'). Tokenization returns a flat list of tokens; parsing returns a syntax tree object you can traverse for static analysis or feed into code transformation.

Prerequisites

• A Node.js runtime, since the documented examples run in the Node.js REPL.
• The esprima module installed and resolvable by require.
• A JavaScript source string to analyze. In the examples below it is a single statement, but the same calls accept larger programs.

Step 1: Load the library

var esprima = require('esprima');

This binds the module to a local variable. All subsequent calls go through that object.

Step 2: Tokenize with esprima.tokenize

Tokenization performs lexical analysis: it breaks the source into tokens without building a tree.

var program = 'const answer = 42';
esprima.tokenize(program);

Expected result:

[
  { type: 'Keyword', value: 'const' },
  { type: 'Identifier', value: 'answer' },
  { type: 'Punctuator', value: '=' },
  { type: 'Numeric', value: '42' }
]

Each entry carries a type (for example Keyword, Identifier, Punctuator, Numeric) and the raw value. Use this when you only need the token stream — for example highlighting identifiers or scanning for specific token categories.

Step 3: Parse with esprima.parse

Parsing performs syntactic analysis and returns a syntax tree.

esprima.parse(program);

Expected result (abbreviated):

{
  type: 'Program',
  body: [
    {
      type: 'VariableDeclaration',
      declarations: [Object],
      kind: 'const'
    }
  ],
  sourceType: 'script'
}

The root node is a Program with a body array of statements. Nested nodes are shown as [Object] in the REPL; inspect them directly or serialize the tree to see the full structure.

Step 4: Use the output

Once you have the full syntax tree, the documented use cases are:

• Static analysis — inspect node types and positions to reason about the code.
• Syntax visualization — render the tree structure.
• Code validation — check whether the source parses cleanly.
• Editing autocomplete — with type inferencing.
• Code transformation — regenerate source from the tree, from simple reformatting to minification.

Tokenize vs. parse: which to call

Choose tokenize when token categories and values are enough. Choose parse when you need structure — statement kinds, nesting, and the relationships that transformation and deeper analysis depend on.

Notes on scope

Esprima is described as a high-performance, standard-compliant ECMAScript parser written in ECMAScript, with full support for ECMAScript 2019 (ECMA-262 10th Edition), a sensible syntax tree format with optional node location info, and experimental JSX support. It is maintained by Ariya Hidayat, and the project documents its API, usage, and videos separately from this quick example.