What Is the Software Techniques Parsing Tool (STPT) and When Do You Need It?

STPT (Software Techniques Parsing Tool) is a data parsing software product from Software Techniques Inc., a Toronto-based firm that also provides Visual Basic and MATLAB programming services. In short, STPT exists to solve one specific and stubborn problem: turning messy, inconsistent, or semi-structured text into clean, structured data that a program can actually use. If you have ever written a pile of string-splitting code to read a log file, a fixed-width report, or a vendor data feed, STPT is the category of tool built to replace that code.

The core problem STPT addresses

Most data in the real world does not arrive as tidy CSV. It arrives as:

• Fixed-width records from mainframes and legacy systems
• Delimited files where the delimiter appears inside quoted fields
• Log files with variable numbers of fields per line
• Reports designed for humans, not machines
• Mixed formats where some records follow one layout and others follow another

Hand-written parsing code handles the easy 80% quickly and the remaining 20% badly. That last portion is where projects stall: an unexpected blank field shifts every column, a quoted comma breaks a split, or a new record type appears and the parser silently produces wrong values instead of failing loudly.

A parsing tool like STPT is designed to describe the structure of the input declaratively — field positions, delimiters, record types, repeating groups — and then apply that description consistently to every record. The value is not that it parses faster than your code. The value is that the parsing rules live in one place, are testable, and can be updated when the input format changes without rewriting program logic.

When a parsing tool beats hand-written string manipulation

Consider a parsing tool when any of the following is true:

The rule of thumb: if you are writing more than a few dozen lines of splitting and trimming logic, or if you have already fixed the same parser twice, a declarative tool pays for itself.

What a parsing tool should provide for real-world feeds

When evaluating STPT or any comparable parser, check for these capabilities, because they are what separate a demo from something you can run in production:

1. Field definition by position, delimiter, or pattern. Real inputs mix all three.
2. Multiple record layouts in a single stream. Header, detail, and trailer records are the norm, not the exception.
3. Repeating groups. Line items, tags, or nested blocks that appear zero or more times.
4. Quoting and escape handling. So a comma inside a quoted name does not split a field.
5. Type conversion and validation. Dates, numbers, and codes checked at parse time, not three steps downstream.
6. Clear error reporting. Which record, which field, what was expected — not just "parse failed."
7. An output structure your code can consume. Records, arrays, or objects rather than raw substrings.
8. Reusability. The same rule set applied to tomorrow's file without recompiling.

If a tool lacks items 2, 4, or 6, expect to write workarounds, which defeats the purpose.

How a parsing tool relates to custom programming services

This is the part people often get backwards. A parsing tool and custom programming are not competitors; they operate at different layers.

• The parsing tool answers: what is the structure of this input, and how do I extract fields reliably?
• Custom Visual Basic or MATLAB code answers: what do I do with the extracted data — model it, plot it, feed it into a simulation, or drive a business process?

A typical division of labor looks like this:

1. STPT (or an equivalent parser) reads the raw feed and produces structured records.
2. Your Visual Basic application consumes those records for data entry, reporting, or integration.
3. Your MATLAB code consumes the same records for numerical analysis, signal processing, or visualization.

The benefit of this split is that format churn stays in the parsing layer. When a vendor changes their file layout, you update the parse rules and your analysis code keeps working. If parsing logic is scattered through your MATLAB scripts, every format change becomes a debugging session across the whole codebase.

Do you need STPT, or a custom-built solution?

Work through these questions in order:

1. Is the format stable and simple? If yes, a short custom routine is the right answer. Do not buy a tool for a problem you solve in twenty lines.

2. Is the format complex but well understood? Multiple record types, repeating groups, or quoting rules point toward a parsing tool. This is STPT's home territory.

3. Is the format undocumented or genuinely irregular? If nobody can state the rules, no tool will save you. You need an exploratory phase — often custom programming work — to reverse-engineer the structure before you can declare parse rules.

4. Do non-developers need to maintain the rules? If the people closest to the data are analysts rather than programmers, a configurable parsing tool reduces the bottleneck on the development team.

5. How often will the format change? Frequent change favors a declarative tool. A one-time migration favors custom code.

6. What does your existing stack look like? If you already have Visual Basic or MATLAB code doing the downstream work, a parser that integrates cleanly with that stack avoids a rewrite. If your stack is entirely different, weigh integration effort honestly.

Practical factors before you commit

• Input variety. Count the distinct formats you must handle today and expect at least one more next year.
• Maintenance ownership. Decide now who updates parse rules and how those changes are tested.
• Integration surface. Confirm how the parser hands data to your existing code — file, library call, or in-memory structure.
• Failure behavior. Test what happens with truncated records, unexpected blank lines, and encoding changes. A parser that guesses is worse than one that stops.
• Licensing and support. Pricing and purchase terms for STPT are not published in the material referenced here; contact Software Techniques Inc. directly for current commercial details rather than assuming a model.

A reasonable starting point

If you are unsure, run a small pilot. Take one real, messy input file — not a cleaned sample — and try to express its structure as parse rules. If the rules stay readable and the parser flags bad records clearly, you have a fit. If you spend the pilot writing escape hatches around the tool, you have learned something useful too: your problem is custom programming, not off-the-shelf parsing.

Either way, keep the parsing layer separate from the analysis layer. That single decision will save you more time than any specific tool choice.