lexical-analysis
A grammar describes the syntax of a programming language, and might be defined in Backus-Naur form (BNF). A lexer performs lexical analysis, turning text into tokens. A parser takes tokens and builds a data structure like an abstract syntax tree (AST). The parser is concerned with context: does the sequence of tokens fit the grammar? A compiler is a combined lexer and parser, built for a specific grammar.
Here are 472 public repositories matching this topic...
百度NLP:分词,词性标注,命名实体识别,词重要性
-
Updated
May 25, 2021 - C++
xmnlp:提供中文分词, 词性标注, 命名体识别,情感分析,文本纠错,文本转拼音,文本摘要,偏旁部首,句子表征及文本相似度计算等功能
-
Updated
Nov 12, 2022 - Python
DFA regular expression library & friends
-
Updated
Dec 3, 2022 - C
Chinese text segmentation with R. R语言中文分词 (文档已更新
-
Updated
Jul 13, 2020 - C++
A lexical analyzer based on DFA that is built using JS and supports multi-language extensions / 一个基于DFA的支持多语言扩展的JS版开源词法分析器
-
Updated
Jan 17, 2023 - JavaScript
Allocators, I/O streams, math, geometry, image and audio processing for D
-
Updated
Nov 16, 2022 - D
OysterKit is a framework that provides a native Swift scanning, lexical analysis, and parsing capabilities. In addition it provides a language that can be used to rapidly define the rules used by OysterKit called STLR
-
Updated
Aug 15, 2019 - Swift
LLVM Tutorial: Kaleidoscope (Implementing a Language with LLVM)
-
Updated
Dec 29, 2022 - C++
Implementing a complete Compiler for a simple C-like language using the C-tools Flex and Bison
-
Updated
Oct 20, 2020 - C
Santiago is a lexing and parsing toolkit for Rust
-
Updated
Aug 28, 2022 - Rust
clex is a simple lexer generator
-
Updated
Nov 20, 2022 - C
LingFeat - A Comprehensive Linguistic Features Extraction ToolKit for Readability Assessment
-
Updated
Jan 13, 2023 - Python
A Lexical Analyzer Generator
-
Updated
Apr 10, 2019 - Common Lisp
-
Updated
Jan 11, 2023 - Python
A compiler that accepts any valid program written in C. It is made using Lex and Yacc. Returns a symbol table, parse tree, annotated syntax tree and intermediate code.
-
Updated
Apr 3, 2022 - Yacc
These are my programs for compiler design lab work in my sixth semester
-
Updated
May 25, 2022 - Python
Platform for parser generators and other grammarware research and development. GLL, RNGLR, graph parsing algorithms, and many others are included.
-
Updated
Jul 25, 2019 - F#
- Wikipedia
- Wikipedia

