The Wayback Machine - https://web.archive.org/web/20201018101312/https://github.com/pytorch/text/pull/997
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use union regex for basic_english #997

Draft
wants to merge 1 commit into
base: master
from
Draft

Conversation

@cpuhrsch
Copy link
Collaborator

@cpuhrsch cpuhrsch commented Sep 21, 2020

Create a single union regular expression that uses a lambda to query a dictionary of patterns for the correct replacement. This causes a significant speedup, however is different from the current behavior where the substitutions are applied consecutively instead of in parallel all at once. We might want to hide this behavior behind a flag, but it is useful and can yield significantly faster code.

Christian Puhrsch
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
1 participant
You can’t perform that action at this time.