How yasbd-lib Reached v1.0 After Stress-Testing 39 Languages
Before 0.12.0, yasbd-lib was a promise. 39 languages, spaCy and pysbd integrations, a CLI, a streaming API, a benchmark suite. Every one of those language profiles took real time to build because we wanted them to survive actual text, not just the sentences we wrote to test them.
Reader, they did not survive actual text.
The fixes in this post come from a lot of people. Some opened a PR with the fix already in it. Some fixed a rule I wrote and got wrong. Names are credited where they did the work.
So we stopped adding languages. We froze the set at 39, locked the door (issue #198), and spent the next few months breaking everything we'd built. That's what this post is about.
If you're here for the pysbd comparison instead, that's a different post: yasbd-lib vs. pysbd: Two Philosophies of Sentence Boundary Detection.
— yasbd-lib...
Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE
