Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[itn] add whitelist #263

Merged
merged 2 commits into from
Jul 4, 2024
Merged

[itn] add whitelist #263

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3aca18a
[itn] add whitelist
xingchensong Jul 4, 2024
4f94a40
[itn] add whitelist
xingchensong Jul 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions itn/chinese/data/default/whitelist.tsv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,3 +61,18 @@
五台山 五台山
六盘水 六盘水
八宿 八宿
十二五 十二五
十三五 十三五
十四五 十四五
几十万 几十万
几百万 几百万
几千万 几千万
十几万 十几万
二十几万 二十几万
三十几万 三十几万
四十几万 四十几万
五十几万 五十几万
六十几万 六十几万
七十几万 七十几万
八十几万 八十几万
九十几万 九十几万
1 change: 1 addition & 0 deletions itn/chinese/test/data/normalizer.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,3 +30,4 @@
可以拨打幺二三零六来咨询 => 可以拨打12306来咨询
二点五平方电线,五,五十五,疑是银河落九天,十二块五 => 2.5平方电线,5,55,疑是银河落9天,12块5
三百九十九三盒 => 3993盒
十三五规划期间获得了十几万和几十万甚至二十几万的投资 => 十三五规划期间获得了十几万和几十万甚至二十几万的投资
3 changes: 2 additions & 1 deletion itn/chinese/test/data/normalizer_disable_standalone_number_disable_0_to_9.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,4 +38,5 @@
这是零百 => 这是零百
这是零千 => 这是零千
这是一百一个，一千两位，一万三天 => 这是100一个，1000两位，10000三天
这是九百九周，九千九月，九万九年 => 这是900九周，9000九月，90000九年
这是九百九周，九千九月，九万九年 => 这是900九周，9000九月，90000九年
十三五规划期间获得了十几万和几十万甚至二十几万的投资 => 十三五规划期间获得了十几万和几十万甚至二十几万的投资
3 changes: 2 additions & 1 deletion itn/chinese/test/data/normalizer_disable_standalone_number_enable_0_to_9.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,5 @@
这是零百 => 这是零百
这是零千 => 这是零千
这是一百一个，一千两位，一万三天 => 这是100 1个，1000 2位，10000 3天
这是九百九周，九千九月，九万九年 => 这是900 9周，9000 9月，90000 9年
这是九百九周，九千九月，九万九年 => 这是900 9周，9000 9月，90000 9年
十三五规划期间获得了十几万和几十万甚至二十几万的投资 => 十三五规划期间获得了十几万和几十万甚至二十几万的投资
3 changes: 2 additions & 1 deletion itn/chinese/test/data/normalizer_enable_standalone_number_disable_0_to_9.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,4 +38,5 @@
这是零百 => 这是零百
这是零千 => 这是零千
这是一百一个，一千两位，一万三天 => 这是100一个，1000两位，10000三天
这是九百九周，九千九月，九万九年 => 这是900九周，9000九月，90000九年
这是九百九周，九千九月，九万九年 => 这是900九周，9000九月，90000九年
十三五规划期间获得了十几万和几十万甚至二十几万的投资 => 十三五规划期间获得了十几万和几十万甚至二十几万的投资