-
Notifications
You must be signed in to change notification settings - Fork 140
Hindi TN: Address Class (context + structural) #359
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
shreeshd-tn
wants to merge
49
commits into
NVIDIA:staging_hi_tn
Choose a base branch
from
shreeshd-tn:staging_hi_tn
base: staging_hi_tn
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+513
−30
Open
Changes from all commits
Commits
Show all changes
49 commits
Select commit
Hold shift + click to select a range
d0a0db7
Ordinals class done
shreeshd-tn 2fceb3d
Merge pull request #1 from shreeshd-tn/ordinals
shreeshd-tn 36866b2
Future Implementations for classes - Measure, Money, and Date (#258)
ngachchi da87d4c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 1a2c48e
Potential fix for code scanning alert no. 821: Unused local variable
mgrafu 513db90
Hindi TN Future Implementations 2.0. - Fraction, Measure and Time (#310)
ngachchi 0d3f383
Hindi TN 2.0 - Telephone class integration from staging branch (#320)
shreeshd-tn d8009c8
template files for new address class
shreeshd-tn 4d2c52d
Basic address structure
shreeshd-tn d5f2726
basic functionality
shreeshd-tn dd85e25
slight improvement - 4 to 19 percent
shreeshd-tn 6b0543e
Suffix based approach
shreeshd-tn f2bb68a
Address is working at the cost of cardinal
shreeshd-tn e0c0a61
telephone_copy
shreeshd-tn d74c975
Good checkpoint, but sparrowhawk is failing
shreeshd-tn 9b48ae9
Context + Complex numbers matching
shreeshd-tn a081e85
Context based and improved format matching
shreeshd-tn 39b0368
formatting changes and new context words
shreeshd-tn 157927c
Fully tested ordinals
shreeshd-tn 10f24c0
address is good, but telephone was affected
shreeshd-tn 7adf096
simplified context + weight changes
shreeshd-tn b484d14
Working greedy address + regressions
shreeshd-tn fcebf16
Merge staging Hindi TN v2 to main (#346)
mgrafu 62d60c1
Merged with latest main
shreeshd-tn 30a6123
Context + window
shreeshd-tn 2d2a70e
Messy working code
shreeshd-tn 844212d
Cleaned code ready for PR
shreeshd-tn 54f23d1
Merge branch 'address_context' into staging_hi_tn
shreeshd-tn a090507
Minor fixes
shreeshd-tn 828b8ef
Slight changes
shreeshd-tn 8e0bec7
Slight fixes
shreeshd-tn ab3a31b
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] c05ac5c
Jenkins date change
shreeshd-tn e295a78
Missed in merge
shreeshd-tn 46e7908
Sparrowhawk punctuation fix
shreeshd-tn de26996
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] c23bf3a
Review changes + sparrowhawk fix
shreeshd-tn dd01128
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] bec1466
Removed unecessary changes
shreeshd-tn 53315e3
Reduced context to improve compilation time
shreeshd-tn c29bb09
Combined approaches
shreeshd-tn 6c4d2ce
Review changes
shreeshd-tn 379d994
Almost done
shreeshd-tn 6cf74e8
Ready to merge
shreeshd-tn ec57c44
Merge branch 'combined' into staging_hi_tn
shreeshd-tn 6f18486
Corrected data files
shreeshd-tn 567c0e7
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] b1dc87a
Jenkins date change
shreeshd-tn dce801a
Variable name change for clarity
shreeshd-tn File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
13 changes: 13 additions & 0 deletions
13
nemo_text_processing/text_normalization/hi/data/address/__init__.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| # Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved. | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
36 changes: 36 additions & 0 deletions
36
nemo_text_processing/text_normalization/hi/data/address/cities.tsv
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| अमरावती | ||
| ईटानगर | ||
| दिसपुर | ||
| पटना | ||
| रायपुर | ||
| पणजी | ||
| गांधीनगर | ||
| चंडीगढ़ | ||
| शिमला | ||
| रांची | ||
| बेंगलुरु | ||
| तिरुवनंतपुरम | ||
| भोपाल | ||
| मुंबई | ||
| इम्फाल | ||
| शिलांग | ||
| आइजोल | ||
| कोहिमा | ||
| भुवनेश्वर | ||
| जयपुर | ||
| गंगटोक | ||
| चेन्नई | ||
| हैदराबाद | ||
| अगरतला | ||
| लखनऊ | ||
| देहरादून | ||
| कोलकाता | ||
| पोर्ट ब्लेयर | ||
| दमन | ||
| नई दिल्ली | ||
| श्रीनगर | ||
| जम्मू | ||
| लेह | ||
| कारगिल | ||
| कवरत्ती | ||
| पुडुचेरी |
49 changes: 49 additions & 0 deletions
49
nemo_text_processing/text_normalization/hi/data/address/context.tsv
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| हाउस | ||
| प्लॉट | ||
| बूथ | ||
| अपार्टमेंट | ||
| फ्लैट | ||
| यूनिट | ||
| टावर | ||
| कॉम्प्लेक्स | ||
| मंजिल | ||
| फ्लोर | ||
| ब्लॉक | ||
| सेक्टर | ||
| फेज | ||
| रोड | ||
| सड़क | ||
| मार्ग | ||
| स्ट्रीट | ||
| गली | ||
| राजमार्ग | ||
| ड्राइव | ||
| डिस्ट्रिक्ट | ||
| बाईपास | ||
| हाइवे | ||
| पार्कवे | ||
| कॉलोनी | ||
| नगर | ||
| पार्क | ||
| एस्टेट | ||
| क्षेत्र | ||
| बोलवार्ड | ||
| मार्केट | ||
| सेंटर | ||
| पिन | ||
| गांव | ||
| पास | ||
| ब्रिगेड | ||
| नियर | ||
| स्क्वेर | ||
| मॉल | ||
| टॉवर | ||
| इंस्टीट्यूट | ||
| पिलर | ||
| मेट्रो | ||
| एवेन्यू | ||
| वेस्ट | ||
| सामने | ||
| पीछे | ||
| वीया | ||
| आर डी |
2 changes: 2 additions & 0 deletions
2
nemo_text_processing/text_normalization/hi/data/address/en_to_hi_mapping.tsv
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| street स्ट्रीट | ||
| southern सदर्न |
26 changes: 26 additions & 0 deletions
26
nemo_text_processing/text_normalization/hi/data/address/letters.tsv
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| A ए | ||
| B बी | ||
| C सी | ||
| D डी | ||
| E ई | ||
| F एफ | ||
| G जी | ||
| H एच | ||
| I आई | ||
| J जे | ||
| K के | ||
| L एल | ||
| M एम | ||
| N एन | ||
| O ओ | ||
| P पी | ||
| Q क्यू | ||
| R आर | ||
| S एस | ||
| T टी | ||
| U यू | ||
| V वी | ||
| W डब्ल्यू | ||
| X एक्स | ||
| Y वाई | ||
| Z ज़ेड |
2 changes: 2 additions & 0 deletions
2
nemo_text_processing/text_normalization/hi/data/address/special_characters.tsv
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| - हाइफ़न | ||
| / बटा |
36 changes: 36 additions & 0 deletions
36
nemo_text_processing/text_normalization/hi/data/address/states.tsv
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| आंध्र प्रदेश | ||
| अरुणाचल प्रदेश | ||
| असम | ||
| बिहार | ||
| छत्तीसगढ़ | ||
| गोवा | ||
| गुजरात | ||
| हरियाणा | ||
| हिमाचल प्रदेश | ||
| झारखंड | ||
| कर्नाटक | ||
| केरल | ||
| मध्य प्रदेश | ||
| महाराष्ट्र | ||
| मणिपुर | ||
| मेघालय | ||
| मिज़ोरम | ||
| नागालैंड | ||
| ओडिशा | ||
| पंजाब | ||
| राजस्थान | ||
| सिक्किम | ||
| तमिलनाडु | ||
| तेलंगाना | ||
| त्रिपुरा | ||
| उत्तर प्रदेश | ||
| उत्तराखंड | ||
| पश्चिम बंगाल | ||
| अंडमान और निकोबार द्वीप समूह | ||
| चंडीगढ़ | ||
| दादरा और नगर हवेली और दमन और दीव | ||
| दिल्ली | ||
| जम्मू और कश्मीर | ||
| लद्दाख | ||
| लक्षद्वीप | ||
| पुडुचेरी |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -8,4 +8,3 @@ hp हॉर्सपॉवर | |
| d दिन | ||
| month महीना | ||
| months महीने | ||
|
|
||
10 changes: 10 additions & 0 deletions
10
nemo_text_processing/text_normalization/hi/data/ordinal/en_to_hi_digit.tsv
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| 0 ० | ||
| 1 १ | ||
| 2 २ | ||
| 3 ३ | ||
| 4 ४ | ||
| 5 ५ | ||
| 6 ६ | ||
| 7 ७ | ||
| 8 ८ | ||
| 9 ९ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let's update to 2026