varnamproject
Varnam Inscript - failed add data base
Idenified the issue. It failes to add contents witht Atomic Chillu, Zero width joiner and Zero width non joiner. All conents are seems to be in the data base.
That was the only message, like -Failed to add "മലയാളം"- Sorry for the wrong information, My terminal doesnt support Malayalam. From the actual logs.
ജോബ്സ് : Can't process ']'. One or more characters in 'ജോബ്സ്' are not known ടാക്സി : Can't process ''. One or more characters in 'ടാക്സി' are not known നിന്നിൽ : Can't process 'ൽ'. One or more characters in 'നിന്നിൽ' are not known
Fixed in this[1] commit to my local repo. Problem was the joiner ']' and non-joiner was being sent to the 'other' function in varnamc from the scheme file. Fixed by passing the joiner to the 'joiner' function and non_joiner to 'non_joiner' function. A few crude tweaks in learn.c as well. Will make it cleaner once @soorajkenoth confirms that the bug is fixed.
[1]https://github.com/lonesword/libvarnam/commit/3cd8e66718d472b0eaa4c6e7ebd072b8489e4844
I intalled the packages from inscript branch of https://github.com/lonesword/libvarnam.
The issue still exist. I think I need to remove all packages before I reinstall.
<കൊണ്ടാൽ : Can't process ']'. One or more characters in 'കൊണ്ടാല്]' are not known>
It seems the syntax is not correct:- nd] => ല ് zwj => ല്
I had a look at the source. The same issue exists there also.
Sure that you cloned the right branch? Just to be sure - check if the scheme file ml-inscript has two functions 'joiner' and 'non_joiner' at the end. That used to be 'other' and it was causing all the problems.
On Thu, Oct 23, 2014 at 12:22 PM, soorajkenoth [email protected] wrote:
<കൊണ്ടാൽ : Can't process ']'. One or more characters in 'കൊണ്ടാല്]' are not known>
It seems the syntax is not correct:- nd] => ല ് zwj => ല്
I had a look at the source. The same issue exists there also.
— Reply to this email directly or view it on GitHub https://github.com/varnamproject/libvarnam/issues/71#issuecomment-60199309 .
Yes. It is from the new repository. I think the old installation creates the problem or there is some some error in the new file. From the the error file I feel there are some mistake in the syntax. ഒരു ബ്രാക്കറ്റ് അധികം വരുന്നു.
കര്ണ്ണന് ഇപ്പോഴും മുറിഞ്ഞ് തന്നെയാണ് വരുന്നത്
Actually the ']' is the joiner right? That is the part of the input word and not the syntax of the program.
On Fri, Oct 24, 2014 at 10:46 AM, soorajkenoth [email protected] wrote:
Yes. It is from the new repository. I think the old installation creates the problem or there is some some error in the new file. From the the error file I feel there are some mistake in the syntax. ഒരു ബ്രാക്കറ്റ് അധികം വരുന്നു.
— Reply to this email directly or view it on GitHub https://github.com/varnamproject/libvarnam/issues/71#issuecomment-60345555 .
Yes, ] is the joiner, but there is an extra "]" after joiner.
I have attached the screenshot.
ചിലപ്പോ ആ ആറ്റോമിക് ചില്ലിന്റെ replacement ആയിരിക്കും പ്രശ്നമുണ്ടാക്കുന്നത്.
See this commit[1]
I haven't changed much. Simply removed the trailing "]" in language specific sanitations. Tested my changes. ml-inscript can learn from the word corpus (varnamc --learn-from) and the words with chills are being suggested correctly in ibus. However, for some reason, ml-inscript cannot learn as we type. That is, all inputs on Ibus with chill letters in them are discarded (learning fails). Words without chill characters are learned correctly from Ibus.
And I do not know why removing the trailing "]" works (though partially). And yes you were correct, ആറ്റോമിക് ചില്ലിന്റെ replacement തന്നെ ആയിരുന്നു പ്രശ്നം.
@soorajkenoth can you clone the repo once again and try learning the corpus? [1] https://github.com/lonesword/libvarnam/commit/5a127418e4cb1814f361e4ed3314c8cfcb6ff7fc
No. I think the fix happened only on my machine. I need to look into this more.