GrmThraxForum < Forum

---+ [[GRM.Thrax][OpenGrm Thrax]] Forum

You need to be a  registered user to participate in the discussions. <br /> 
%IF{"context authenticated" then='%MAKETEXT{"You are logged in as [_1]." args="[[%WIKIUSERNAME%][%SPACEOUT{%WIKINAME%}%]]"}%%BR%%LOGOUT%' else='%LOGIN% %MAKETEXT{"or"}% [[%TWIKIWEB%.TWikiRegistration][%MAKETEXT{"Register"}%]]'}%

%TOC{depth="2"}%

You can start a new discussion here:
<blockquote>
%ICON{help}% You can use the formatting commands describes in TWiki.TextFormattingRules in your comment. <br /> 
%ICON{tip}% If you want to post some code, surround it with =&lt;verbatim&gt;= and =&lt;/verbatim&gt;= tags. <br /> 
%ICON{warning}% Auto-linking of TWiki.WikiWords is now disabled in comments, so you can type !VectorFst and it won't result in a broken link.<br /> 
%ICON{warning}% You now need to use =&lt;br&gt;= to force new lines in your comment (unless inside =verbatim= tags). However, a blank line will automatically create a new paragraph. </blockquote> 
%COMMENT{type="newtopic"}%



---++ Simple tool for running FARs? 
---+++ Main.FilipG - 2015-09-12 - 02:06
<noautolink>
Is there a simple tool available for transforming standard input into standard ouput with a FAR (within OpenFST or Thrax or somewhere else)? I mean something like 

&#60;verbatim&#62;
process-with-far transducer.far &#60; in.txt &#62; out.txt
&#60;/verbatim&#62;

Of course, you&#39;ve got thraxrewrite-tester, but it pollutes the output with &#34;Input/Output string:&#34; and it was not written with efficiency in mind. It wouldn&#39;t be difficult to hack it, but I am wondering whether anything else is available.
</noautolink>


---+++ Main.RichardSproat - 2015-09-12 - 09:29
<noautolink>
Not exactly, since you presumably want to select which FSTs to select from the far. 

I will be releasing a new version of Thrax at some point (when I can get around to it) that will have various changes, and I could add that as a feature to the rewrite-tester. But that may not be soon enough for you.
</noautolink>
%COMMENT{type="mythreadmode"}% 



---++ Undefined error while compiling thrax 
---+++ Main.PrashantGupta - 2015-08-31 - 06:27
<noautolink>
Hi, i am using the thrax(1.0.2) and fst(1.3.4) in gcc version of 4.4.7(I need to make it work on this version). I have built the fst with --enable-far=yes --enable-pdt=yes, but i still get the following errors

//usr/local/lib/libthrax.so: undefined reference to `fst::IsSTList(std::basic_string&#60;char, std::char_traits&#60;char&#62;, std::allocator&#60;char&#62; &#62; const&)&#39;
//usr/local/lib/libthrax.so: undefined reference to `fst::IsSTTable(std::basic_string&#60;char, std::char_traits&#60;char&#62;, std::allocator&#60;char&#62; &#62; const&)&#39;

I use  &#34; g++  -g -O2 -std=c++0x    -o nersuite nersuite-main.o nersuite-nersuite.o nersuite-FExtor.o nersuite-crfsuite2.o ../nersuite_common/libnersuite_common.a -lcrfsuite -llbfgs -lm -ldl -lfst -lthrax -Wl -lboost_unit_test_framework&#34; 
to compile my code. I also tried using the command in ldconfig.

Any help would be appreciated. Thank you
</noautolink>


---+++ Main.RichardSproat - 2015-08-31 - 09:05
<noautolink>
The latest version of Thrax is 1.1.0. Have you tried that version? It requires OpenFst 1.4.0, but that should work with your compiler. I would strongly recommend using that route. You also get more features in Thrax that way.



</noautolink>

---+++ Main.PrateekBaranwal - 2015-09-01 - 16:33
<noautolink>
Richard 1.4.0 does not build with gcc 4.4.7 [Default RedHat Servers running RHEL 6.x]. 

./../include/fst/union.h:140:   instantiated from ‘fst::UnionFst&#60;A&#62;::UnionFst(const fst::Fst&#60;A&#62;&, const fst::Fst&#60;A&#62;&) [with A = fst::ArcTpl&#60;fst::LogWeightTpl&#60;float&#62; &#62;]’
stl_pair.h:90: error: invalid conversion from ‘int’ to ‘const fst::Fst&#60;fst::ArcTpl&#60;fst::LogWeightTpl&#60;float&#62; &#62; &#62;*’
</noautolink>

---+++ Main.RichardSproat - 2015-09-02 - 10:15
<noautolink>
I see. I misread your version number. 

Unfortunately in general it&#39;s a little hard to support older versions, with compilers changing and so forth. For me to reproduce your error would require me to replicate your set of conditions, which would include the out-of-date compiler you are using. 

So I have two suggestions for you.

1) Upgrade your compiler to 4.7. Then you&#39;ll get the benefit of the latest version of OpenFst and the latest version of Thrax.

Or if you cannot do that, then:

2) Read further down on this page where you will find that someone reported what looks like the exact same error about a year and a half ago. See my reply dated 12 Jan 2014 - 13:40. See if my suggestion works.

In fact one of the reasons for forums like this is to archive these sorts of problems, so it&#39;s good to check if someone else has reported the same or a similar issue before posting.
</noautolink>

---+++ Main.KennethRBeesley - 2015-12-02 - 13:25
<noautolink>
On the issue of configuration options, the example above shows  --enable-far=yes --enable-pdt=yes
Is that correct?
An example on http://www.cslu.ogi.edu/~sproatr/Courses/textNorm/tutorial.html shows something different:    --enable-far=true
Should --enable-far    all by itself work?

</noautolink>

---+++ Main.KennethRBeesley - 2015-12-02 - 15:45
<noautolink>
For OpenFst, ./configure --help lists optional features --enable-far and --enable-pdf without any suggestion that it needs or takes =yes or =true or =anything.
</noautolink>
%COMMENT{type="mythreadmode"}% 



---++ Pluralization 
---+++ Main.AlexanderSolovets - 2015-08-15 - 18:13
<noautolink>
Suppose I have a transducer that turns numbers into their spoken representation, e.g. 23 -&#62; twenty-three. Now I want to handle US currency, so $23 becomes &#34;twenty-three dollars&#34;. Obviously for &#34;$1&#34; it is &#34;one dollar&#34;. To implement it in Thrax I might just add the whole string as the alternative path with the lower weight, but as I have many different units (&#34;2m&#34; -&#62; &#34;two meters&#34;, but &#34;1m&#34; meter) I wonder what would be the idiomatic way to implement pluralization? I feel like I should use Features and Paradigms, but I lack good examples of their application. Thank you.
</noautolink>


---+++ Main.RichardSproat - 2015-08-16 - 09:16
<noautolink>
You could use the features functionality, though for English this might be a bit of overkill. For simple cases like English I would just have two StringFiles, one for the singulars and one for the plurals, then define singular_nouns to use the first and plural_nouns the second, then just do the obvious combination with &#34;1&#34; versus all the other numbers.

If you wanted to use the features/paradigms functionality, there&#39;s an example for a more complex case in the distribution. See: src/grammars/paradigms_and_features.grm 

</noautolink>
%COMMENT{type="mythreadmode"}% 



---++ regex lookahead in Thrax 
---+++ Main.BernardR - 2015-06-29 - 10:39
<noautolink>
Is it possible to do lookahead in the Thrax grm files? For example, require at least one digit, one lowercase, and one uppercase as in regex below:

( (?=.*\d) (?=.*[a-z]) (?=.*[A-Z]) .{6,20} )

Thanks
</noautolink>


---+++ Main.RichardSproat - 2015-07-06 - 18:01
<noautolink>
I&#39;m not sure what you are trying to do, but you may just want to use a CDRewrite rule, which allows you to change one regexp to another in the context of two other regexps that are not considered part of the first two regexps.

</noautolink>

---+++ Main.BernardR - 2015-07-08 - 15:26
<noautolink>
So there is no simple way to use regex lookahead? So Thrax does not support this? Would like to create FSA to detect the pattern described. Thanks.
</noautolink>

---+++ Main.RichardSproat - 2015-07-09 - 09:04
<noautolink>
Regex lookahead is not something that is implemented per se. But CDRewrite implements all of the functionality that one uses regexp lookahead in PCRE&#39;s for, as far as I can tell. If you want to detect a regular expression in the context of another regular expression and know that you have detected it, an easy way is to write a CDRewrite rule that inserts some marker after (or before) the first regular expression if it occurs in the context of the second regexp.  This gives you all the functionality that the PCRE lookahead would give you.



</noautolink>
%COMMENT{type="mythreadmode"}% 



---++ User defined symbol tables on PDTs 
---+++ Main.SofiaK - 2014-12-18 - 09:41
<noautolink>
Hi all,

I am new to Thrax and OpenFst and I would appreciate it a lot if you could help me with the following issue. I need to use my own symbol table with a PDT or to be able to extract the symbol table in a non-binary format. So far I was not able to do so as the fst extracted from my far has an empty symbol table.

Let me show you how I worked:

1. I created my grammar that will cover digits one to nine and I got the symbol table I use let&#39;s say with another fst.

*numbers_en_US.grm*

# Numbers simple grammar for en-US.
# Covers numbers 0 to 9

my_symbol_table=SymbolTable[&#39;numbers.txt&#39;];

export PARENS = (&#34;[&#60;s&#62;]&#34; : &#34;[&#60;/s&#62;]&#34;);

space = &#34; &#34; ;

units = Optimize 
[
	(&#34;zero&#34;.my_symbol_table) | 
	(&#34;one&#34;.my_symbol_table) | 
	(&#34;two&#34;.my_symbol_table) | 
	(&#34;three&#34;.my_symbol_table) | 
	(&#34;four&#34;.my_symbol_table) | 
	(&#34;five&#34;.my_symbol_table) | 
	(&#34;six&#34;.my_symbol_table) | 
	(&#34;seven&#34;.my_symbol_table) | 
	(&#34;eight&#34;.my_symbol_table) | 
	(&#34;nine&#34;.my_symbol_table)  
];

export NUMBERS = (&#34;[&#60;s&#62;]&#34; (units space)* units &#34;[&#60;/s&#62;]&#34;)* ;

*numbers.txt*

eight 0

extra1 1

extra2 2

&#60;eps&#62; 3

five 4

four 5

nine 6

one 7

&#60;/s&#62; 8

&#60;s&#62; 9

seven 10

six 11

three 12

two 13

zero 14

2. Then I compiled my grammar, extracted the fst from the far and checked the fst info:

$ *fstinfo NUMBERS*

fst type                                          vector

arc type                                          standard

input symbol table                                *none*

output symbol table                               *none*

# of states                                       12

# of arcs                                         32

initial state                                     11

 ...

3. So as the symbol table is empty, when I test, it is impossible to get rewrites:

$ thraxrewrite-tester --far=numbers_en_US.far  --rules=NUMBERS\$PARENS   *--output_mode=numbers.txt*

Input string: one

*Rewrite failed.*

$ thraxrewrite-tester --far=numbers_en_US.far --rules=NUMBERS\$PARENS

Input string: one

*Rewrite failed.*


So, any ideas on how to use my symbol table? Or even how to get the internal symbol table in a non-binary format?

Thanks,
Sofia

</noautolink>


---+++ Main.RichardSproat - 2014-12-19 - 10:39
<noautolink>

</noautolink>

---+++ Main.RichardSproat - 2014-12-19 - 10:45
<noautolink>
The symbols generated for the PARENS will be in the FST named *StringFstSymbolTable, which you will see if you do a farextract on the far.

But it looks as if you are assuming two symbol tables here, one being your own, the other being the one that will be generated for those extended labels. I think what you want to do is something like this:

export PARENS = (&#34;&#60;s&#62;&#34;.my_symbol_table : &#34;&#60;/s&#62;&#34;.my_symbol_table);

Then you need to run the compiler with the --save_symbols flag. Finally you will need to use the --input_mode and probably the --output_mode flags to thraxrewrite-tester with the argument being your symbol table.

If that still doesn&#39;t work, can you send me (rws@google.com) the complete set of files needed to build your target, and I will have a look.

--R


</noautolink>

---+++ Main.SofiaK - 2014-12-24 - 05:12
<noautolink>
Hi Richard, I followed your advice but the .far I get with my symbol table is completely different from the one without it. Which is expected but &#34;initial state 0&#34; worries me for example. I will send you my set of files to get an idea.
</noautolink>
%COMMENT{type="mythreadmode"}% 



---++ compile error on openSuse 13.1 
---+++ Main.RogerB - 2014-11-19 - 14:38
<noautolink>
Hi,
I downloaded openfst 1.4.1 and opengrm-ngram 1.2.1 but the latter won&#39;t compile on openSuse 13.1.

./configure says &#34;configure: error: fst/extensions/far/far.h header not found&#34;

however i find this file at /home/roger/sphinx/openfst-1.4.1/src/include/fst/extensions/far/far.h

compile&installation of openfst was successfull (as far i can tell yet)

do I need to add this path/header file somewhere?

Thanks
Roger
</noautolink>


---+++ Main.RogerB - 2014-11-19 - 16:19
<noautolink>
oh, i found out openfst must be &#39;built&#39; with ./configure --enable-far=true
</noautolink>

---+++ Main.RichardSproat - 2014-11-20 - 09:01
<noautolink>
Right, glad you found it.

</noautolink>
%COMMENT{type="mythreadmode"}% 



---++ Russian phonetic transcription rules 
---+++ Main.AlexisWilpert - 2014-08-22 - 09:40
<noautolink>
Hi all, nice to meet you!

Let me introduce myself, as I am new here. My name is Alexis and I am a computational linguist and software developer. I was very excited with the discovery of the Thrax framework and after a short investigation I decided this was my thing :-) I immediately started digging into it, but unfortunately I was not able to find &#34;real-world&#34; examples of usage, which would have simplified my task.

However, I just kept going on. I have been working for Yandex and developing a rule-based system for generating Russian phonetic transcriptions (in the context of speech synthesis). My company has been very generous and allowed me to open source the rules I wrote.

Probably I do not even use half of the power of Thrax, but I managed to write a working rule-based system just sticking to the basics :-) I thought this could be useful for someone else (as it would have been for myself at the beginning). That is why I thought I should post here about them. Please, take in account that this was my first try with Thrax and that I probably could have written the rules in a much better way, if I had more knowledge.

In case someone is interested, you will find them here: https://github.com/wilpert/RusPhonetizer/tree/master/grammars

Thrax was a wonderfully powerful and easy to use framework for my work, something I did not experience before. I am utterly thankful to the authors for their amazing achievement. And to Yandex for allowing me to share my work.

Thanks to you all and be happy :-)


Alexis
</noautolink>


---+++ Main.RichardSproat - 2014-11-17 - 09:11
<noautolink>

</noautolink>

---+++ Main.RichardSproat - 2014-11-17 - 09:15
<noautolink>
Hi Alexis:

Glad it has proved useful to you.  Yeah there are various toy examples around, but not much &#34;real world&#34; examples that I know of that are public, at least not yet.

I&#39;ll be happy to take a look sometime at your grammars and send along suggestions if I have any.

Richard Sproat

</noautolink>

---+++ Main.AlexisWilpert - 2014-11-29 - 12:57
<noautolink>
Hi Richard,

yes, it would be great if you would find any time to have a look at my grammars, any feedback would be terribly appreciated!

Thanks again for the software,


Alexis
</noautolink>
%COMMENT{type="mythreadmode"}% 



---++ Error compiling on Ubuntu VM 
---+++ Main.EstherJudd - 2014-06-19 - 13:02
<noautolink>
I am trying to compile Thrax in a Ubuntu VM using VirtualBox. I have gcc 4.8.2 installed and compiled openfst with far and pet enabled and in shared mode. I have 1Gb of RAM dedicated to the VM. If I try ./configure --enable-shared, it fails because I run out of memory. If I try just ./configure and then make, everything seems to compile ok until I get an internal compilation error:

/bin/bash ../../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H   -I./../include    -std=c++0x -MT loader.lo -MD -MP -MF .deps/loader.Tpo -c -o loader.lo `test -f &#39;walker/loader.cc&#39; || echo &#39;./&#39;`walker/loader.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I./../include -std=c++0x -MT loader.lo -MD -MP -MF .deps/loader.Tpo -c walker/loader.cc  -fPIC -DPIC -o .libs/loader.o
g++: internal compiler error: Killed (program cc1plus)

</noautolink>


---+++ Main.RichardSproat - 2014-06-20 - 09:12
<noautolink>
Try commenting out the lines that refer to Log64Arc in src/include/thrax/function.h, viz

function.h:70:extern Registry&#60;Function&#60;fst::Log64Arc&#62;* &#62; kLog64ArcRegistry;
function.h:87:  typedef name&#60;fst::LogArc&#62; Log64Arc ## name; \
function.h:88:  REGISTER_LOGARC_FUNCTION(Log64Arc ## name)

(Obviously be careful in that #define REGISTER_GRM_FUNCTION to leave the continuation &#34;\&#34;s all happy.

The downside is you won&#39;t get log64 arcs. The upside is it should be smaller. The fact that it&#39;s running out of memory in compiling the loader makes me suspect that may be the problem because for each of the different arc types, all of the templated classes have to be expanded. This should reduce the size, therefore. If that still doesn&#39;t work, remove log arcs too. You won&#39;t likely be using them. Indeed, for precisely these sorts of issues I have been thinking of disabling those in future versions.
</noautolink>

---+++ Main.EstherJudd - 2014-06-20 - 12:22
<noautolink>
I did that and also had to comment out similar lines in src/lib/walker/evaluator-specialization.cc (lines 35 and 49-53). 

I also tried taking out LogArc and all it&#39;s mentions in function.h and evaluator-specialization.cc. But I still get an internal compilation error. 
</noautolink>

---+++ Main.LemOmogbai - 2014-11-16 - 11:42
<noautolink>
Did you ever get this to work? I have the same problem compiling Thrax.
</noautolink>
%COMMENT{type="mythreadmode"}% 



---++ utils/utils.cc &#39;close&#39; not declared? 
---+++ Main.StevenBedrick - 2014-02-17 - 18:01
<noautolink>
Hello, Richard et al.-

While compiling Thrax 1.1 (against OpenFST 1.3.4 on an Ubuntu 13.10 system), I&#39;m getting the following compilation error:

&#60;pre&#62;
...
/bin/bash ../../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H   -I./../include    -g -O2 -MT utils.lo -MD -MP -MF .deps/utils.Tpo -c -o utils.lo `test -f &#39;util/utils.cc&#39; || echo &#39;./&#39;`util/utils.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I./../include -g -O2 -MT utils.lo -MD -MP -MF .deps/utils.Tpo -c util/utils.cc  -fPIC -DPIC -o .libs/utils.o
util/utils.cc: In function &#39;bool thrax::Readable(const string&)&#39;:
util/utils.cc:139:13: error: &#39;close&#39; was not declared in this scope
   close(fdes);
             ^
make[3]: *** [utils.lo] Error 1
make[3]: Leaving directory `/home/steven/thrax-1.1.0/src/lib&#39;
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/home/steven/thrax-1.1.0/src&#39;
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/steven/thrax-1.1.0&#39;
make: *** [all] Error 2

&#60;/pre&#62;

Any ideas what might be going on here?
</noautolink>


---+++ Main.StevenBedrick - 2014-02-17 - 18:02
<noautolink>
OK, having wiki formatting trouble. Trying the code snippet again:

&#60;verbatim&#62;
/bin/bash ../../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H   -I./../include    -g -O2 -MT utils.lo -MD -MP -MF .deps/utils.Tpo -c -o utils.lo `test -f &#39;util/utils.cc&#39; || echo &#39;./&#39;`util/utils.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I./../include -g -O2 -MT utils.lo -MD -MP -MF .deps/utils.Tpo -c util/utils.cc  -fPIC -DPIC -o .libs/utils.o
util/utils.cc: In function &#39;bool thrax::Readable(const string&)&#39;:
util/utils.cc:139:13: error: &#39;close&#39; was not declared in this scope
   close(fdes);
             ^
make[3]: *** [utils.lo] Error 1
make[3]: Leaving directory `/home/steven/thrax-1.1.0/src/lib&#39;
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/home/steven/thrax-1.1.0/src&#39;
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/steven/thrax-1.1.0&#39;
make: *** [all] Error 2
&#60;/verbatim&#62;
</noautolink>

---+++ Main.StevenBedrick - 2014-02-17 - 18:03
<noautolink>
Third time&#39;s the charm?
&#60;!-- &#60;pre&#62; --&#62;
/bin/bash ../../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H   -I./../include    -g -O2 -MT utils.lo -MD -MP -MF .deps/utils.Tpo -c -o utils.lo `test -f &#39;util/utils.cc&#39; || echo &#39;./&#39;`util/utils.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I./../include -g -O2 -MT utils.lo -MD -MP -MF .deps/utils.Tpo -c util/utils.cc  -fPIC -DPIC -o .libs/utils.o
util/utils.cc: In function &#39;bool thrax::Readable(const string&)&#39;:
util/utils.cc:139:13: error: &#39;close&#39; was not declared in this scope
   close(fdes);
             ^
make[3]: *** [utils.lo] Error 1
make[3]: Leaving directory `/home/steven/thrax-1.1.0/src/lib&#39;
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/home/steven/thrax-1.1.0/src&#39;
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/steven/thrax-1.1.0&#39;
make: *** [all] Error 2
&#60;!-- &#60;/pre&#62; --&#62;
</noautolink>

---+++ Main.StevenBedrick - 2014-02-17 - 18:04
<noautolink>
OK, this is ridiculous. Click here to see a Gist:

https://gist.github.com/stevenbedrick/809dbe2c921d745fbcc6
</noautolink>

---+++ Main.RichardSproat - 2014-02-18 - 09:07
<noautolink>
I don&#39;t know. I will have to investigate. 
</noautolink>

---+++ Main.RichardSproat - 2014-02-18 - 09:31
<noautolink>
Does explicitly including unistd.h help?
</noautolink>

---+++ Main.StevenBedrick - 2014-02-23 - 23:01
<noautolink>
Yup, adding that #include to util/utils.cc does the trick.
</noautolink>

---+++ Main.RichardSproat - 2014-02-24 - 09:01
<noautolink>

</noautolink>

---+++ Main.RichardSproat - 2014-02-24 - 09:09
<noautolink>
Ok thanks.

So the question is why you aren&#39;t getting that by inheritance. This is the first time I&#39;ve seen this problem and I have no idea where it has suddenly broken.
</noautolink>
%COMMENT{type="mythreadmode"}% 



---++ compilation fails 
---+++ Main.KyleGorman - 05 Nov 2013 - 14:53
<noautolink>
Hi Richard (etc.), using Thrax 1.1.0 (and with OpenFst 1.3.4 already installed), compilation fails while making the file `ast/identifier-node.cc` due to an issue in the `include/thrax/compat/utils.h` header. Here's the error:

/bin/sh ../../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H   -I./../include    -g -O2 -MT identifier-node.lo -MD -MP -MF .deps/identifier-node.Tpo -c -o identifier-node.lo `test -f 'ast/identifier-node.cc' || echo './'`ast/identifier-node.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I./../include -g -O2 -MT identifier-node.lo -MD -MP -MF .deps/identifier-node.Tpo -c ast/identifier-node.cc  -fno-common -DPIC -o .libs/identifier-node.o
In file included from ast/identifier-node.cc:22:
./../include/thrax/compat/utils.h:119:8: error: field has incomplete type
      'char []'
  char buf[];
       ^

I presume this is because buf[] doesn't have a length defined (nor is it initialized with a string), and when I change the line to

char buf[1024];

compilation goes through. (I'm not sure this is a sensible default; I spent no time trying to understand what this code is doing.)

I'd include a patch but it's one line.

Kyle
</noautolink>


---+++ Main.RichardSproat - 05 Nov 2013 - 16:38
<noautolink>
Just remove that line: that variable is not used. Apparently it's a holdover from some earlier implementation, and I just forgot to update it.  I'll fix this in the next release.
</noautolink>
%COMMENT{type="mythreadmode"}% 



---++ TEST 
---+++ Main.RichardSproat - 13 Sep 2013 - 12:16
<noautolink>
This is a test. Please ignore.
</noautolink>

%COMMENT{type="mythreadmode"}% 



---++  Recommended way to obtain FST+symbols for use 
---+++ Main.JosefNovak - 10 Jun 2013 - 09:46
<noautolink>
Hi, 

I am currently using thrax to extend my some features of an alignment tool I wrote for my g2p system.  

The basic idea is that the user can specify some alignment correspondence rules and optional default penalties, and then these can be incorporated into the EM training process.  

At present I have kind of hacked the functionality of the thraxcompiler command tool to read in the grammar, and then return the desired FST+symbol table to the alignment program.  

EDIT: Maybe it makes more sense to just provide a couple of snippets:

*GetFstFromGrammar*
<verbatim>
template <typename Arc>
VectorFst<Arc> GetFstFromGrammar(const string& input_grammar, const string& rules_name) {
  GrmCompilerSpec<Arc> grammar;
  VectorFst<StdArc> rules;
  if ( grammar.ParseFile(input_grammar) && grammar.EvaluateAst() ) {
    const GrmManagerSpec<Arc>* manager = grammar.GetGrmManager();
    FstMap fsts = manager->GetFstMap();
    for( typename FstMap::const_iterator it=fsts.begin();
         it != fsts.end(); ++it ){
      cout << "Echo: " << it->first << endl;
    }
    rules = *fsts[rules_name];
    return rules;
  }

  return rules;
}
</verbatim>

*toy.grm*
<verbatim>
sy = SymbolTable['simple.syms'];

zero  = "0".sy : "zero".sy;
units = ( "these're".sy : ( "these're".sy | "[these]" | "[these]" "are".sy ) );
split = ( "[these]" "are".sy : "these're".sy );
sigma = "<sigma>".sy : "<sigma>".sy;
abc   = ( "a".sy "b c".sy : "a b b".sy );
export RULES = Optimize[ sigma* ( units | zero | abc ) sigma* ];
</verbatim>

Here the 'sigma' is used in combination with a specialized 1-state alignment transducer that relies on RHO and SIGMA matchers.

Is there an alternative or recommended way to do this?  It would be great if I could either specify the symbol table just once at the beginning, or automatically infer/generate the whole symbol table and return it - or even better modify the grammar from my C++ application to simply what the user is responsible for doing.

I went through the FAQ but did not notice any answers to these questions.

Thanks for your time.

UPDATE:
 I solved this by creating some bindings with [[https://code.google.com/p/pybindgen/][pybindgen]] and then writing a generator that interprets a simplified version of the Thrax grammar, then expands it to the versbose version with the extra quotes and symfile suffixes, etc.
</noautolink>


%COMMENT{type="mythreadmode"}% 



---++ Need some help, New to "Thrax" 
---+++ Main.GoudjilKamel - 03 Jan 2013 - 17:29
<noautolink>
compiling under unbuntu LTS 12.04 : got the msg below at linking
libtool: link: g++ -g -O2 -o .libs/thraxcompiler compiler.o  -L/usr/local/lib/fst -lm -ldl -lfst /usr/local/lib/fst/libfstfar.so ../lib/.libs/libthrax.so -Wl,-rpath -Wl,/usr/local/lib/fst -Wl,-rpath -Wl,/usr/local/lib
../lib/.libs/libthrax.so: undefined reference to `fst::IsSTList(std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
../lib/.libs/libthrax.so: undefined reference to `fst::IsSTTable(std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
collect2: ld returned 1 exit status
</noautolink>


---+++ Main.RichardSproat - 29 Aug 2013 - 11:47
<noautolink>
Did you compile the fst library with the far extension?
</noautolink>

---+++ Main.DanXu - 08 Jan 2014 - 02:55
<noautolink>
I also have encountered the same problem with v1.1.0(compile export/batch_test), and compiled thrax with far enable.
</noautolink>

---+++ Main.RichardSproat - 08 Jan 2014 - 09:06
<noautolink>
Yes, but did you also compile the fst library with far enabled?
</noautolink>

---+++ Main.DanXu - 09 Jan 2014 - 09:53
<noautolink>
yes (openfst 1.3.4 compiled with --enable-far and some other enable options ), thrax compiled successfully,but compilation fails while making the file `batch_test.c` (extracted form export.tgz), can you me some advice
</noautolink>

---+++ Main.RichardSproat - 10 Jan 2014 - 09:11
<noautolink>
I'd like to but first I need to understand what is going on. I can't reproduce your error (apparently) and I don't know what batch_test.c is since it's not part of the Thrax distribution. Is this your own code? If so then I need to see EXACTLY what you are doing, including probably your sending me a directory with all of the additional code.

If this *is* part of the Thrax distribution then please tell me where it is because I can't find it (nor do I remember such a file).
</noautolink>

---+++ Main.DanXu - 11 Jan 2014 - 09:18
<noautolink>
thank you for your reply.

in this page:

http://openfst.cs.nyu.edu/twiki/bin/view/Contrib/ThraxContrib,  

you can see

 Projects using the OpenGrm Thrax tools:
 export.tgz: Grammars and software developed as part of a text  normalization class taught at the Center for Spoken Language Understanding, Fall 2011. URL for the course: http://www.cslu.ogi.edu/~sproatr/Courses/TextNorm/

i download "export.tgz" .
there is a file called batch_tester.cc in batch_tester directory(extract from export.tgz)&#12290;
</noautolink>

---+++ Main.RichardSproat - 12 Jan 2014 - 09:08
<noautolink>
Ok that helps. Yes, I did write that, but it wasn't obvious from your query that this is what you were referring to. Please in future give all necessary information when reporting a bug.

In the meantime I will have a look. I do not know off the top of my head what the problem is.
</noautolink>

---+++ Main.RichardSproat - 12 Jan 2014 - 13:40
<noautolink>
Ok it's the usual nonsense about ordering of shared object libraries. If you do things in this order it should work:

 g++ -g -O2 -o batch_tester batch_tester.o  -L/usr/local/lib/fst -lm -ldl -lfst -lthrax   -Wl,--rpath -Wl,/usr/local/lib/fst -Wl,--rpath -Wl,/usr/local/lib/fst -Wl,--rpath -Wl,/usr/local/lib /usr/local/lib/fst/libfstfar.so 

Evidently there is a bug in the configuration of the distribution that was not causing problems before, but is now. I will look into that, but in the meantime, please try linking manually as above.
</noautolink>

---+++ Main.DanXu - 14 Jan 2014 - 03:47
<noautolink>
it's ok using above command you wrote,thanks!
</noautolink>

---+++ Main.RichardSproat - 14 Jan 2014 - 09:01
<noautolink>
Ok good, I'll update the tar file. Not sure why it worked before and not now, but I won't think about that.
</noautolink>
%COMMENT{type="mythreadmode"}% 



---++ Weight semiring 
---+++ Main.LauriLyly - 21 Nov 2012 - 00:34
<noautolink>
So far I find thrax a very neat piece of software but I have two questions...

Can I somehow use probability semiring as weights, because it seems Thrax only allows specifying log and tropical semirings? How about the other ones... Or should I somehow postprocess the generated far file?

Another question: I tried to use "fstdraw" on a far file, but got: ERROR: FstHeader::Read: Bad FST header: example.far

Is this a version mismatch?
</noautolink>


---+++ Main.LauriLyly - 29 Nov 2012 - 07:34
<noautolink>
Sorry, obviously my bad as it's a far and not an fst file :p Still not too familiar. But the weight question still applies ;)
</noautolink>

---+++ Main.RichardSproat - 29 Nov 2012 - 10:07
<noautolink>
Sorry, I missed the earlier comment -- for some reason I didn't get email about it.  

Unfortunately the restriction to Log and Tropical is due to a similar restriction in the fst library: the real semiring does not come predefined. The best suggestion would be to use Tropical and then just do the obvious e^-cost conversion.
</noautolink>
%COMMENT{type="mythreadmode"}% 

Access control:
   * Set ALLOWTOPICCHANGE = Main.AllAuthUsersGroup

-- Main.CyrilAllauzen - 13 Aug 2012
This topic: Forum > WebHome > GrmThraxForum
Topic revision: r54 - 2015-12-02 - KennethRBeesley