Difference: PythonExtension (1 vs. 18)

Revision 182016-12-07 - KyleGorman

Line: 1 to 1
 
META TOPICPARENT name="FstExtensions"

OpenFst Python extension

Line: 119 to 119
 >>> f.set_start(s)
>>> n = f.add_state()
>>> f.reserve_arcs(s, 1) # Optional.
Changed:
<
<
>>> f.add_arc(s, 98, 98, fst.Weight.One(f.weight_type), n)
>
>
>>> f.add_arc(s, fst.Arc(98, 98, fst.Weight.One(f.weight_type), n))
 >>> s = n
>>> n = f.add_state()
>>> f.reserve_arcs(s, 1) # Optional.
Changed:
<
<
>>> f.add_arc(s, 97, 97, fst.Weight.One(f.weight_type), n)
>
>
>>> f.add_arc(s, fst.Arc(97, 97, fst.Weight.One(f.weight_type), n))
 >>> s = n
>>> n = f.add_state()
>>> f.reserve_arcs(s, 1) # Optional.
Changed:
<
<
>>> f.add_arc(s, 97, 97, fst.Weight.One(f.weight_type), n)
>
>
>>> f.add_arc(s, fst.Arc(97, 97, fst.Weight.One(f.weight_type), n))
 >>> f.reserve_arcs(n, 1) # Optional.
Changed:
<
<
>>> f.add_arc(n, 97, 97, fst.Weight.One(f.weight_type), n)
>
>
>>> f.add_arc(n, fst.Arc(97, 97, fst.Weight.One(f.weight_type), n))
 >>> f.set_final(n, fst.Weight.One(f.weight_type))
>>> f.verify() # Checks FST's sanity.
True

Revision 172016-02-18 - KyleGorman

Line: 1 to 1
 
META TOPICPARENT name="FstExtensions"

OpenFst Python extension

Line: 219 to 219
 

Worked example

Putting it all together, the following example, based on Mohri et al. 2002, 2008, shows the construction of an ASR recognition transducer from a pronunciation lexicon L, grammar G, a transducer from context-dependent phones to context-independent phones C, and an HMM set H (where we assume that the components are all determinizable and, preferably, in the log semiring).

Changed:
<
<
>>> reader = fst.FarReader("hclg.far")
>
>
>>> reader = fst.FarReader.open("hclg.far")
 >>> LG = fst.determinize(fst.compose(reader["L"], reader["G"]))
>>> CLG = fst.determinize(fst.compose(reader["C"], LG))
>>> HCLG = fst.determinize(fst.compose(reader["H"], CLG))

Revision 162016-02-13 - KyleGorman

Line: 1 to 1
 
META TOPICPARENT name="FstExtensions"
Changed:
<
<

OpenFst Python extension Work in progress, under construction

>
>

OpenFst Python extension

 
Changed:
<
<
This describes the OpenFst 1.5.1 beta API.

This extension exposes nearly all of the OpenFst scripting interface in Python. Like the scripting interface, it supports arbitrary arcs and weights. The extension allows for rapid prototyping and interactive construction of FSTs using the Python REPL.

>
>
This extension exposes the OpenFst scripting API to Python. Like the scripting API, it supports arbitrary arcs and weights. The extension allows for rapid prototyping and interactive construction of FSTs using the Python REPL.
  Note that this extension is unrelated to, and incompatible with, any other third-party Python extensions for OpenFst (e.g., pyfst).

To install this package, either:

  • issue --enable-python during configuration of OpenFst
Changed:
<
<
  • Or, install OpenFst with FAR support (--enable-far) then install the PyPi package openfst using Pip: pip install openfst
>
>
  • Or, install OpenFst with FAR support (--enable-far) then install the PyPi package openfst using Pip: pip install openfst
 NB: >>> indicates the Python interactive prompt; all other typewriter-text lines are print to stdout or stderr.

Module import

Revision 152016-02-09 - KyleGorman

Line: 1 to 1
 
META TOPICPARENT name="FstExtensions"

OpenFst Python extension Work in progress, under construction

Line: 83 to 83
 
output_symbols The output SymbolTable, or None if no output table is set
start The state ID for the start state
weight_type A string indicating the weight type
Changed:
<
<

Mutable FSTs also provide the num_states attribute, which indicates the number of states in the FST.
>
>

Mutable FSTs also provide the num_states attribute, which indicates the number of states in the FST.
  To access FST properties (i.e., cyclicity, weightedness), use the properties method.
>>> print "Is f cyclic?", f.properties(fst.CYCLIC, True) == fst.CYCLIC

Revision 142016-02-08 - KyleGorman

Line: 1 to 1
 
META TOPICPARENT name="FstExtensions"

OpenFst Python extension Work in progress, under construction

Line: 26 to 26
 >>> print >> compiler, "0 1 98 121 1.5"
>>> print >> compiler, "1 2 99 123 2.5"
>>> print >> compiler, "2 3.5"
Changed:
<
<
>>> f = compiler.compile()
>
>
>>> f = compiler.compile() # Creates the FST and flushes the compiler buffer.
>>> f.num_states
3
 >>> f.final(2)

FSTs can be read in from disk using Fst.read, which takes a string argument specifying the input file's location.

Changed:
<
<
>>> f = fst.Fst.read("vector.fst")
>
>
>>> v = fst.Fst.read("vector.fst")
  This class method takes an optional second argument, a string indicating the desired FST type. The FST is converted to this type if it the on-disk FST is not already of the desired type.
>>> c = fst.Fst.read("const.fst")
Line: 85 to 87
  To access FST properties (i.e., cyclicity, weightedness), use the properties method.
>>> print "Is f cyclic?", f.properties(fst.CYCLIC, True) == fst.CYCLIC
Changed:
<
<
True
>
>
Is f cyclic? True
 

FST access and iteration

FST arcs and states can be accessed via the StateIterator, ArcIterator, and MutableArcIterator objects. These are most naturally constructed using the states and arcs methods, as follows.

Line: 105 to 107
  The following function can be used to count the number of arcs and states in an FST.
Changed:
<
<
def num_arcs_and_states(f):
  return sum(1 + f.num_arcs(s) for s in f.states())
>
>
>>> def num_arcs_and_states(f):
...   return sum(1 + f.num_arcs(s) for s in f.states())
 

FST mutation

Revision 132016-02-05 - KyleGorman

Line: 1 to 1
 
META TOPICPARENT name="FstExtensions"

OpenFst Python extension Work in progress, under construction

Line: 6 to 6
  This extension exposes nearly all of the OpenFst scripting interface in Python. Like the scripting interface, it supports arbitrary arcs and weights. The extension allows for rapid prototyping and interactive construction of FSTs using the Python REPL.
Changed:
<
<
Note that this extension is unrelated to, and incompatible with, any other third-party Python extensions for OpenFst (e.g., pyfst). Install should not install this extension in the same Python environment as any third-party extensions.
>
>
Note that this extension is unrelated to, and incompatible with, any other third-party Python extensions for OpenFst (e.g., pyfst).
  To install this package, either:
  • issue --enable-python during configuration of OpenFst
Changed:
<
<
  • Or, install OpenFst normally then install it the PyPi package openfst using Pip: pip install openfst
>
>
  • Or, install OpenFst with FAR support (--enable-far) then install the PyPi package openfst using Pip: pip install openfst
 NB: >>> indicates the Python interactive prompt; all other typewriter-text lines are print to stdout or stderr.

Module import

Line: 38 to 38
 >>> c.fst_type
'const' >>> v = fst.Fst.read("const.fst", fst_type="vector")
Added:
>
>
>>> v.fst_type
 'vector'

This conversion can also be accomplished after instantiation using the convert function.

Line: 84 to 85
  To access FST properties (i.e., cyclicity, weightedness), use the properties method.
>>> print "Is f cyclic?", f.properties(fst.CYCLIC, True) == fst.CYCLIC
Changed:
<
<
True >>> print "Does f have weighted cycles?", ... printf.properties(fst.WEIGHTED_CYCLES, True) == fst.WEIGHTED_CYCLES
False
>
>
True
 

FST access and iteration

FST arcs and states can be accessed via the StateIterator, ArcIterator, and MutableArcIterator objects. These are most naturally constructed using the states and arcs methods, as follows.

Line: 171 to 169
 

FST operations

All FSTs support constructive operations such as composition (compose), intersection (intersect), and reversal (reverse), storing the result in a vector FST.

Changed:
<
<
>>> cv = fst.compose(c, v)
>>> print cv
 <vector Fst at 0x7f1e45890190>
>
>
>>> cv = fst.compose(c, v)
  FSTs also support tests for equality (equal), equivalence (equivalent), stochastic equivalence (randequivalent), and isomorphism (isomorphic).
Changed:
<
<
>>> print fst.isomorphic(c, v)
>
>
>>> fst.isomorphic(c, v)
 True

FSTs which are mutable (e.g., vector FSTs) also support destructive operations such as arc-sorting (arcsort), inversion (invert), projection (project), and union (union). These operations work in place, mutating the instance they are called on and returning nothing. These instance methods are not available for immutable FST types (e.g., const FSTs).

Line: 213 to 209
 FSTs can be written to disk using the write instance method.
>>> f.write("f.fst")
Changed:
<
<
They also can be written into FARs using the FarWriter object. This takes a filename argument, and optionally, specifications of the arc type and FAR type. Once created, an FST can be written to the FarWriter object using dictionary-style assignment.
>
>
They also can be written into FARs using the FarWriter object. Once created, an FST can be written to the FarWriter object using dictionary-style assignment.
 
>>> writer = fst.FarWriter.create("lattice.far")
>>> writer["1best"] = 1best
>>> writer["2best] = 2best

Revision 122016-02-05 - KyleGorman

Line: 1 to 1
 
META TOPICPARENT name="FstExtensions"

OpenFst Python extension Work in progress, under construction

Line: 175 to 175
 >>> print cv
Changed:
<
<
FSTs also support tests for equality (equal), equivalence (equivalent), [[RandEquivalentDoc][stochastic equivalence] (randequivalent), and isomorphism (isomorphic).
>
>
FSTs also support tests for equality (equal), equivalence (equivalent), stochastic equivalence (randequivalent), and isomorphism (isomorphic).
 
>>> print fst.isomorphic(c, v)
True

Revision 112016-02-05 - KyleGorman

Line: 1 to 1
 
META TOPICPARENT name="FstExtensions"
Changed:
<
<

Python Extensions to OpenFst

>
>

OpenFst Python extension Work in progress, under construction

 
Changed:
<
<
This extension exposes nearly all of the FST script-level interface in Python. Like the script-level interface, it supports arbitrary arcs and weights.
>
>
This describes the OpenFst 1.5.1 beta API.
 
Changed:
<
<
Note that this extension is unrelated to, and incompatible with, any other third-party Python extensions for OpenFst (e.g., pyfst). Install should not install this extension in the same Python environment as any third-party extensions.
>
>
This extension exposes nearly all of the OpenFst scripting interface in Python. Like the scripting interface, it supports arbitrary arcs and weights. The extension allows for rapid prototyping and interactive construction of FSTs using the Python REPL.
 
Changed:
<
<
To install this package, issue --enable-python= during configuration. Or, you can install it from PyPi using tools like pip or easy_install.
>
>
Note that this extension is unrelated to, and incompatible with, any other third-party Python extensions for OpenFst (e.g., pyfst). Install should not install this extension in the same Python environment as any third-party extensions.
 
Changed:
<
<
Users may use this extension to rapidly prototype or construct FSTs interactively using the Python REPL. Note that >>> indicates the Python interactive prompt; all other typewriter-text lines are print to stdout or stderr.
>
>
To install this package, either:
  • issue --enable-python during configuration of OpenFst
  • Or, install OpenFst normally then install it the PyPi package openfst using Pip: pip install openfst
NB: >>> indicates the Python interactive prompt; all other typewriter-text lines are print to stdout or stderr.

Module import

The Python module itself is called pywrapfst but in this tutorial, we will alias it to fst.

>>> import pywrapfst as fst

FST construction

FSTs can be compiled from arc-lists in the same format used by the fstcompile binary.

>>> compiler = fst.Compiler()
>>> print >> compiler, "0 1 97 120 .5"
>>> print >> compiler, "0 1 98 121 1.5"
>>> print >> compiler, "1 2 99 123 2.5"
>>> print >> compiler, "2 3.5"
>>> f = compiler.compile()
>>> f.final(2)
<tropical Weight 3.5 at 0x1215ed0>

FSTs can be read in from disk using Fst.read, which takes a string argument specifying the input file's location.

>>> f = fst.Fst.read("vector.fst")

This class method takes an optional second argument, a string indicating the desired FST type. The FST is converted to this type if it the on-disk FST is not already of the desired type.

>>> c = fst.Fst.read("const.fst")
>>> c.fst_type
'const'
>>> v = fst.Fst.read("const.fst", fst_type="vector")
'vector'

This conversion can also be accomplished after instantiation using the convert function.

>>> v = fst.convert(c, fst_type="vector")
>>> v.fst_type
'vector'

Note that this conversion to the vector FST type is mandatory if one wishes to perform mutation operations on an const FST.

FSTs can be read in from FST Archives (FARs) using the FarReader object.

>>> reader = fst.FarReader.open("lattice.far")

Each FST stored within a FAR has a unique string ID which can be used to extract it from the reader object.

>>> f = reader["1best"]

Or, all FSTs stored within a FAR may be accessed via iteration over the reader object.

>>> for (name, f) in reader:
...     print name, f.num_states
('1best', 23)
('2best', 27)
('3best', 27)
...

Finally, an empty mutable vector FST can be created using Fst.

>>> f = fst.Fst()

By default, the resulting FST uses standard (tropical-weight) arcs, but users can specify other arc types (e.g., log) via an optional argument.

>>> f.arc_type
'standard'
>>> g = fst.Fst("log")
>>> g.arc_type
'log'

FST object attributes and properties

All FSTs have the following read-only attributes ("properties" in Python jargon):

arc_type A string indicating the arc type
input_symbols The input SymbolTable, or None if no input table is set
fst_type A string indicating the FST (container) type
output_symbols The output SymbolTable, or None if no output table is set
start The state ID for the start state
weight_type A string indicating the weight type

Mutable FSTs also provide the num_states attribute, which indicates the number of states in the FST.
 
Changed:
<
<
The Python module itself is called fst.

>>> import fst

FSTs are read in from disk using fst.Fst, which takes a filename string as its a required first argument.

>>> f = fst.Fst("vector1.fst")
>
>
To access FST properties (i.e., cyclicity, weightedness), use the properties method.
>>> print "Is f cyclic?", f.properties(fst.CYCLIC, True) == fst.CYCLIC
True
>>> print "Does f have weighted cycles?", \
... printf.properties(fst.WEIGHTED_CYCLES, True) == fst.WEIGHTED_CYCLES
False

FST access and iteration

FST arcs and states can be accessed via the StateIterator, ArcIterator, and MutableArcIterator objects. These are most naturally constructed using the states and arcs methods, as follows.

>>> for state in f.states():
...     for arc in f.arcs(state):
...         print state, arc.ilabel, arc.olabel, arc.weight, arc.nextstate
0 97 120 1.5 1
0 98 121 2.5 1
1 99 123 2.5 2

The final weight of a state can be accessed using the final instance method.

>>> for state in f.states():
...     print state, f.final(state)
0 Infinity
1 Infinity
2 3.5

The following function can be used to count the number of arcs and states in an FST.

def num_arcs_and_states(f):
  return sum(1 + f.num_arcs(s) for s in f.states())

FST mutation

Mutable FSTs can be modified by adding states (add_state), adding arcs leaving existing states (add_arc), marking a existing state as the start state (set_start), or giving a non-infinite final weight to an existing state (set_final). Optionally, the user can reserve states before adding them using the reserve_states instance method, and reserve arcs leaving an existing state using the reserve_arcs method. The following snippet creates an acceptor which, when its arc labels are interpreted as bytes, accepts the well-known "sheep language" /baa+/.

>>> f = fst.Fst()
>>> f.reserve_states(3)  # Optional.
>>> s = f.add_state()
>>> f.set_start(s)
>>> n = f.add_state()
>>> f.reserve_arcs(s, 1)  # Optional.
>>> f.add_arc(s, 98, 98, fst.Weight.One(f.weight_type), n)
>>> s = n
>>> n = f.add_state()
>>> f.reserve_arcs(s, 1)  # Optional.
>>> f.add_arc(s, 97, 97, fst.Weight.One(f.weight_type), n)
>>> s = n
>>> n = f.add_state()
>>> f.reserve_arcs(s, 1)  # Optional.
>>> f.add_arc(s, 97, 97, fst.Weight.One(f.weight_type), n)
>>> f.reserve_arcs(n, 1)  # Optional.
>>> f.add_arc(n, 97, 97, fst.Weight.One(f.weight_type), n)
>>> f.set_final(n, fst.Weight.One(f.weight_type))
>>> f.verify()  # Checks FST's sanity.
True
 >>> print f
Changed:
<
<
>
>
0 1 98 98 1 2 97 97 2 3 97 97 3 3 97 97 3
 
Changed:
<
<
fst.Fst takes an optional second argument, a string indicating the desired FST type. The FST is converted to this type if it is not already of the appropriate type.

>>> c = fst.Fst("const1.fst")
>>> print c
<const Fst at 0x7f1e49aefbe8>
>>> v = fst.Fst("const1.fst", fst_type="vector")
>>> print v
<vector Fst at 0x7f1e45890f10>
>
>
While it is possible to add arcs whose destination state has not yet been added, any other references to states not yet created (by add_state) is forbidden and will raise an FstIndexError.
 
Changed:
<
<
This conversion can also be accomplished after instantiation using fst.convert.
>
>
Existing arcs and states can also be deleted using delete_states, and arcs leaving an existing state can be deleted using delete_arcs. For example, the following function can be used to remove all arcs and states from an FST.
 
Changed:
<
<
>>> v = fst.convert(c, fst_type="vector")
>>> print v
<vector Fst at 0x7f1e45890290>
>
>
>>> def clear(f):
...     for state in f.states():
...         f.delete_arcs(state)
...     f.delete_states()
 
Changed:
<
<
All FSTs support constructive operations such as composition (fst.compose), intersection (fst.intersect), and reversal (fst.reverse), and the result is stored in a vector FST.
>
>

FST visualization

 
Changed:
<
<
>>> cv = fst.compose(c, v)
>>> print cv
 <vector Fst at 0x7f1e45890190>

All FSTs also support tests for equality (fst.equal), equivalence (fst.equivalent), and isomorphism (fst.isomorphic).

>>> print fst.isomorphic(c, v)
True

Finally, all FSTs can be written to disk using their write instance method.

>
>
The instance method text returns a string representing the FST as an arc-list using the same format and options as the fstprint binary. If f is an FST, then print f is an alias for print f.text().
 
Changed:
<
<
>>> c.write("const-copy.fst")
>
>
>>> print f
0       1       98      98
1       2       97      97
2       3       97      97
3       3       97      97
3
 
Changed:
<
<
FSTs which are mutable (e.g., vector FSTs) also support destructive operations such as arc-sorting, inversion, and projection. These operations work in place, mutating the instance they are called on, and return nothing. These instance methods are not available for immutable FST types (e.g., const FSTs), and only become available once the FST has been converted to a mutable FST type.
>
>
FSTs can also be written to a GraphViz file using the draw instance method.
 
Changed:
<
<
>>> v.arcsort(sort_type="olabel")
>>> v.invert()
>>> v.project()
>
>
>>> f.draw("f.gv")
 
Changed:
<
<
A few operations (e.g., weight-pushing) are available in both constructive and destructive forms, albeit with slightly different options.
>
>

FST operations

 
Changed:
<
<
FST properties can be inspected using the properties instance method and module-level "CONSTANT_CASE" constants.
>
>
All FSTs support constructive operations such as composition (compose), intersection (intersect), and reversal (reverse), storing the result in a vector FST.
>>> cv = fst.compose(c, v)
>>> print cv
 <vector Fst at 0x7f1e45890190>
 
Changed:
<
<
>>> print bool(c.properties(fst.ACCEPTOR))
False
>>> print bool(v.properties(fst.MUTABLE))
True
>
>
FSTs also support tests for equality (equal), equivalence (equivalent), [[RandEquivalentDoc][stochastic equivalence] (randequivalent), and isomorphism (isomorphic).
>>> print fst.isomorphic(c, v)
True
 
Changed:
<
<
The FstError exception signals an error during FST operations.
>
>
FSTs which are mutable (e.g., vector FSTs) also support destructive operations such as arc-sorting (arcsort), inversion (invert), projection (project), and union (union). These operations work in place, mutating the instance they are called on and returning nothing. These instance methods are not available for immutable FST types (e.g., const FSTs).
>>> v.arcsort(sort_type="olabel")
>>> v.invert()
>>> v.project()
 
Changed:
<
<
>>> fst.intersect(c, v)
Error: IntersectFst: input FSTs are not acceptors
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pywrapfst.pyx", line 1481, in pywrapfst.intersect (pywrapfst.cc:10151)
  File "pywrapfst.pyx", line 1128, in pywrapfst._init_MutableFst (pywrapfst.cc:7154)
pywrapfst.FstError: Operation failed
>
>
A few operations (e.g., weight-pushing, epsilon-removal) are available in both constructive and destructive forms, albeit with slightly different options.
  To read documentation on individual FST operations, use Python's built-in help function.
Changed:
<
<
>>> help(fst.equal)
>
>
>>> help(fst.equal)
 Help on built-in function equal in module pywrapfst:

equal(...)

Added:
>
>
equal(ifst1, ifst2, delta=fst.kDelta)
  Are two FSTs equal?

Changed:
<
<
This function tests whether two FSTs have the same states with the same
>
>
This function tests whether two FSTS have the same states with the same
  numbering and the same transitions with the same labels and weights in the same order.

Args:

Changed:
<
<
ifst1
The first input Fst.
ifst2
The second input Fst.
>
>
ifst1
The first input FST.
ifst2
The second input FST.
 
delta
Comparison/quantization delta.

Returns:

Changed:
<
<
True if the two transducers satisfy the above condition, else False.

See also: `equivalent`, `isomorphic`, `randequivalent`.

>
>
True if the two FSTs satisfy the above conditions, otherwise False.
 
Changed:
<
<
>>> help(v.project)
Help on built-in function project:
>
>
See also: `equivalent`, `isomorphic`, `randequivalent`.

FST output

 
Changed:
<
<
project(...) Converts the FST to an acceptor using input or output labels.
>
>
FSTs can be written to disk using the write instance method.
>>> f.write("f.fst")
 
Changed:
<
<
This operation destructively projects an FST onto its domain or range by either copying each arc's input label to its output label (the default) or vice versa.

Args:

project_output
Should the output labels be projected?

See also: `decode`, `encode`, `relabel`.

Putting it all together, the following example, based on Mohri et al. 2002, 2008, shows the construction of an ASR recognition transducer from a pronunciation lexicon L, grammar G, a transducer from context-dependent phones to context-independent phones C, and an HMM set H (where we assume that the components are all determinizable and, preferably, in the log semiring).

>>> L = fst.Fst("L.fst")
>>> G = fst.Fst("G.fst")
>>> C = fst.Fst("C.fst")
>>> H = fst.Fst("H.fst")
>>> LG = fst.determinize(fst.compose(L, G))
>>> CLG = fst.determinize(fst.compose(C, LG))
>>> HCLG = fst.determinize(fst.compose(H, CLG))
>
>
They also can be written into FARs using the FarWriter object. This takes a filename argument, and optionally, specifications of the arc type and FAR type. Once created, an FST can be written to the FarWriter object using dictionary-style assignment.
>>> writer = fst.FarWriter.create("lattice.far")
>>> writer["1best"] = 1best
>>> writer["2best] = 2best

Note that the FAR itself is not guaranteed to be flushed to disk until the FarWriter is garbage-collected. Under normal circumstances, calling del on the FarWriter variable will decrement the reference count to zero and trigger garbage collection on the next cycle.

>>> del writer

Worked example

Putting it all together, the following example, based on Mohri et al. 2002, 2008, shows the construction of an ASR recognition transducer from a pronunciation lexicon L, grammar G, a transducer from context-dependent phones to context-independent phones C, and an HMM set H (where we assume that the components are all determinizable and, preferably, in the log semiring).

>>> reader = fst.FarReader("hclg.far")
>>> LG = fst.determinize(fst.compose(reader["L"], reader["G"]))
>>> CLG = fst.determinize(fst.compose(reader["C"], LG))
>>> HCLG = fst.determinize(fst.compose(reader["H"], CLG))
 >>> HCLG.minimize()
Changed:
<
<
>>> HCLG.write("hclg.fst")
>
>
>>> HCLG.write("hclg.fst")

Revision 102015-10-23 - KyleGorman

Line: 1 to 1
 
META TOPICPARENT name="FstExtensions"
Changed:
<
<

Python Extensions to OpenFst Work in progress, under construction

>
>

Python Extensions to OpenFst

  This extension exposes nearly all of the FST script-level interface in Python. Like the script-level interface, it supports arbitrary arcs and weights.
Added:
>
>
Note that this extension is unrelated to, and incompatible with, any other third-party Python extensions for OpenFst (e.g., pyfst). Install should not install this extension in the same Python environment as any third-party extensions.
 To install this package, issue --enable-python= during configuration. Or, you can install it from PyPi using tools like pip or easy_install.

Users may use this extension to rapidly prototype or construct FSTs interactively using the Python REPL. Note that >>> indicates the Python interactive prompt; all other typewriter-text lines are print to stdout or stderr.

Line: 22 to 24
 
Changed:
<
<
fst.Fst takes an optional second argument, a string indicating the desired Fst type. The FST is converted to this type if it is not already of the appropriate type.
>
>
fst.Fst takes an optional second argument, a string indicating the desired FST type. The FST is converted to this type if it is not already of the appropriate type.
 
>>> c = fst.Fst("const1.fst")

Revision 92015-10-22 - KyleGorman

Line: 1 to 1
 
META TOPICPARENT name="FstExtensions"

Python Extensions to OpenFst Work in progress, under construction

This extension exposes nearly all of the FST script-level interface in Python. Like the script-level interface, it supports arbitrary arcs and weights.

Added:
>
>
To install this package, issue --enable-python= during configuration. Or, you can install it from PyPi using tools like pip or easy_install.
 Users may use this extension to rapidly prototype or construct FSTs interactively using the Python REPL. Note that >>> indicates the Python interactive prompt; all other typewriter-text lines are print to stdout or stderr.

The Python module itself is called fst.

Revision 82015-07-07 - KyleGorman

Line: 1 to 1
 
META TOPICPARENT name="FstExtensions"

Python Extensions to OpenFst Work in progress, under construction

Line: 9 to 9
 The Python module itself is called fst.
Changed:
<
<
import fst
>
>
>>> import fst
 

FSTs are read in from disk using fst.Fst, which takes a filename string as its a required first argument.

Revision 72015-07-01 - KyleGorman

Line: 1 to 1
 
META TOPICPARENT name="FstExtensions"

Python Extensions to OpenFst Work in progress, under construction

Line: 12 to 12
 import fst
Changed:
<
<
FSTs are instantiated by using fst.Fst, which takes a filename string as its a required first argument.
>
>
FSTs are read in from disk using fst.Fst, which takes a filename string as its a required first argument.
 
>>> f = fst.Fst("vector1.fst")

Revision 62015-07-01 - MichaelRiley

Line: 1 to 1
 
META TOPICPARENT name="FstExtensions"

Python Extensions to OpenFst Work in progress, under construction

Changed:
<
<
This extension exposes nearly all of the FST script-level interface in Python. Like the script-level interface, it supports arbitrary arcs and weights.
>
>
This extension exposes nearly all of the FST script-level interface in Python. Like the script-level interface, it supports arbitrary arcs and weights.
  Users may use this extension to rapidly prototype or construct FSTs interactively using the Python REPL. Note that >>> indicates the Python interactive prompt; all other typewriter-text lines are print to stdout or stderr.
Line: 71 to 71
  A few operations (e.g., weight-pushing) are available in both constructive and destructive forms, albeit with slightly different options.
Changed:
<
<
FST properties can be inspected using properties instance method and module-level "CONSTANT_CASE" constants.
>
>
FST properties can be inspected using the properties instance method and module-level "CONSTANT_CASE" constants.
 
>>> print bool(c.properties(fst.ACCEPTOR))
Line: 132 to 132
 
Changed:
<
<
Putting it all together, the following example, based on Mohri et al. 2002, 2008, shows the construction of an ASR lattice from a pronunciation lexicon L, grammar G, a transducer from context-dependent phones to context-independent phones C, and an HMM set H:
>
>
Putting it all together, the following example, based on Mohri et al. 2002, 2008, shows the construction of an ASR recognition transducer from a pronunciation lexicon L, grammar G, a transducer from context-dependent phones to context-independent phones C, and an HMM set H (where we assume that the components are all determinizable and, preferably, in the log semiring).
 
>>> L = fst.Fst("L.fst")

Revision 52015-07-01 - KyleGorman

Line: 1 to 1
 
META TOPICPARENT name="FstExtensions"

Python Extensions to OpenFst Work in progress, under construction

Line: 132 to 132
 
Changed:
<
<
Putting it all together, the following example, based on Mohri et al. 2002, 2008, shows the construction of an ASR system given a pronunciation lexicon L, grammar G, a transducer from context-dependent phones to context-independent phones C, and an HMM set H:
>
>
Putting it all together, the following example, based on Mohri et al. 2002, 2008, shows the construction of an ASR lattice from a pronunciation lexicon L, grammar G, a transducer from context-dependent phones to context-independent phones C, and an HMM set H:
 
>>> L = fst.Fst("L.fst")

Revision 42015-06-26 - KyleGorman

Line: 1 to 1
 
META TOPICPARENT name="FstExtensions"

Python Extensions to OpenFst Work in progress, under construction

Changed:
<
<
This extension exposes nearly all of the FST script-level interface in Python. The current target is Python 2.7, though the module should be trivially portable to Python 3.
>
>
This extension exposes nearly all of the FST script-level interface in Python. Like the script-level interface, it supports arbitrary arcs and weights.
 
Changed:
<
<
Users may use this extension to rapidly prototype or construct FSTs interactively using the Python REPL.
>
>
Users may use this extension to rapidly prototype or construct FSTs interactively using the Python REPL. Note that >>> indicates the Python interactive prompt; all other typewriter-text lines are print to stdout or stderr.
 
Changed:
<
<
To enable it, provide the --enable-python flag to configure, then build and install as normal. This will install a Python module pywrapfst to the site-packages (or dist-packages, on Debian/Ubuntu and their derivatives) below the requested --prefix; by default, this is /usr/local/lib/python2.7/{site,dist}-packages/. Users who have installed Python in their home directory may install it by providing --prefix=$HOME flag to configure. In the worst case, users can manually build the shared object and then place it in Python's path.

The module is called pywrapfst but users can alias it; we will assume the fst alias is used in the following examples.

>
>
The Python module itself is called fst.
 
Changed:
<
<
import pywrapfst as fst
>
>
import fst
 
Deleted:
<
<
Note also that >>> indicates the Python interactive prompt; all other typewriter-text lines are print to stdout or stderr.
 FSTs are instantiated by using fst.Fst, which takes a filename string as its a required first argument.

Revision 32015-06-26 - KyleGorman

Line: 1 to 1
 
META TOPICPARENT name="FstExtensions"

Python Extensions to OpenFst Work in progress, under construction

Line: 8 to 8
  To enable it, provide the --enable-python flag to configure, then build and install as normal. This will install a Python module pywrapfst to the site-packages (or dist-packages, on Debian/Ubuntu and their derivatives) below the requested --prefix; by default, this is /usr/local/lib/python2.7/{site,dist}-packages/. Users who have installed Python in their home directory may install it by providing --prefix=$HOME flag to configure. In the worst case, users can manually build the shared object and then place it in Python's path.
Changed:
<
<
The module is called pywrapfst but users can alias it; we will assume the fst path is used in the following examples.
>
>
The module is called pywrapfst but users can alias it; we will assume the fst alias is used in the following examples.
 
import pywrapfst as fst
Changed:
<
<
Note also that >>> indicates the Python interactive prompt; all other typewriter-text lines are print to STDOUT or STDERR.
>
>
Note also that >>> indicates the Python interactive prompt; all other typewriter-text lines are print to stdout or stderr.
  FSTs are instantiated by using fst.Fst, which takes a filename string as its a required first argument.
Line: 136 to 136
 
Changed:
<
<
Putting it all together, the following example, based on Mohri et al. 2002, shows the construction of an ASR system given a pronunciation lexicon L, grammar G, a transducer from context-dependent phones to context-independent phones C, and an HMM set H:
>
>
Putting it all together, the following example, based on Mohri et al. 2002, 2008, shows the construction of an ASR system given a pronunciation lexicon L, grammar G, a transducer from context-dependent phones to context-independent phones C, and an HMM set H:
 
>>> L = fst.Fst("L.fst")

Revision 22015-06-26 - MichaelRiley

Line: 1 to 1
 
META TOPICPARENT name="FstExtensions"
Changed:
<
<

Python extensions to OpenFst

>
>

Python Extensions to OpenFst Work in progress, under construction

 
Changed:
<
<
This extension exposes nearly all of the "script" interface in Python. The current target is Python 2.7, though the module should be trivially portable to Python 3.
>
>
This extension exposes nearly all of the FST script-level interface in Python. The current target is Python 2.7, though the module should be trivially portable to Python 3.
 
Changed:
<
<
Users may use this script to rapidly prototype or construct FSTs interactively using the Python REPL.
>
>
Users may use this extension to rapidly prototype or construct FSTs interactively using the Python REPL.
 
Changed:
<
<
To enable this extension, provide the --enable-python flag to configure, then build and install as normal. This will install a Python module pywrapfst to the site-packages (or dist-packages, on Debian/Ubuntu and their derivatives) below the requested --prefix; by default, this is /usr/local/lib/python2.7/{site,dist}-packages/. Users who have installed Python in their home directory may install it by providing --prefix=$HOME flag to configure. In the worst case, users can manually build the shared object and then place it [[Python's path][http://stackoverflow.com/questions/10161568/import-python-module-not-on-path].
>
>
To enable it, provide the --enable-python flag to configure, then build and install as normal. This will install a Python module pywrapfst to the site-packages (or dist-packages, on Debian/Ubuntu and their derivatives) below the requested --prefix; by default, this is /usr/local/lib/python2.7/{site,dist}-packages/. Users who have installed Python in their home directory may install it by providing --prefix=$HOME flag to configure. In the worst case, users can manually build the shared object and then place it in Python's path.
  The module is called pywrapfst but users can alias it; we will assume the fst path is used in the following examples.

Revision 12015-06-25 - KyleGorman

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="FstExtensions"

Python extensions to OpenFst

This extension exposes nearly all of the "script" interface in Python. The current target is Python 2.7, though the module should be trivially portable to Python 3.

Users may use this script to rapidly prototype or construct FSTs interactively using the Python REPL.

To enable this extension, provide the --enable-python flag to configure, then build and install as normal. This will install a Python module pywrapfst to the site-packages (or dist-packages, on Debian/Ubuntu and their derivatives) below the requested --prefix; by default, this is /usr/local/lib/python2.7/{site,dist}-packages/. Users who have installed Python in their home directory may install it by providing --prefix=$HOME flag to configure. In the worst case, users can manually build the shared object and then place it [[Python's path][http://stackoverflow.com/questions/10161568/import-python-module-not-on-path].

The module is called pywrapfst but users can alias it; we will assume the fst path is used in the following examples.

import pywrapfst as fst

Note also that >>> indicates the Python interactive prompt; all other typewriter-text lines are print to STDOUT or STDERR.

FSTs are instantiated by using fst.Fst, which takes a filename string as its a required first argument.

>>> f = fst.Fst("vector1.fst")
>>> print f
<vector Fst at 0x7f1e45890e70>

fst.Fst takes an optional second argument, a string indicating the desired Fst type. The FST is converted to this type if it is not already of the appropriate type.

>>> c = fst.Fst("const1.fst")
>>> print c
<const Fst at 0x7f1e49aefbe8>
>>> v = fst.Fst("const1.fst", fst_type="vector")
>>> print v
<vector Fst at 0x7f1e45890f10>

This conversion can also be accomplished after instantiation using fst.convert.

>>> v = fst.convert(c, fst_type="vector")
>>> print v
<vector Fst at 0x7f1e45890290>

All FSTs support constructive operations such as composition (fst.compose), intersection (fst.intersect), and reversal (fst.reverse), and the result is stored in a vector FST.

>>> cv = fst.compose(c, v)
>>> print cv
 <vector Fst at 0x7f1e45890190>

All FSTs also support tests for equality (fst.equal), equivalence (fst.equivalent), and isomorphism (fst.isomorphic).

>>> print fst.isomorphic(c, v)
True

Finally, all FSTs can be written to disk using their write instance method.

>>> c.write("const-copy.fst")

FSTs which are mutable (e.g., vector FSTs) also support destructive operations such as arc-sorting, inversion, and projection. These operations work in place, mutating the instance they are called on, and return nothing. These instance methods are not available for immutable FST types (e.g., const FSTs), and only become available once the FST has been converted to a mutable FST type.

>>> v.arcsort(sort_type="olabel")
>>> v.invert()
>>> v.project()

A few operations (e.g., weight-pushing) are available in both constructive and destructive forms, albeit with slightly different options.

FST properties can be inspected using properties instance method and module-level "CONSTANT_CASE" constants.

>>> print bool(c.properties(fst.ACCEPTOR))
False
>>> print bool(v.properties(fst.MUTABLE))
True

The FstError exception signals an error during FST operations.

>>> fst.intersect(c, v)
Error: IntersectFst: input FSTs are not acceptors
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pywrapfst.pyx", line 1481, in pywrapfst.intersect (pywrapfst.cc:10151)
  File "pywrapfst.pyx", line 1128, in pywrapfst._init_MutableFst (pywrapfst.cc:7154)
pywrapfst.FstError: Operation failed

To read documentation on individual FST operations, use Python's built-in help function.

>>> help(fst.equal)
Help on built-in function equal in module pywrapfst:

equal(...)
    Are two FSTs equal?
    
    This function tests whether two FSTs have the same states with the same
    numbering and the same transitions with the same labels and weights in the
    same order.
    
    Args:
      ifst1: The first input Fst.
      ifst2: The second input Fst.
      delta: Comparison/quantization delta.
    
    Returns:
      True if the two transducers satisfy the above condition, else False.
    
    See also: `equivalent`, `isomorphic`, `randequivalent`.

>>> help(v.project)
Help on built-in function project:

project(...)
    Converts the FST to an acceptor using input or output labels.
    
    This operation destructively projects an FST onto its domain or range by
    either copying each arc's input label to its output label (the default) or
    vice versa.
    
    Args:
      project_output: Should the output labels be projected?
    
    See also: `decode`, `encode`, `relabel`.

Putting it all together, the following example, based on Mohri et al. 2002, shows the construction of an ASR system given a pronunciation lexicon L, grammar G, a transducer from context-dependent phones to context-independent phones C, and an HMM set H:

>>> L = fst.Fst("L.fst")
>>> G = fst.Fst("G.fst")
>>> C = fst.Fst("C.fst")
>>> H = fst.Fst("H.fst")
>>> LG = fst.determinize(fst.compose(L, G))
>>> CLG = fst.determinize(fst.compose(C, LG))
>>> HCLG = fst.determinize(fst.compose(H, CLG))
>>> HCLG.minimize()
>>> HCLG.write("hclg.fst")
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback