The Non-Feasibility of FAHQT: February 2011

Sunday, February 27, 2011

Slide-urile au la bază:

cursul de Traducere Automată de la Universitatea din Edinburgh predat de Philipp Koehn şi Miles Osborne (http://www.inf.ed.ac.uk/teaching/courses/mt/)

cursul de Traducere Automată bazată pe Metode Statistice ţinut la ESSLLI 2010 de Adam Lopez (http://www.cs.jhu.edu/~alopez/esslli2010.html)

Thursday, February 17, 2011

Moses Scripts install

Note: The word alignment stage in the training scripts needs Giza++ in order to run -- see the post on how to install Mgiza++ (a multi-threaded version of Giza++).

in mosesdecoder/scripts
edit Makefile

instead of:
< TARGETDIR?=/home/s0565741/terabyte/bin
< BINDIR?=/home/s0565741/terabyte/bin
replace with:
> TARGETDIR=/home/user/smt
> BINDIR=/home/user/smt/mgiza/bin

in check-dependencies.pl
modify line 28: GIZA++ with mgiza and snt2cooc.out to snt2cooc
to something like:
unless (-x "$bin_dir/mgiza" && -x "$bin_dir/snt2cooc" && -x "$bin_dir/mkcls" ) {

make release

## Don't forget to set your SCRIPTS_ROOTDIR with:
export SCRIPTS_ROOTDIR=/home/user/smt/scripts-20110216-1712
or add it in .bashrc

Install multi-threaded Giza++

checkout latest version of mgizapp

A word alignment tool based on famous GIZA++, extended to support multi-threading, resume training and incremental training.

svn co https://mgizapp.svn.sourceforge.net/svnroot/mgizapp/trunk/mgizapp mgizapp

create a folder for mgiza installation (something like /home/user/smt/mgiza)

configure and install mgiza
./configure --prefix=/home/user/smt/mgiza
make install

Install Moses Decoder

Tested for mosesdecoder from SVN up to rev. 4169
for Ubuntu 10.10 - 11.04 32bit / 64bit

install compression library

zlib is a library implementing the deflate compression method found
in gzip and PKZIP.

sudo apt-get install zlib1g-dev

install boost-thread library

Toolkit for writing C++ programs that execute as multiple,
asynchronous, independent, threads-of-execution.

sudo apt-get install libboost-thread1.42-dev

install xmlrpc for c

XML-RPC is a quick-and-easy way to make procedure calls over the Internet.
It converts the procedure call into an XML document, sends it to a remote
server using HTTP, and gets back the response as XML. This library provides a modular implementation of XML-RPC for C and C++.

sudo apt-get install libxmlrpc-c3-dev

check-out last version of moses
(Recently, Moses development moved to git repository. This installation tutorial was tested for versions up to 4169. Just add -r 4169 to the following command if you have problems compiling it.)
svn co https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk mosesdecoder

build moses (in moses directory)
./regenerate-makefiles.sh
./configure
(opt) for multi-threaded moses run:
./configure --enable-threads
(opt) if you also want the moses XML-RPC server run instead:
./configure --with-xmlrpc-c=/usr/bin/xmlrpc-c-config --enable-threads
after configuration run make:
make -j 4

Prerequisites

for Ubuntu 10.10 i386

install ssh

Ssh (Secure Shell) is a program for logging into a remote machine and for executing commands on a remote machine.

sudo apt-get ssh

install build essentials

This package contains an informational list of packages which are
considered essential for building Debian packages.

sudo apt-get install build-essential

install autoconfiguration

The standard for FSF source packages. This is only useful if you
write your own programs or if you extensively modify other people's
programs.

sudo apt-get install autoconf

install libtool

This is GNU libtool, a generic library support script. Libtool hides
the complexity of generating special library types (such as shared
libraries) behind a consistent interface.

sudo apt-get install libtool

install subversion

Subversion is a version control system. Version control systems allow
many individuals to collaborate on a set of files (typically source code).

sudo apt-get install subversion

Labels