Install Moses Scripts Error

Hi,

I was about to install the moses Scripts following the step-by-step
instruction here :
_________________________________________________
The support scripts used by Moses are "released" by a Makefile which
edits their paths to match your local environment. First, you need to
edit the Makefile definition of two variables:
mkdir -p bin/moses-scripts

###Edit moses/scripts/Makefile
TARGETDIR=/full-path-to-workspace/bin/moses-scripts
BINDIR=/full-path-to-workspace/bin
###

cd moses/scripts/
make release

This will create a folder named
bin/moses-scripts/scripts-YYYYMMDD-HHMM with released versions of all
the scripts. You will call these versions when training/tuning Moses.
Moses scripts also require a SCRIPTS_ROOTDIR environment variable to
be set. The output of make release should indicate this.
export SCRIPTS_ROOTDIR=/full-path-to-workspace/bin/moses-scripts/scripts-YYYYMMDD-HHMM
_________________________________________________

However,   I received an error saying that some files have vanished,
and at least one file "symal" which is going to be used in training
procedure is missing. Does anybody know what happened? Following is
the snapshot of the error.

Re: Install Moses Scripts Error

Jie,

  Since you're doing this on cygwin, gcc makes all the binary files
into .exe files.  The released-files list has the wrong file names in
it for that, so it can't find them at rsync time.

  Try using the MakefileWIN32 makefile if it's still there.  It points
to the released-filesWIN32, in which the gcc-compiled files are
indicated with the appropriate suffix.

  I hope those two WIN32 files are still there.  I'm looking at a
somewhat older release.

-John

On 10/1/07, Jie Wu <jie82.wu@...> wrote:
> Hi,
>
> I was about to install the moses Scripts following the step-by-step
> instruction here :
> _________________________________________________
> The support scripts used by Moses are "released" by a Makefile which
> edits their paths to match your local environment. First, you need to
> edit the Makefile definition of two variables:
> mkdir -p bin/moses-scripts
>
> ###Edit moses/scripts/Makefile
> TARGETDIR=/full-path-to-workspace/bin/moses-scripts
> BINDIR=/full-path-to-workspace/bin
> ###

Re: Install Moses Scripts Error

Dear Jie Wu,

I don't have a definite answer, I can just explain a bit of the log:

Jie Wu wrote:
> C:\JIE\MT\Moses\MOSES\src\scripts>make -B Makefile release
> make: Nothing to be done for `Makefile'.

The command you issued is suspicious: -B forces make to rebuild all targets. I 
wonder if our Makefiles might dislike that. Specifying 'Makefile' (after -B or 
anywhere else) means that you're asking make to make the Makefile. I assume you 
rather wanted to say 'make -f Makefile release', that is to make using the 
specified Makefile.

> # Compile the parts
> make all
> make[1]: Entering directory `/cygdrive/c/JIE/MT/Moses/MOSES/src/scripts'
> pwd=`pwd`; \
>         for subdir in cmert-0.5 phrase-extract symal mbr; do \
>           cd training/$subdir/ && make || exit 1; \
>           echo "### Compiler $subdir"; \
>           cd $pwd; \
>         done
> make[2]: Entering directory `/cygdrive/c/JIE/MT/Moses/MOSES/src/scripts/training
> /cmert-0.5'
> gcc -O3   -c -o mert.o mert.c
> gcc -O3   -c -o data.o data.c
> gcc -O3   -c -o point.o point.c
> gcc -O3   -c -o score.o score.c
> gcc mert.o data.o point.o score.o -lm  -o mert

GCC and VC compilation

Re: GCC and VC compilation

Re: GCC and VC compilation

Re: GCC and VC compilation

Re: GCC and VC compilation

A Question on Moses / not recognizing compiled in SRILM language model on OSX

Hello moses-support,

   Just for fun I'm using Moses to learn a translation system based
off of 42 translations of Japanese comic books that I've done over
the years.

    I thought that I had completed working through the learning cycle,
but when I try to run moses as a decoder, I ran into this error:

...
Start loading LanguageModel /Users/devans/Documents/workspace/
GMAOParallelDataExtractor/europarl.en.lm : [16.000] seconds
ERROR:Language model type unknown. Probably not compiled into library
ERROR:no LM created. We probably don't have it compiled ...

    I believe that it is compiled into the library though, since I
configured with:

$ ./configure --with-srilm=/usr/local/srilm

   and the make process properly found Ngram.h, and it looks like it
included in the lib directory (LDFLAGS =  -L/usr/local/srilm/lib/macosx)

   Has anyone else run into this problem?

   I compiled on
$ uname -a
Darwin Dhalsim.local 8.10.0 Darwin Kernel Version 8.10.0: Wed May 23  
16:50:59 PDT 2007; root:xnu-792.21.3~1/RELEASE_PPC Power Macintosh  
powerpc PowerBook5,8 Darwin
$ gcc --version
powerpc-apple-darwin8-gcc-4.0.0 (GCC) 4.0.0 (Apple Computer, Inc.  
build 5026)
$ automake --version
automake (GNU automake) 1.9.6

   I haven't used Xcode myself, but I do have it around so if that is  
the recommended way to get things running under OSX perhaps I should  
try that?

   Anyway, I hope I can work this out since I curious to see if I can  
get any sort of reasonable translations out of approximately 48,290  
aligned comic book "bubbles" done by an amateur translator...

   By the way, I used the English portion of the Europarl corpus to  
build the language model since it was the only data I knew of that  
was freely available.  Does anyone know if someone has built a SRILM  
compatible language model off of the google n-gram data, or some  
other sort of data that would be less parlimentary-like and more  
general text-like?

   Thanks in advance,

David K. Evans

Re: A Question on Moses / not recognizing compiled in SRILM language model on OSX

Hi David,
I think your project sounds fun.

Can you send me the line in your moses.ini file that specifies the
language model?  It may be that you have a format error which is
causing the trouble.

As for the training data.  First, with 48000 segments, you may well
have a reasonable basic system.  Although system quality is highly
dependent on language, and the genres being translated, there is a
popular MT corpus (BTEC, consisting of tourism type phrases) that only
has about 40000 sentence pairs.

To train a language model, you'll definitely want to include the
English translations you're using to train the translation model since
these are closest to the kinds of sentences your system will be able
to generate.  In fact, a reasonable starting point would just be to
use your English translations as the basis for the language model.
Beyond that, I'm not familiar with which corpora are freely available,
but perhaps someone else on the list who's looked into this could make
a suggestion.

--Chris

Chris

On 10/5/07, David Kirk Evans <dave@...> wrote:
> Hello moses-support,
>
>    Just for fun I'm using Moses to learn a translation system based
> off of 42 translations of Japanese comic books that I've done over
> the years.
>
>     I thought that I had completed working through the learning cycle,
> but when I try to run moses as a decoder, I ran into this error:
>
> ...
> Start loading LanguageModel /Users/devans/Documents/workspace/
> GMAOParallelDataExtractor/europarl.en.lm : [16.000] seconds
> ERROR:Language model type unknown. Probably not compiled into library
> ERROR:no LM created. We probably don't have it compiled ...
>
>     I believe that it is compiled into the library though, since I
> configured with:
>
> $ ./configure --with-srilm=/usr/local/srilm
>
>    and the make process properly found Ngram.h, and it looks like it
> included in the lib directory (LDFLAGS =  -L/usr/local/srilm/lib/macosx)
>
>    Has anyone else run into this problem?
>
>    I compiled on
> $ uname -a
> Darwin Dhalsim.local 8.10.0 Darwin Kernel Version 8.10.0: Wed May 23
> 16:50:59 PDT 2007; root:xnu-792.21.3~1/RELEASE_PPC Power Macintosh
> powerpc PowerBook5,8 Darwin
> $ gcc --version
> powerpc-apple-darwin8-gcc-4.0.0 (GCC) 4.0.0 (Apple Computer, Inc.
> build 5026)
> $ automake --version
> automake (GNU automake) 1.9.6
>
>    I haven't used Xcode myself, but I do have it around so if that is
> the recommended way to get things running under OSX perhaps I should
> try that?
>
>    Anyway, I hope I can work this out since I curious to see if I can
> get any sort of reasonable translations out of approximately 48,290
> aligned comic book "bubbles" done by an amateur translator...
>
>    By the way, I used the English portion of the Europarl corpus to
> build the language model since it was the only data I knew of that
> was freely available.  Does anyone know if someone has built a SRILM
> compatible language model off of the google n-gram data, or some
> other sort of data that would be less parlimentary-like and more
> general text-like?
>
>    Thanks in advance,
>
> David K. Evans
> _______________________________________________
> Moses-support mailing list
> Moses-support@...
> http://mailman.mit.edu/mailman/listinfo/moses-support
>