Lane Schwartz | 3 Apr 17:57 2007
Picon

Understanding decoder output fields

When moses decodes a sentence, and produces an BEST TRANSLATION, it  
gives a total score, as well as an array of 15 other numbers.

I'm trying to figure out which of those 15 numbers is the target  
language model score for the translation.

Here's what I get when I run the decoder:

Defined parameters (per moses.ini or switch):
         config: /export/scratch/moses.nc.fr-en/working-dir/ 
evaluation/filtered.nc-devtest2007/moses.ini
         distortion-file: 0-0 msd-bidirectional-fe 6 /export/scratch/ 
moses.nc.fr-en/working-dir/evaluation/filtered.nc-devtest2007/ 
reordering-table.msd-bidirectional-fe.0.5.0-0
         distortion-limit: 6
         input-factors: 0
         input-file: /export/scratch/moses.nc.fr-en/working-dir/ 
evaluation/nc-devtest2007.input
         lmodel-file: 0 0 5 /export/scratch/moses.nc.fr-en/working- 
dir/lm/europarl-v3.lm
         mapping: 0 T 0
         ttable-file: 0 0 5 /export/scratch/moses.nc.fr-en/working- 
dir/evaluation/filtered.nc-devtest2007/phrase-table.0-0
         ttable-limit: 20 0
         weight-d: 0.086065 0.110904 -0.015132 0.075996 0.115685  
0.056790 0.024530
         weight-l: 0.107263
         weight-t: 0.019468 0.102550 0.127223 0.020522 0.060418
         weight-w: 0.077454
Start loading distortion table /export/scratch/moses.nc.fr-en/working- 
(Continue reading)

Hieu Hoang | 3 Apr 22:27 2007
Picon
Picon

Re: Understanding decoder output fields

Hi lane,

The scores are listed in the following order:
	1. basic distortion
	2. reordering
	3. language models
	4. phrase tables
	5. word penalty
	6. generation tables
I believe that's the order in which they're loaded, but it may not be
noticeable from the debugging output.

The order can be verified in the function 
	IOStream::OutputNBestList()
In the file
	IOStream.cpp

If u add a 
	-labeled-n-best-list 1

The array of 15 numbers is annotated with some info of what they refer
to.

Hope that helps

Hieu Hoang
www.hoang.co.uk/hieu

-----Original Message-----
From: moses-support-bounces@...
(Continue reading)

Hieu Hoang | 3 Apr 22:54 2007
Picon
Picon

Re: Student - Doubts regarding Statistical Machine Translation - Decoder - Moses

Haven't been knighted by the queen yet. But Sir is a good title for
me...

A1.
Hypos are discarded when the number of hypos in a stack is twice as high
as the beam size.
The hypos with the worst scores are thrown away. The scores are pretty
simple to understand except 1 - the future score. However, this is
critical to making sure good results are obtained.

The scores are described in the pharaoh manual.

A2.
We expand all hypotheses in the previous stack. The order is dependant
on the iterator of the stack, which is dependant on how hypotheses are
compared for recombination. However, u should view the order as
arbitary.

I suppose this could affect the final outcome of the decoder - however,
I think the difference is small.

A3. If you have a trigram LM scoring the target sentence, then the last
2 target words translated is the only thing that can influence the LM
score of future hypos.

For example, 2 hypos with the following target sentence is guaranteed to
have the same incremental LM score going forward:
	This is a
	That is a
Because adding 'house' means both hypos will have an additional trigram
(Continue reading)

Hieu Hoang | 3 Apr 22:57 2007
Picon
Picon

Re: Student - Doubts regarding Statistical Machine Translation - Decoder - Moses

Ps. Please email questions to the 'moses-support' mailing list, rather
than directly to me.

I've got a spam filter which is highly likely to trash your mail if it
not from 'moses-support' and I've never emailed u before

Hieu Hoang
www.hoang.co.uk/hieu

-----Original Message-----
From: Hieu Hoang [mailto:h.hoang@...] 
Sent: 03 April 2007 21:55
To: 'ravi_agarwal@...'
Cc: 'moses-support'
Subject: RE: Student - Doubts regarding Statistical Machine Translation
- Decoder - Moses

Haven't been knighted by the queen yet. But Sir is a good title for
me...

A1.
Hypos are discarded when the number of hypos in a stack is twice as high
as the beam size. The hypos with the worst scores are thrown away. The
scores are pretty simple to understand except 1 - the future score.
However, this is critical to making sure good results are obtained.

The scores are described in the pharaoh manual.

A2.
We expand all hypotheses in the previous stack. The order is dependant
(Continue reading)

Philipp Koehn | 4 Apr 20:30 2007
Picon
Picon

Re: Getting final scores from moses decoder

Hi,

you could also simply produce an n-best list file with
list size one by adding the flag "-n-best-list FILENAME 1"
which then contains the info you are looking for.

-phi

On 3/29/07, Lane Schwartz <schwa717@...> wrote:
> I am running some experiments using the moses decoder, and I would
> like to get the final score for each translation. For example:
>
> BEST TRANSLATION: this is a small house [11111]  [total=-27.78]
> <<-27.09, 0.00, -5.00, -0.69>>
>   Sentence Decoding Time: part(sec)=0 cum(sec)=2
>
> The actual best translation and the total field are what I care about.
>
> I have figured out that I can get this by adding the -v 2 flag to
> moses. However, that adds a lot of extra output that I'm not
> interested in. Is there a way to get the final score for each
> translated sentence without turning on -v 2?
>
> Thanks,
> Lane Schwartz
>
> University of Minnesota
>
>
> _______________________________________________
(Continue reading)

Maria Holmqvist | 5 Apr 11:13 2007
Picon
Picon
Picon

MERT tuning phase time issues

Hello,

I have questions regarding the time it takes to perform MERT tuning. I'm
running Moses on AMD 64 Athlon 4000+ processor with 4 gigs of RAM and a 32
bit dist of Ubuntu. 
Since I had trouble earlier with MERT I reverted to svn 1295 as suggested.

For the current experiment with factored translation the first iteration
of MERT required approx. 6 min/sentence . Iteration 2 requires approx. 30
min!!! /sentence and the result in run2.out is definitely worse than in
run1.out.

What time can usually be expected for MERT tuning? 
Why is iteration 2 so much slower than iteration 1?

Best regards,
Maria

ravi agarwal | 5 Apr 20:03 2007
Picon

Re: Student - Doubts regarding Statistical Machine

Reply of Message 2 - regarding A1.
Sir, I am still confused of what do you mean by worse scores. Kindly give me example particular to the concerned case of 'das ist ein kleines haus' to 'This is a small house'.

e.g. -> In the 4th stack, hypothesis 312 with final score -42.55 (-38.29 -4.26) is discarded, where as hypothesis 93 with final score -48.12 (-32.94 -15.18) is added. So, I really do not understand, which is discarded and how to decide, which one is "too bad for stack".

I mean, there are several instances, when hypothesis of lesser (i.e. good) final score is discarded and of more (bad) final score is added.


I request you to please explain it to me, giving hypothesis number, which is discarded in comparison and why.

Thanks & Regards
Ravi Agarwal

moses-support-request-3s7WtUTddSA@public.gmane.org wrote:

Send Moses-support mailing list submissions to
moses-support-3s7WtUTddSA@public.gmane.org

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request-3s7WtUTddSA@public.gmane.org

You can reach the person managing the list at
moses-support-owner-3s7WtUTddSA@public.gmane.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Understanding decoder output fields (Hieu Hoang)
2. Re: Student - Doubts regarding Statistical Machine
Translation - Decoder - Moses (Hieu Hoang)
3. Re: Student - Doubts regarding Statistical Machine
Translation - Decoder - Mo ses (Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Tue, 3 Apr 2007 21:27:08 +0100
From: "Hieu Hoang"
Subject: Re: [Moses-support] Understanding decoder output fields
To: "'Lane Schwartz'" , "'moses-support'"

Message-ID: <001501c7762e$73b98620$05000100 <at> very>
Content-Type: text/plain; charset="iso-8859-1"

Hi lane,

The scores are listed in the following order:
1. basic distortion
2. reordering
3. language models
4. phrase tables
5. word penalty
6. generation tables
I believe that's the order in which they're loaded, but it may not be
noticeable from the debugging output.

The order can be verified in the function
IOStream::OutputNBestList()
In the file
IOStream.cpp

If u add a
-labeled-n-best-list 1

The array of 15 numbers is annotated with some info of what they refer
to.

Hope that helps

Hieu Hoang
www.hoang.co.uk/hieu


-----Original Message-----
From: moses-support-bounces-3s7WtUTddSA@public.gmane.org
[mailto:moses-support-bounces-3s7WtUTddSA@public.gmane.org] On Behalf Of Lane Schwartz
Sent: 03 April 2007 16:57
To: moses-support
Subject: [Moses-support] Understanding decoder output fields


When moses decodes a sentence, and produces an BEST TRANSLATION, it
gives a total score, as well as an array of 15 other numbers.

I'm trying to figure out which of those 15 numbers is the target
language model score for the translation.


Here's what I get when I run the decoder:


Defined parameters (per moses.ini or switch):
config: /export/scratch/moses.nc.fr-en/working-dir/
evaluation/filtered.nc-devtest2007/moses.ini
distortion-file: 0-0 msd-bidirectional-fe 6 /export/scratch/
moses.nc.fr-en/workin g-dir/evaluation/filtered.nc-devtest2007/
reordering-table.msd-bidirectional-fe.0.5.0-0
distortion-limit: 6
input-factors: 0
input-file: /export/scratch/moses.nc.fr-en/working-dir/
evaluation/nc-devtest2007.input
lmodel-file: 0 0 5 /export/scratch/moses.nc.fr-en/working-
dir/lm/europarl-v3.lm
mapping: 0 T 0
ttable-file: 0 0 5 /export/scratch/moses.nc.fr-en/working-
dir/evaluation/filtered.nc-devtest2007/phrase-table.0-0
ttable-limit: 20 0
weight-d: 0.086065 0.110904 -0.015132 0.075996 0.115685
0.056790 0.024530
weight-l: 0.107263
weight-t: 0.019468 0.102550 0.127223 0.020522 0.060418
weight-w: 0.077454
Start loading distortion table /export/scratch/moses.nc.fr-en/working-
dir/evaluation/filtered.nc-devtest2007/reordering-table.msd-
bidirectional-fe.0.5.0-0: [0.000] seconds
Start loading LanguageMod el /export/scratch/moses.nc.fr-en/working-
dir/lm/europarl-v3.lm: [3.000] seconds
Finished loading LanguageModels : [18.000] seconds
Start loading PhraseTable /export/scratch/moses.nc.fr-en/working-dir/
evaluation/filtered.nc-devtest2007/phrase-table.0-0: [18.000] seconds
Finished loading phrase tables : [24.000] seconds Created input-output
object : [24.000] seconds
Translating: apprivoiser les politiciens des deux c?t?s de l' atlantique

BEST TRANSLATION: taming politicians on both sides of the atlantic
[111111111] [total=-7.621] <<0.000, -8.000, 0.000, -1.585, 0.000,
0.000, -1.820, 0.000, 0.000, -46.545, -3.730, -12.789, -1.979,
-11.109, 4.000>>
Finished translating


Thanks,
Lane Schwartz

_______________________________________________
Moses-support mailing list
Moses-support-3s7WtUTddSA@public.gmane.org
http://mailman.mit.edu/mailman/listinfo/moses-support




-------- ----------------------

Message: 2
Date: Tue, 3 Apr 2007 21:54:39 +0100
From: "Hieu Hoang"
Subject: Re: [Moses-support] Student - Doubts regarding Statistical
Machine Translation - Decoder - Moses
To:
Cc: 'moses-support'
Message-ID: <001601c77632$4e715100$05000100 <at> very>
Content-Type: text/plain; charset="US-ASCII"

Haven't been knighted by the queen yet. But Sir is a good title for
me...

A1.
Hypos are discarded when the number of hypos in a stack is twice as high
as the beam size.
The hypos with the worst scores are thrown away. The scores are pretty
simple to understand except 1 - the future score. However, this is
critical to making sure good results are obtained.

The scores are described in the pharaoh manual.

A2.
We expand all hypotheses in the previous stack. The order is dependant
on the iterator of the stack, which is dependant on how hypotheses are
compared for recombination. However, u should view the order as
arbitary.

I suppose this could affect the final outcome of the decoder - however,
I think the difference is small.

A3. If you have a trigram LM scoring the target sentence, then the last
2 target words translated is the only thing that can influence the LM
score of future hypos.

For example, 2 hypos with the following target sentence is guaranteed to
have the same incremental LM score going forward:
This is a
That is a
Because adding 'house' means both hypos will have an additional trigram
of
is a house

If the LM is a 4-gram, then the next 3 target words is important. If the
LM is a 100-gram, then the next 99 words is important.

In fact, moses is cleverer than the algorithm described above, the logic
of which I'm not going to go into.
Moses can also handle multiple LMs,

For n ow, the above explanation is a good starting point.

The code for reconbination is in the class
HypothesisRecombinationOrderer
In file
HypothesisStack.h

Hieu Hoang
www.hoang.co.uk/hieu


-----Original Message-----
From: Ravi Agarwal [mailto:ravi_agarwal-7uby7gWlsC8swqFILYFc3g@public.gmane.org]
Sent: 03 April 2007 11:37
To: hieuhoang1972-/E1597aS9LT10XsdtD+oqA@public.gmane.org
Subject: Student - Doubts regarding Statistical Machine Translation -
Decoder - Moses
Importance: High


Hello Sir Hoang,
This is Ravi Agarwal, a B.Tech Final year student in Computer
Science from Dhirubhai Ambani Institute Of Information and Communication
Technology, India. I request you to please clarify some of my doubts.
My final year project is to implement a Toy Decoder on
Statistical Machine Translation. I have read the 'User Manual' of
Pharaoh properly, and am performing experiments on the source code of
Moses. On executing the following comm and on the sample model provided
on Moses 'Get Started' web page - command : moses -f moses.ini -v 3
A detailed description of partial hypotheses is generated. I saved
this description in a file 'BehindDecoding.txt'. I've following doubts
regarding partial hypothesis building. You can answer my doubts in
respect to this saved file itself i.e. for this particular case of
example 'das ist ein kleines haus' to 'This is a small house'.

Q1 - On what basis hypotheses (partial translations) are discarded from
a stack ? Q2 - When we start processing a new stack, the hypothesis from
the previous stack are expanded. In which order, do you pick the
previous hypothesis i.e. how do you decide which hypothesis should be
expanded first ? Is it order inspecific ? Q3 - I am not exactly very
clear about when to recombine the hypotheses. In user manual of Pharaoh,
you have explained 3 points to be considered for recombining, But what
do you mean by the second point - 'the last two English words generated
?' Last two word counts presently translated word or just past two
words.
Moreover, Do you think that just for the specific example taken
- 'das ist ein kleines haus' to 'This is a small house', can we think of
some alternate condition for recombining. I mean the same conditions but
with some bypass due to the particular case.
If you are unable to understand my Q3, do not worry, may be I am
unable to explain it properly. Q4 - What is the use of the best
hypothesis. I mean the best hypothesis in the final stack gives us the
final and best translation of the given input foreign language sentence.
But what is the use of best hypothesis in the intermediate stacks ?


Please clarify my doubts.
Kindly look into the matter.
I will be highly obliged by you.
Thanking You.

Sincerely,
Ravi Agarwal
BTec h - Final Year (Computer Science)
Dhirubhai Ambani Institute Of Information and Communication Technology
Gujarat, India. http://www.daiict.ac.in Tel : (+91) 9924125684 gmail
users : ravi.rickie-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org






------------------------------

Message: 3
Date: Tue, 3 Apr 2007 21:57:00 +0100
From: "Hieu Hoang"
Subject: Re: [Moses-support] Student - Doubts regarding Statistical
Machine Translation - Decoder - Moses
To:
Cc: 'moses-support'
Message-ID: <001701c77632$a012af90$05000100 <at> very>
Content-Type: text/plain; charset="US-ASCII"

Ps. Please email questions to the 'moses-support' mailing list, rather
than directly to me.

I've got a spam filter which is highly likely to trash your mail if it
not from 'moses-support' and I've never emailed u before

Hieu Hoang
www.hoang.co.uk/hieu


-----Original Messa ge-----
From: Hieu Hoang [mailto:h.hoang-Y3tGgqFSo3OFxr2TtlUqVg@public.gmane.org]
Sent: 03 April 2007 21:55
To: 'ravi_agarwal-7uby7gWlsC8swqFILYFc3g@public.gmane.org'
Cc: 'moses-support'
Subject: RE: Student - Doubts regarding Statistical Machine Translation
- Decoder - Moses


Haven't been knighted by the queen yet. But Sir is a good title for
me...

A1.
Hypos are discarded when the number of hypos in a stack is twice as high
as the beam size. The hypos with the worst scores are thrown away. The
scores are pretty simple to understand except 1 - the future score.
However, this is critical to making sure good results are obtained.

The scores are described in the pharaoh manual.

A2.
We expand all hypotheses in the previous stack. The order is dependant
on the iterator of the stack, which is dependant on how hypotheses are
compared for recombination. However, u should view the order as
arbitary.

I suppose this could affect the final outcome of the d ecoder - however,
I think the difference is small.

A3. If you have a trigram LM scoring the target sentence, then the last
2 target words translated is the only thing that can influence the LM
score of future hypos.

For example, 2 hypos with the following target sentence is guaranteed to
have the same incremental LM score going forward:
This is a
That is a
Because adding 'house' means both hypos will have an additional trigram
of
is a house

If the LM is a 4-gram, then the next 3 target words is important. If the
LM is a 100-gram, then the next 99 words is important.

In fact, moses is cleverer than the algorithm described above, the logic
of which I'm not going to go into.
Moses can also handle multiple LMs,

For now, the above explanation is a good starting point.

The code for reconbination is in the class
HypothesisRecombinationOrderer
In file
HypothesisStack.h

Hieu Hoang
www.hoang.co.uk/hieu


-----Original Message-----
From: Ravi Agarwal [mailto:ravi_agarwal-7uby7gWlsC8swqFILYFc3g@public.gmane.org]
Sent: 03 April 2007 11:37
To: hieuhoang1972-/E1597aS9LT10XsdtD+oqA@public.gmane.org
Subject: Student - Doubts regarding Statistical Machine Translation -
Decoder - Moses
Importance: High


Hello Sir Hoang,
This is Ravi Agarwal, a B.Tech Final year student in Computer
Science from Dhirubhai Ambani Institute Of Information and Communication
Technology, India. I request you to please clarify some of my doubts.
My final year project is to implement a Toy Decoder on
Statistical Machine Translation. I have read the 'User Manual' of
Pharaoh properly, and am performing experiments on the source code of
Moses. On executing the following command on the sample model provided
on Moses 'Get Started' web page - command : moses -f moses.ini -v 3
A detailed description of partial hypotheses is gener ated. I saved
this description in a file 'BehindDecoding.txt'. I've following doubts
regarding partial hypothesis building. You can answer my doubts in
respect to this saved file itself i.e. for this particular case of
example 'das ist ein kleines haus' to 'This is a small house'.

Q1 - On what basis hypotheses (partial translations) are discarded from
a stack ? Q2 - When we start processing a new stack, the hypothesis from
the previous stack are expanded. In which order, do you pick the
previous hypothesis i.e. how do you decide which hypothesis should be
expanded first ? Is it order inspecific ? Q3 - I am not exactly very
clear about when to recombine the hypotheses. In user manual of Pharaoh,
you have explained 3 points to be considered for recombining, But what
do you mean by the second point - 'the last two English words generated
?' Last two word counts presently translated word or just past two
words.
Moreover, Do you think that just for the specific example taken
- 'das ist ein kleines haus' to 'This is a small house', can we think of
some alternate condition for recombining. I mean the same conditions but
with some bypass due to the particular case.
If you are unable to understand my Q3, do not worry, may be I am
unable to explain it properly. Q4 - What is the use of the best
hypothesis. I mean the best hypothesis in the final stack gives us the
final and best translation of the given input foreign language sentence.
But what is the use of best hypothesis in the intermediate stacks ?


Please clarify my doubts.
Kindly look into the matter.
I will be highly obliged by you.
Thanking You.

Sincerely,
Ravi Agarwal
BTech - Final Year (Computer Science)
Dhirubhai Ambani Institute Of Information and Communication Technology
Gujarat, India. http://www.daiict.ac.in Tel : (+91) 99241 25684 gmail
users : ravi.rickie-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org






------------------------------

_______________________________________________
Moses-support mailing list
Moses-support-3s7WtUTddSA@public.gmane.org
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 6, Issue 3
*******************************************
@public.gmane.org>@public.gmane.org>@public.gmane.org>@public.gmane.org>@public.gmane.org>@public.gmane.org>@public.gmane.org>@public.gmane.org>@public.gmane.org>

Here’s a new way to find what you're looking for - Yahoo! Answers
Hieu Hoang | 6 Apr 16:37 2007
Picon
Picon

lexicalised reordering

hi guys and girls,
 
konrad rawlik, philipp koehn and i have been working on a new implementation of lexicalised reordering to increase the speed, and to be able to use a binary load-on-demand re-oredring table, similar to the binary phrase table.
 
we introduced it a few weeks ago but rolled back due to problems people were having.
 
we've looked at it and fixed the bugs and re-introduced it. please update via svn to get the changes
 
hopefully, it will work this time, however, please let us know if it doesn't
 
for the moses coder out there, please update the regression test directory too, and download new regression test data from
thx
 
Hieu Hoang
 
Hieu Hoang | 6 Apr 16:46 2007
Picon
Picon

Re: MERT tuning phase time issues

Hiya maria,

Mert takes a long time, this is why mert has been developed to be able
use multiple machines using gridEngine. I urge you to make use of this
feature.

However, 6 min/sentence sounds excessive, and 30min/sentence is even
worse. 

If you are doing lexicalised reordering, this may explain some of the
slowness. We've just re-implmeneted this to be signifanctly faster,
please update via svn to get these changes.

I think the most common problem is not enough memory, however, u seem to
have enough - just check that moses isn't disk scratching.

I'm not sure why iteration 2 take so much more time. I have heard that
it may give worse result before getting better. Anyone else have any
experience ?

Hieu Hoang
www.hoang.co.uk/hieu

-----Original Message-----
From: moses-support-bounces@...
[mailto:moses-support-bounces@...] On Behalf Of Maria Holmqvist
Sent: 05 April 2007 10:13
To: moses-support@...
Subject: [Moses-support] MERT tuning phase time issues

Hello,

I have questions regarding the time it takes to perform MERT tuning. I'm
running Moses on AMD 64 Athlon 4000+ processor with 4 gigs of RAM and a
32 bit dist of Ubuntu. 
Since I had trouble earlier with MERT I reverted to svn 1295 as
suggested.

For the current experiment with factored translation the first iteration
of MERT required approx. 6 min/sentence . Iteration 2 requires approx.
30 min!!! /sentence and the result in run2.out is definitely worse than
in run1.out.

What time can usually be expected for MERT tuning? 
Why is iteration 2 so much slower than iteration 1?

Best regards,
Maria
Hieu Hoang | 6 Apr 16:59 2007
Picon
Picon

Re: Student - Doubts regarding Statistical Machine

hi ravi,

i don't think i can go through in as much details as you would wish.

if u want to find out more, i would advise u to look thru the code. 

for stuff relating to re-combination, look at the function
	void HypothesisStack::AddPrune(Hypothesis *hypo)
in HypothesisStack.cpp

The comparison of 2 hypotheses to determine whether they are 'the same'
so should be recombined is in the class
	HypothesisRecombinationOrderer
In HypothesisStack.h

The pharaoh manual has explanation of recombination, the moses
implementation follows this closely.

Hopes that helps

Hieu Hoang
www.hoang.co.uk/hieu
-----Original Message-----
From: moses-support-bounces@...
[mailto:moses-support-bounces@...] On Behalf Of ravi agarwal
Sent: 05 April 2007 19:04
To: moses-support@...
Subject: Re: [Moses-support] Student - Doubts regarding Statistical
Machine

Reply of Message 2 - regarding A1.
Sir, I am still confused of what do you mean by worse scores. Kindly
give me example particular to the concerned case of 'das ist ein kleines
haus' to 'This is a small house'.

e.g. -> In the 4th stack, hypothesis 312 with final score -42.55 (-38.29
-4.26) is discarded, where as hypothesis 93 with final score -48.12
(-32.94 -15.18) is added. So, I really do not understand, which is
discarded and how to decide, which one is "too bad for stack".

I mean, there are several instances, when hypothesis of lesser (i.e.
good) final score is discarded and of more (bad) final score is added.

I request you to please explain it to me, giving hypothesis number,
which is discarded in comparison and why. 

Thanks & Regards
Ravi Agarwal

moses-support-request@... wrote:
Send Moses-support mailing list submissions to
moses-support@...

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@...

You can reach the person managing the list at
moses-support-owner@...

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."

Today's Topics:

1. Re: Understanding decoder output fields (Hieu Hoang)
2. Re: Student - Doubts regarding Statistical Machine
Translation - Decoder - Moses (Hieu Hoang)
3. Re: Student - Doubts regarding Statistical Machine
Translation - Decoder - Moses (Hieu Hoang)

----------------------------------------------------------------------

Message: 1
Date: Tue, 3 Apr 2007 21:27:08 +0100
From: "Hieu Hoang" 
Subject: Re: [Moses-support] Understanding decoder output fields
To: "'Lane Schwartz'" , "'moses-support'"

Message-ID: <001501c7762e$73b98620$05000100 <at> very>
Content-Type: text/plain; charset="iso-8859-1"

Hi lane,

The scores are listed in the following order:
1. basic distortion
2. reordering
3. language models
4. phrase tables
5. word penalty
6. generation tables
I believe that's the order in which they're loaded, but it may not be
noticeable from the debugging output.

The order can be verified in the function 
IOStream::OutputNBestList()
In the file
IOStream.cpp

If u add a 
-labeled-n-best-list 1

The array of 15 numbers is annotated with some info of what they refer
to.

Hope that helps

Hieu Hoang
www.hoang.co.uk/hieu

-----Original Message-----
From: moses-support-bounces@...
[mailto:moses-support-bounces@...] On Behalf Of Lane Schwartz
Sent: 03 April 2007 16:57
To: moses-support
Subject: [Moses-support] Understanding decoder output fields

When moses decodes a sentence, and produces an BEST TRANSLATION, it 
gives a total score, as well as an array of 15 other numbers.

I'm trying to figure out which of those 15 numbers is the target 
language model score for the translation.

Here's what I get when I run the decoder:

Defined parameters (per moses.ini or switch):
config: /export/scratch/moses.nc.fr-en/working-dir/ 
evaluation/filtered.nc-devtest2007/moses.ini
distortion-file: 0-0 msd-bidirectional-fe 6 /export/scratch/ 
moses.nc.fr-en/working-dir/evaluation/filtered.nc-devtest2007/ 
reordering-table.msd-bidirectional-fe.0.5.0-0
distortion-limit: 6
input-factors: 0
input-file: /export/scratch/moses.nc.fr-en/working-dir/ 
evaluation/nc-devtest2007.input
lmodel-file: 0 0 5 /export/scratch/moses.nc.fr-en/working- 
dir/lm/europarl-v3.lm
mapping: 0 T 0
ttable-file: 0 0 5 /export/scratch/moses.nc.fr-en/working- 
dir/evaluation/filtered.nc-devtest2007/phrase-table.0-0
ttable-limit: 20 0
weight-d: 0.086065 0.110904 -0.015132 0.075996 0.115685 
0.056790 0.024530
weight-l: 0.107263
weight-t: 0.019468 0.102550 0.127223 0.020522 0.060418
weight-w: 0.077454
Start loading distortion table /export/scratch/moses.nc.fr-en/working- 
dir/evaluation/filtered.nc-devtest2007/reordering-table.msd- 
bidirectional-fe.0.5.0-0: [0.000] seconds
Start loading LanguageModel /export/scratch/moses.nc.fr-en/working- 
dir/lm/europarl-v3.lm: [3.000] seconds
Finished loading LanguageModels : [18.000] seconds
Start loading PhraseTable /export/scratch/moses.nc.fr-en/working-dir/ 
evaluation/filtered.nc-devtest2007/phrase-table.0-0: [18.000] seconds
Finished loading phrase tables : [24.000] seconds Created input-output
object : [24.000] seconds
Translating: apprivoiser les politiciens des deux c?t?s de l' atlantique

BEST TRANSLATION: taming politicians on both sides of the atlantic 
[111111111] [total=-7.621] <<0.000, -8.000, 0.000, -1.585, 0.000, 
0.000, -1.820, 0.000, 0.000, -46.545, -3.730, -12.789, -1.979, 
-11.109, 4.000>>
Finished translating

Thanks,
Lane Schwartz

_______________________________________________
Moses-support mailing list
Moses-support@...
http://mailman.mit.edu/mailman/listinfo/moses-support

------------------------------

Message: 2
Date: Tue, 3 Apr 2007 21:54:39 +0100
From: "Hieu Hoang" 
Subject: Re: [Moses-support] Student - Doubts regarding Statistical
Machine Translation - Decoder - Moses
To: 
Cc: 'moses-support' 
Message-ID: <001601c77632$4e715100$05000100 <at> very>
Content-Type: text/plain; charset="US-ASCII"

Haven't been knighted by the queen yet. But Sir is a good title for
me...

A1.
Hypos are discarded when the number of hypos in a stack is twice as high
as the beam size.
The hypos with the worst scores are thrown away. The scores are pretty
simple to understand except 1 - the future score. However, this is
critical to making sure good results are obtained.

The scores are described in the pharaoh manual.

A2.
We expand all hypotheses in the previous stack. The order is dependant
on the iterator of the stack, which is dependant on how hypotheses are
compared for recombination. However, u should view the order as
arbitary.

I suppose this could affect the final outcome of the decoder - however,
I think the difference is small.

A3. If you have a trigram LM scoring the target sentence, then the last
2 target words translated is the only thing that can influence the LM
score of future hypos.

For example, 2 hypos with the following target sentence is guaranteed to
have the same incremental LM score going forward:
This is a
That is a
Because adding 'house' means both hypos will have an additional trigram
of
is a house

If the LM is a 4-gram, then the next 3 target words is important. If the
LM is a 100-gram, then the next 99 words is important.

In fact, moses is cleverer than the algorithm described above, the logic
of which I'm not going to go into. 
Moses can also handle multiple LMs, 

For now, the above explanation is a good starting point.

The code for reconbination is in the class
HypothesisRecombinationOrderer
In file
HypothesisStack.h

Hieu Hoang
www.hoang.co.uk/hieu

-----Original Message-----
From: Ravi Agarwal [mailto:ravi_agarwal@...] 
Sent: 03 April 2007 11:37
To: hieuhoang1972@...
Subject: Student - Doubts regarding Statistical Machine Translation -
Decoder - Moses
Importance: High

Hello Sir Hoang,
This is Ravi Agarwal, a B.Tech Final year student in Computer
Science from Dhirubhai Ambani Institute Of Information and Communication
Technology, India. I request you to please clarify some of my doubts.
My final year project is to implement a Toy Decoder on
Statistical Machine Translation. I have read the 'User Manual' of
Pharaoh properly, and am performing experiments on the source code of
Moses. On executing the following command on the sample model provided
on Moses 'Get Started' web page - command : moses -f moses.ini -v 3
A detailed description of partial hypotheses is generated. I saved
this description in a file 'BehindDecoding.txt'. I've following doubts
regarding partial hypothesis building. You can answer my doubts in
respect to this saved file itself i.e. for this particular case of
example 'das ist ein kleines haus' to 'This is a small house'.

Q1 - On what basis hypotheses (partial translations) are discarded from
a stack ? Q2 - When we start processing a new stack, the hypothesis from
the previous stack are expanded. In which order, do you pick the
previous hypothesis i.e. how do you decide which hypothesis should be
expanded first ? Is it order inspecific ? Q3 - I am not exactly very
clear about when to recombine the hypotheses. In user manual of Pharaoh,
you have explained 3 points to be considered for recombining, But what
do you mean by the second point - 'the last two English words generated
?' Last two word counts presently translated word or just past two
words.
Moreover, Do you think that just for the specific example taken
- 'das ist ein kleines haus' to 'This is a small house', can we think of
some alternate condition for recombining. I mean the same conditions but
with some bypass due to the particular case.
If you are unable to understand my Q3, do not worry, may be I am
unable to explain it properly. Q4 - What is the use of the best
hypothesis. I mean the best hypothesis in the final stack gives us the
final and best translation of the given input foreign language sentence.
But what is the use of best hypothesis in the intermediate stacks ?

Please clarify my doubts.
Kindly look into the matter.
I will be highly obliged by you.
Thanking You.

Sincerely,
Ravi Agarwal
BTech - Final Year (Computer Science)
Dhirubhai Ambani Institute Of Information and Communication Technology
Gujarat, India. http://www.daiict.ac.in Tel : (+91) 9924125684 gmail
users : ravi.rickie@...

------------------------------

Message: 3
Date: Tue, 3 Apr 2007 21:57:00 +0100
From: "Hieu Hoang" 
Subject: Re: [Moses-support] Student - Doubts regarding Statistical
Machine Translation - Decoder - Moses
To: 
Cc: 'moses-support' 
Message-ID: <001701c77632$a012af90$05000100 <at> very>
Content-Type: text/plain; charset="US-ASCII"

Ps. Please email questions to the 'moses-support' mailing list, rather
than directly to me.

I've got a spam filter which is highly likely to trash your mail if it
not from 'moses-support' and I've never emailed u before

Hieu Hoang
www.hoang.co.uk/hieu

-----Original Message-----
From: Hieu Hoang [mailto:h.hoang@...] 
Sent: 03 April 2007 21:55
To: 'ravi_agarwal@...'
Cc: 'moses-support'
Subject: RE: Student - Doubts regarding Statistical Machine Translation
- Decoder - Moses

Haven't been knighted by the queen yet. But Sir is a good title for
me...

A1.
Hypos are discarded when the number of hypos in a stack is twice as high
as the beam size. The hypos with the worst scores are thrown away. The
scores are pretty simple to understand except 1 - the future score.
However, this is critical to making sure good results are obtained.

The scores are described in the pharaoh manual.

A2.
We expand all hypotheses in the previous stack. The order is dependant
on the iterator of the stack, which is dependant on how hypotheses are
compared for recombination. However, u should view the order as
arbitary.

I suppose this could affect the final outcome of the decoder - however,
I think the difference is small.

A3. If you have a trigram LM scoring the target sentence, then the last
2 target words translated is the only thing that can influence the LM
score of future hypos.

For example, 2 hypos with the following target sentence is guaranteed to
have the same incremental LM score going forward:
This is a
That is a
Because adding 'house' means both hypos will have an additional trigram
of
is a house

If the LM is a 4-gram, then the next 3 target words is important. If the
LM is a 100-gram, then the next 99 words is important.

In fact, moses is cleverer than the algorithm described above, the logic
of which I'm not going to go into. 
Moses can also handle multiple LMs, 

For now, the above explanation is a good starting point.

The code for reconbination is in the class
HypothesisRecombinationOrderer
In file
HypothesisStack.h

Hieu Hoang
www.hoang.co.uk/hieu

-----Original Message-----
From: Ravi Agarwal [mailto:ravi_agarwal@...] 
Sent: 03 April 2007 11:37
To: hieuhoang1972@...
Subject: Student - Doubts regarding Statistical Machine Translation -
Decoder - Moses
Importance: High

Hello Sir Hoang,
This is Ravi Agarwal, a B.Tech Final year student in Computer
Science from Dhirubhai Ambani Institute Of Information and Communication
Technology, India. I request you to please clarify some of my doubts.
My final year project is to implement a Toy Decoder on
Statistical Machine Translation. I have read the 'User Manual' of
Pharaoh properly, and am performing experiments on the source code of
Moses. On executing the following command on the sample model provided
on Moses 'Get Started' web page - command : moses -f moses.ini -v 3
A detailed description of partial hypotheses is generated. I saved
this description in a file 'BehindDecoding.txt'. I've following doubts
regarding partial hypothesis building. You can answer my doubts in
respect to this saved file itself i.e. for this particular case of
example 'das ist ein kleines haus' to 'This is a small house'.

Q1 - On what basis hypotheses (partial translations) are discarded from
a stack ? Q2 - When we start processing a new stack, the hypothesis from
the previous stack are expanded. In which order, do you pick the
previous hypothesis i.e. how do you decide which hypothesis should be
expanded first ? Is it order inspecific ? Q3 - I am not exactly very
clear about when to recombine the hypotheses. In user manual of Pharaoh,
you have explained 3 points to be considered for recombining, But what
do you mean by the second point - 'the last two English words generated
?' Last two word counts presently translated word or just past two
words.
Moreover, Do you think that just for the specific example taken
- 'das ist ein kleines haus' to 'This is a small house', can we think of
some alternate condition for recombining. I mean the same conditions but
with some bypass due to the particular case.
If you are unable to understand my Q3, do not worry, may be I am
unable to explain it properly. Q4 - What is the use of the best
hypothesis. I mean the best hypothesis in the final stack gives us the
final and best translation of the given input foreign language sentence.
But what is the use of best hypothesis in the intermediate stacks ?

Please clarify my doubts.
Kindly look into the matter.
I will be highly obliged by you.
Thanking You.

Sincerely,
Ravi Agarwal
BTech - Final Year (Computer Science)
Dhirubhai Ambani Institute Of Information and Communication Technology
Gujarat, India. http://www.daiict.ac.in Tel : (+91) 9924125684 gmail
users : ravi.rickie@...

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@...
http://mailman.mit.edu/mailman/listinfo/moses-support

End of Moses-support Digest, Vol 6, Issue 3
*******************************************

Here's a new way to find what you're looking for - Yahoo! Answers 


Gmane