Martin Mueller | 21 Dec 20:30 2014
Picon
Picon

Re: Move Evaluation in Go Using Deep Convolutional Neural Networks

Aja and co-authors,

first of all, congratulations on outstanding results!

I have some questions: 

- Would you be willing to share some of the sgf game records played by your network with the community? I tried
to replay the game record in your paper, but got stuck since it does not show any of the moves that got captured.

- Do you know how large is the effect from using the extra features that are not in the paper by Clarke and
Storkey, i.e. the last move info and the extra tactics? As a related question, would you get an OK result if
you just zeroed out some inputs in the existing net, or would you need to re-train a new network from fewer inputs.

- Is there a way to simplify the final network so that it is faster to compute and/or easier to understand? Is
there something computed, maybe on an intermediate layer, that would be usable as a new feature in itself?

Thanks

	Martin
_______________________________________________
Computer-go mailing list
Computer-go <at> computer-go.org
http://computer-go.org/mailman/listinfo/computer-go
Martin Mueller | 21 Dec 20:02 2014
Picon
Picon

Re: Fuego 1.1 vs current Fuego

> You remind me we should write Fuego's version as "1.1.SVN" rather than "1.1”.

Aja,

"1.1.SVN" does not tell the reader anything useful.

the svn revision number is given by the svn info command. That revision number allows you to check out the
exactly same program version.

% svn info
…
Revision: 1967
…

	Martin
_______________________________________________
Computer-go mailing list
Computer-go <at> computer-go.org
http://computer-go.org/mailman/listinfo/computer-go
Hugh Perkins | 21 Dec 00:59 2014
Picon

Interesting paper from Nando de Freitas team, possible replacement for UCT?

Interesting looking paper: "On correlation and budget constraints in
model-based bandit optimization with application to automatic machine
learning", Hoffman, Shahriari, de Freitas,  AISTATS 2014

I can't say I've entirely understood yet, but I *think* that:
- targets scenario where there are many more arms than we can try, ie
trying an arm is quite expensive.
  - I think this sounds like the situation somewhere around depth 2 to
4 of an MCTS tree, where we only have time to try a couple of
different moves?
- using Gaussian Processes to model correlations between the arms
   - means that they dont actually have to try all arms
   - though I haven't figured out yet where they are getting this
correlation information from yet, if they havent tried the arms yet...
- in their experiments, their UGap compares favorably with UCBE, which
is a variant of UCB (and which UCB is the basis of UCT, right?)

From the point of view of UCT, I'm thinking:
- might be more principled approach to deciding what to do when there
are still few children of an MCTS node
    - my understanding is that currently we are using 'magic numbers',
like 'if there are less than 30 children, then always explore, dont
use UCT'?

My own interest in this line of thought initially is slightly
different angle from replacing UCT.  It seems to me that there are
lots of hyper-parameters we need to optimize, or choose from, lots of
"magic numbers", like the "< 30" magic number for applying UCT, but
also things like:
- what is the value of adding RAVE?
(Continue reading)

Martin Mueller | 21 Dec 00:27 2014
Picon
Picon

Fuego 1.1 vs current Fuego

> I ran Fuego (latest svn version) on our machine (Intel(R) Xeon(R) CPU

Aja,
the 1.1 release of Fuego is from 2011 and much different from current svn Fuego. Current version has many
different kinds of knowledge added. It is also much slower, maybe 5x. If you read my blog it explains this a
little more.

	Martin
_______________________________________________
Computer-go mailing list
Computer-go <at> computer-go.org
http://computer-go.org/mailman/listinfo/computer-go
Martin Mueller | 21 Dec 00:06 2014
Picon
Picon

Re: Move Evaluation in Go Using Deep Convolutional Neural Networks

I think many of the programs have a mechanism for dealing with “slow” knowledge. For example in Fuego, you can call a knowledge function for each node that reaches some threshold T of playouts. The new technical challenge is dealing with the GPU. I know nothing about it myself, but from what I read it seems to work best in batch mode - you don’t want to send single positions for GPU evaluation back and forth.

My impression is that we will see a combination of both in the future - “normal”, fast knowledge which can be called as initialization in every node, and can be learned by Remi Coulom’s method (e.g. Crazy Stone, Aya, Erica) or by Wistuba’s (e.g. Fuego). And then, on top of that a mechanism to improve the bias using the slower deep networks running on the GPU.

It would be wonderful if some of us could work on an open source network evaluator to integrate with Fuego (or pachi or oakfoam). I know that Clark and Storkey are planning to open source theirs, but not in the very near future. I do not know about the plans of the Google DeepMind group, but they do mention something about a strong Go program in their paper :)

Martin

Thanks for sharing. I'm intrigued by your strategy for integrating
with MCTS. It's clear that latency is a challenge for integration. Do
you have any statistics on how many searches new nodes had been
through by the time the predictor comes back with an estimation? Did
you try any prefetching techniques? Because the CNN will guide much of
the search at the frontier of the tree, prefetching should be
tractable.

Did you do any comparisons between your MCTS with and w/o CNN? That's
the direction that many of us will be attempting over the next few
months it seems :)

_______________________________________________
Computer-go mailing list
Computer-go <at> computer-go.org
http://computer-go.org/mailman/listinfo/computer-go
Álvaro Begué | 20 Dec 16:43 2014
Picon

Building a database for training CNNs


Hi,

There are things a CNN could probably do well, if only we had the right database to train it. I have in mind these two possibilities:
 * using a CNN as an evaluation function,
 * using a CNN to estimate "ownership" for each point (i.e., a number between -1 and 1 that is an estimate of who is going to end up scoring it).

So we need a large set of positions labelled with a final score for the game and who ended up scoring each point.

I believe the right database to use for this purpose would consist of positions from games played by strong computer opponents which play to maximize score and which play to the bitter end, passing only when the opponent has no dead stones left on the board.

I would like to know if you think this would be an interesting resource to have, if you have any recommendations on what engine(s) to use and if you would be willing to collaborate in creating it. Any other comments are welcome too, of course.

Cheers,
Álvaro.



_______________________________________________
Computer-go mailing list
Computer-go <at> computer-go.org
http://computer-go.org/mailman/listinfo/computer-go
Hugh Perkins | 20 Dec 14:03 2014
Picon

Re: Computer-go Digest, Vol 59, Issue 25

Detlef wrote:
> P.S.: As we all might be trying to start incorporating NN into our
engines, we might bundle our resources, at least for the first start?!
Maybe exchanging open source software links for NN. I personally would
have started trying NN some time ago, if iOS had OpenCL support, as my
aim is to get a strong iPad go program....

I would think that if we just want the 'playing game' part, that we
don't each need the whole training-bit, just the results of the
training, and since this is the most expensive bit, one option could
be to somehow 'crowd-source' generation of these weights?  (Although,
if that turns out anything like voxforge, that will takes years ....
http://voxforge.org/home/downloads/metrics ... so it might be easier
to just shell out the 2000usd or so to run training oneself.)

On 12/20/14, computer-go-request <at> computer-go.org
<computer-go-request <at> computer-go.org> wrote:
> Send Computer-go mailing list submissions to
> 	computer-go <at> computer-go.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> 	http://computer-go.org/mailman/listinfo/computer-go
> or, via email, send a message with subject or body 'help' to
> 	computer-go-request <at> computer-go.org
>
> You can reach the person managing the list at
> 	computer-go-owner <at> computer-go.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Computer-go digest..."
>
>
> Today's Topics:
>
>    1. Re: Teaching Deep Convolutional Neural Networks	toPlay Go
>       (Hiroshi Yamashita)
>    2. Teaching Deep Convolutional Neural Networks to Play Go
>       (Hugh Perkins)
>    3. Move Evaluation in Go Using Deep Convolutional Neural
>       Networks (Hugh Perkins)
>    4. Re: Move Evaluation in Go Using Deep Convolutional Neural
>       Networks (Stefan Kaitschick)
>    5. Re: Move Evaluation in Go Using Deep Convolutional Neural
>       Networks (Robert Jasiek)
>    6. Re: Move Evaluation in Go Using Deep Convolutional Neural
>       Networks (Detlef Schmicker)
>    7. Re: Move Evaluation in Go Using Deep Convolutional
>       NeuralNetworks (Hiroshi Yamashita)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sat, 20 Dec 2014 14:16:18 +0900
> From: "Hiroshi Yamashita" <yss <at> bd.mbn.or.jp>
> To: <computer-go <at> computer-go.org>
> Subject: Re: [Computer-go] Teaching Deep Convolutional Neural Networks
> 	toPlay Go
> Message-ID: <40BCB936B9524E928ECA04073CBD800D <at> i3540>
> Content-Type: text/plain; format=flowed; charset="iso-8859-1";
> 	reply-type=original
>
> Hi Martin,
>
>> I put two commented games on
>> http://webdocs.cs.ualberta.ca/~mmueller/fuego/Convolutional-Neural-Network.html
>>
>
> Thank you for the report. It was fun.
> I'm also surprised CNN can play move 185 in Game 1.
> CNN uses "1, 2, or 3 or more liberties" info. B libs changed
>  from 4 to 3. And W libs was 3. It looks CNN can not understand
>  this difference, but he could.
>
> Hiroshi Yamashita
>
>
>
> ------------------------------
>
> Message: 2
> Date: Sat, 20 Dec 2014 15:54:02 +0800
> From: Hugh Perkins <hughperkins <at> gmail.com>
> To: computer-go <at> computer-go.org
> Subject: [Computer-go] Teaching Deep Convolutional Neural Networks to
> 	Play Go
> Message-ID:
> 	<CANvm6qAUHLNoM16BdRaKDkCQzBOEicv8NtSGjvjRifj9zWzB0A <at> mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
>
> On Sun Dec 14 23:53:45 UTC 201, Hiroshi Yamashita wrote:
>> Teaching Deep Convolutional Neural Networks to Play Go
>> http://arxiv.org/pdf/1412.3409v1.pdf
>
> Wow, this resembles somewhat what I was hoping to do!  But now I
> should look for some other avenue :-)  But
> I'm surprised it's only published on arxiv, I was hoping that
> something like this could
> be conference-worthy?
>
>
> ------------------------------
>
> Message: 3
> Date: Sat, 20 Dec 2014 16:37:21 +0800
> From: Hugh Perkins <hughperkins <at> gmail.com>
> To: computer-go <at> computer-go.org
> Subject: [Computer-go] Move Evaluation in Go Using Deep Convolutional
> 	Neural	Networks
> Message-ID:
> 	<CANvm6qCGKcuafdaSY2bcgH16KF=VG15SYTjV8ByZKq8viPt8Sg <at> mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
>
> On Fri Dec 19 23:17:23 UTC 2014, Aja Huang wrote:
>> We've just submitted our paper to ICLR. We made the draft available at
>> http://www.cs.toronto.edu/~cmaddis/pubs/deepgo.pdf
>
> Cool... just out of curiosity, did a back-of-an-envelope estimation of the
> cost of training your and Clark and Storkey's network, if renting time
> on AWS GPU instances and came up with:
>    - Clark and Storkey: 125 usd (4 days * 2 instances * 0.65usd/hour)
>    - Yours: 2025usd(cost of Clark and Storkey * 25/7 epochs *
>        29.4/14.7 action-pairs * 12/8 layers)
>
> Probably a bit high for me personally to just spend one weekend for fun,
> but not outrageous at all in fact, if the same technique was being used by
> an organization.
>
>
> ------------------------------
>
> Message: 4
> Date: Sat, 20 Dec 2014 09:43:49 +0100
> From: Stefan Kaitschick <stefan.kaitschick <at> hamburg.de>
> To: computer-go <at> computer-go.org
> Subject: Re: [Computer-go] Move Evaluation in Go Using Deep
> 	Convolutional Neural Networks
> Message-ID:
> 	<CAJRqE7xjy5r=rWXQ9pJMOxSV3BAcA1j1qiq3a-cgFzaJnREjrw <at> mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Great work. Looks like the age of nn is here.
> How does this compare in computation time to a heavy MC move generator?
>
> One very minor quibble, I feel like a nag for even mentioning it:  You write
> "The most frequently cited reason for the difficulty of Go, compared to
> games such as Chess, Scrabble
> or Shogi, is the difficulty of constructing an evaluation function that can
> differentiate good moves
> from bad in a given position."
>
> If MC has shown anything, it's that computationally, it's much easier to
> suggest a good move, than to evaluate the position.
> This is still true with your paper, it's just that the move suggestion has
> become even better.
>
> Stefan
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> <http://computer-go.org/pipermail/computer-go/attachments/20141220/27c1afe7/attachment-0001.html>
>
> ------------------------------
>
> Message: 5
> Date: Sat, 20 Dec 2014 10:24:03 +0100
> From: Robert Jasiek <jasiek <at> snafu.de>
> To: computer-go <at> computer-go.org
> Subject: Re: [Computer-go] Move Evaluation in Go Using Deep
> 	Convolutional Neural Networks
> Message-ID: <54954033.8090200 <at> snafu.de>
> Content-Type: text/plain; charset=UTF-8; format=flowed
>
> On 20.12.2014 09:43, Stefan Kaitschick wrote:
>> If MC has shown anything, it's that computationally, it's much easier to
>> suggest a good move, than to evaluate the position.
>
> Such can only mean an improper understanding of positional judgement.
> Positional judgement depends on reading (or MC simulation of reading)
> but the reading has a much smaller computational complexity because
> localisation and quiescience apply.
>
> The major aspects of positional judgement are territory and influence.
> Evaluating influence is much easier than evaluating territory if one
> uses a partial influence concept: influence stone difference. Its major
> difficulty is the knowledge of which stones are alive or not, however,
> MC simulations applied to outside stones should be able to assess such
> with reasonable certainty fairly quickly. Hence, the major work of
> positional judgement is assessment of territory. See my book Positional
> Judgement 1 - Territory for that. By designing (heuristically or using a
> low level expert system) MC for its methods, territorial positional
> judgement by MC should be much faster than ordinary MC because much
> fewer simulations should do. However, it is not as elegant as ordinary
> MC because some expert knowledge is necessary or must be approximated
> heuristically. Needless to say, keep the computational complexity of
> this expert knowledge low.
>
> --
> robert jasiek
>
>
> ------------------------------
>
> Message: 6
> Date: Sat, 20 Dec 2014 11:21:16 +0100
> From: Detlef Schmicker <ds2 <at> physik.de>
> To: computer-go <at> computer-go.org
> Subject: Re: [Computer-go] Move Evaluation in Go Using Deep
> 	Convolutional Neural Networks
> Message-ID: <1419070876.13427.7.camel <at> physik.de>
> Content-Type: text/plain; charset="UTF-8"
>
> Am Samstag, den 20.12.2014, 09:43 +0100 schrieb Stefan Kaitschick:
>> Great work. Looks like the age of nn is here.
>>
>> How does this compare in computation time to a heavy MC move
>> generator?
>>
>>
>> One very minor quibble, I feel like a nag for even mentioning it:  You
>> write
>> "The most frequently cited reason for the difficulty of Go, compared
>> to games such as Chess, Scrabble
>> or Shogi, is the difficulty of constructing an evaluation function
>> that can differentiate good moves
>> from bad in a given position."
>>
>>
>> If MC has shown anything, it's that computationally, it's much easier
>> to suggest a good move, than to evaluate the position.
>>
>> This is still true with your paper, it's just that the move suggestion
>> has become even better.
>
> It is, but I do not think, that this is necessarily a feature of NN.
> NNs might be a good evaluators, but it is much easier to train them for
> a move predictor, as it is not easy to get training data sets for an
> evaluation function?!
>
> Detlef
>
> P.S.: As we all might be trying to start incorporating NN into our
> engines, we might bundle our resources, at least for the first start?!
> Maybe exchanging open source software links for NN. I personally would
> have started trying NN some time ago, if iOS had OpenCL support, as my
> aim is to get a strong iPad go program....
>
>>
>>
>> Stefan
>>
>>
>>
>>
>> _______________________________________________
>> Computer-go mailing list
>> Computer-go <at> computer-go.org
>> http://computer-go.org/mailman/listinfo/computer-go
>
>
>
>
> ------------------------------
>
> Message: 7
> Date: Sat, 20 Dec 2014 20:33:30 +0900
> From: "Hiroshi Yamashita" <yss <at> bd.mbn.or.jp>
> To: <computer-go <at> computer-go.org>
> Subject: Re: [Computer-go] Move Evaluation in Go Using Deep
> 	Convolutional	NeuralNetworks
> Message-ID: <BC600A945BD9499E9D0B86865BA36BD5 <at> i3540>
> Content-Type: text/plain; format=flowed; charset="UTF-8";
> 	reply-type=original
>
> Hi Aja,
>
>> I hope you enjoy our work. Comments and questions are welcome.
>
> I have three questions.
>
> I don't understand minibatch.
> Does CNN need 0.15sec for a positon, or 0.15sec for 128 positions?
>
>   ABCDEFGHJ
>  9.........   White(O) to move.
>  8...OO....   Previous Black move is H5(X)
>  7..XXXOO..
>  6.....XXO.
>  5.......X.
>  4.........
>  3....XO...
>  2....OX...
>  1.........
>   ABCDEFGHJ
>
> "Liberties after move" means
>   H7(O) is 5, F8(O) is 6.
> "Liberties" means
>   H5(X) is 3, H6(O) is 2.
> "Ladder move" means
>   G2(O), not E6(O).
>
> Is this correct?
>
> Is "KGS rank" set 9 dan when it plays against Fuego?
>
> Regards,
> Hiroshi Yamashita
>
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> Computer-go mailing list
> Computer-go <at> computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
>
> ------------------------------
>
> End of Computer-go Digest, Vol 59, Issue 25
> *******************************************
>
_______________________________________________
Computer-go mailing list
Computer-go <at> computer-go.org
http://computer-go.org/mailman/listinfo/computer-go
Hugh Perkins | 20 Dec 09:37 2014
Picon

Move Evaluation in Go Using Deep Convolutional Neural Networks

On Fri Dec 19 23:17:23 UTC 2014, Aja Huang wrote:
> We've just submitted our paper to ICLR. We made the draft available at
> http://www.cs.toronto.edu/~cmaddis/pubs/deepgo.pdf

Cool... just out of curiosity, did a back-of-an-envelope estimation of the
cost of training your and Clark and Storkey's network, if renting time
on AWS GPU instances and came up with:
   - Clark and Storkey: 125 usd (4 days * 2 instances * 0.65usd/hour)
   - Yours: 2025usd(cost of Clark and Storkey * 25/7 epochs *
       29.4/14.7 action-pairs * 12/8 layers)

Probably a bit high for me personally to just spend one weekend for fun,
but not outrageous at all in fact, if the same technique was being used by
an organization.
_______________________________________________
Computer-go mailing list
Computer-go <at> computer-go.org
http://computer-go.org/mailman/listinfo/computer-go
Hugh Perkins | 20 Dec 08:54 2014
Picon

Teaching Deep Convolutional Neural Networks to Play Go

On Sun Dec 14 23:53:45 UTC 201, Hiroshi Yamashita wrote:
> Teaching Deep Convolutional Neural Networks to Play Go
> http://arxiv.org/pdf/1412.3409v1.pdf

Wow, this resembles somewhat what I was hoping to do!  But now I
should look for some other avenue :-)  But
I'm surprised it's only published on arxiv, I was hoping that
something like this could
be conference-worthy?
_______________________________________________
Computer-go mailing list
Computer-go <at> computer-go.org
http://computer-go.org/mailman/listinfo/computer-go
Martin Mueller | 20 Dec 04:23 2014
Picon
Picon

Re: Teaching Deep Convolutional Neural Networks to Play Go

I put two commented games on

Enjoy!

Martin

_______________________________________________
Computer-go mailing list
Computer-go <at> computer-go.org
http://computer-go.org/mailman/listinfo/computer-go
Aja Huang | 20 Dec 00:17 2014
Picon

Move Evaluation in Go Using Deep Convolutional Neural Networks

Hi all,

We've just submitted our paper to ICLR. We made the draft available at

I hope you enjoy our work. Comments and questions are welcome.

Regards,
Aja
_______________________________________________
Computer-go mailing list
Computer-go <at> computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Gmane