Razvan Cojocaru | 1 Nov 02:32 2010

Re: Large amounts of actors?

I think that if you read down this same thread, you will see that tests with 
100k+ akka actors were fine with minimal to no  overhead. Even the standard 
actor library should be fine with large numbers of actors - what actors are 
is nothing but a package of work to be done and a mailbox - their size will 
really depend on your runtime logic and performance.

What you have here is about 1G so perfectly feasible, especially with a 
64bit JVM.

-----Original Message----- 
From: Malte Finsterwalder
Sent: Friday, October 29, 2010 12:09 PM
To: scala-user@...
Subject: [scala-user] Large amounts of actors?

Hi there,

I'm evaluating scala at the moment for a specific problem. And I'm
wondering, whether my design ideas are at all reasonable. I have at
least 10000 objects, with an average of 100k each. Is it viable and
smart to create all of there objects as actors? That would probably
make my design simple, but I'm wondering, whether the actors
implementation can handle such a large amount of actors. Where are
reasonable limits?

Any ideas? Is there some material that I should look into?

Thanks,
    Malte 

(Continue reading)

Razvan Cojocaru | 1 Nov 02:44 2010

Re: Large amounts of actors?

Well, if you put it this way...I still disagree  
 
I think actors are a paradigm and once chosen, one should stick with it.
 
That being said, if you still need or feel like forking threads inside an actor, take a look at the seq/par goodness in scala workflows: http://github.com/razie/gremlins/blob/master/ScalaWorkflows.markdown
 
Indeed, I can see reasons why you would, in an actor, spawn something like
 
(later { do long running X; send message to actor Y }) start
 
But really, all you’re doing here is
1. simplifying invocation of the body of later (as opposed to wrapping it in an actor and sending it messages, which is why I think this workflow style presentation is easier, sometimes) and
2. use a different thread pool...
 
Cheers,
Razie
 
Sent: Friday, October 29, 2010 12:25 PM
Subject: Re: [scala-user] Large amounts of actors?
 
You can have as many actors as you want, as long as:
1) They don't block (IO, receive, locking objects, etc)
2) They don't do long running computations
Basically an actor is "just an object" with minimal overhead over a "plain object" as long as it's not tying up a thread.  The threads actors use by default weigh a ton (the usual stack baggage of threads, plus a rather large array-based deque for local task manageent). So you need to use react, avoid IO or other blocking operations, and don't have it do long running computations, especially without occasionally freeing up its thread.
 
On Fri, Oct 29, 2010 at 12:09 PM, Malte Finsterwalder <malte-pYrRtGl7Kb9PKalg20xefUEMvNT87kid@public.gmane.org> wrote:
Hi there,

I'm evaluating scala at the moment for a specific problem. And I'm
wondering, whether my design ideas are at all reasonable. I have at
least 10000 objects, with an average of 100k each. Is it viable and
smart to create all of there objects as actors? That would probably
make my design simple, but I'm wondering, whether the actors
implementation can handle such a large amount of actors. Where are
reasonable limits?

Any ideas? Is there some material that I should look into?

Thanks,
   Malte



--
http://erikengbrecht.blogspot.com/
Matthias Guenther | 1 Nov 05:02 2010
Picon
Picon

Re: Implementing a security typed variant of Scala - which method is best?


Hello Miguel,

I'm back from my trip to San Francisco and it was the most impressing city I
have ever seen. Just now I read you post, gathered the paper and create some
arguments which method for the implementation is best. Tomorrow I will
discuss the the thing with my advisor. It gives me new motivation, after 4
weeks just reading paper and writing the basics stuff down into my, to see
code manifested as the compiler plugins you mentioned before - I now have
some guide how to do it.

I can post my progress so far and will ask additional questions if I go on.

Matthias 
--

-- 
View this message in context: http://scala-programming-language.1934581.n4.nabble.com/Implementing-a-security-typed-variant-of-Scala-which-method-is-best-tp3005805p3021750.html
Sent from the Scala - User mailing list archive at Nabble.com.

Miguel Garcia | 1 Nov 10:09 2010
Picon

Re: Implementing a security typed variant of Scala - which method is best?


Matthias,

Great to hear back from you. In the meantime I came across another piece of 
information that might be useful.

After posting [1] at the JSR-308 discussion group, I realized that the 
JSR-308 Checker Framework provides a Tainted Checker [2], to track 
information provenance in Java programs and disallow certain actions 
depending on whether it's 'tainted' or not. (The Checker Framework allows 
coding more stringent type-checks than those a Java compiler performs, and 
the Tainted Checker builds upon that functionality to provide a 
security-related type-checks).

*If* there were something like the Checker Framework for Scala, you would 
have an easier time implementing security checks as originally proposed.

Just maybe, it *might be* a useful goal in itself to complete the 
"infrastructure" that the Scala compiler provides (AnnotationChecker [3]) 
with JSR-308-style functionality. A proof-of-concept around security 
checking, as originally planned, or some of the other checker provided by 
JSR-308, would be a welcome addition. Just an idea.

Miguel

[1]
http://groups.google.com/group/jsr308-discuss/browse_thread/thread/cac28dce604f3289

[2]
http://types.cs.washington.edu/checker-framework/current/checkers-manual.html

[3]
http://lampsvn.epfl.ch/trac/scala/browser/scala/trunk/src/compiler/scala/tools/nsc/symtab/AnnotationCheckers.scala

--

-- 
Miguel Garcia
Swiss Federal Institute of Technology
EPFL - IC - LAMP1 - INR 328 - Station 14
CH-1015 Lausanne - Switzerland
http://lamp.epfl.ch/~magarcia/

Matthew Pocock | 1 Nov 14:18 2010
Picon

weirdness with traversibleLike

Hi,

I have some code that compiles from clean but class-casts at run-time.

Exception in thread "main" java.lang.ClassCastException: uk.ac.turingatemyhamster.vw152093.ExtendedNodeList cannot be cast to scala.collection.Traversable
    at scala.collection.generic.TraversableFactory$GenericCanBuildFrom.apply(TraversableFactory.scala:44)
    at scala.collection.TraversableLike$class.map(TraversableLike.scala:204)
    at uk.ac.turingatemyhamster.ExtendedNodeList.map(Scrape.scala:32)

import collection.IndexedSeqLike
import org.w3c.dom.NodeList

case class ExtendedNodeList(nl: NodeList) extends IndexedSeqLike[Node, List[Node]] { // <- line #44
  def length: Int = nl.getLength
  def apply(i: Int) = nl.item(i)
  def newBuilder = List.newBuilder
}

---------------

Triggered in:

  myNodeList map foo

Not triggered in:

  for(n <- myNodeList) foo(n)

Any thoughts? Is this a compiler bug, or something in the libraries, or what?

Matthew

Kenneth McDonald | 1 Nov 17:57 2010
Picon
Picon

Re: Call for better (easier) documentation standards

Tim,

Thanks for the interest. I'll take a look at it if I get a chance, but as I've mentioned, I'm really not a
programmer, I'm a tech writer with an inclination towards programming. I would need to do a lot of coming up
to speed to be able to do that, I suspect.

Basically, I like doing rather small projects, and then giving them the docs they need to be useful. Twenty
years ago, I would've been in there hacking with the best (or at least most enthusiastic), but my tastes
have changed over time.

Cheers,
Ken

On Oct 31, 2010, at 2:56 PM, Tim Perrett wrote:

> Ken,
> 
> Interesting discussion. Have you considered making some kind of SBT
> plugin to assist in this process?  I mean, perhaps a gh-pages
> plugin... it could automate the setup of the steps your originally
> outlined. Just spitballing :-)
> 
> Cheers, Tim
> 
> On 31 October 2010 17:50, Kenneth McDonald
> <kenneth.m.mcdonald@...> wrote:
>> 
>> On Oct 30, 2010, at 4:54 PM, Derek Williams wrote:
>> 
>>> Are you talking about a complete build system / web hosting setup, or
>>> using an existing build system and web host?
>> 
>> The second. Bye the way thanks for all the help!
>>> As far as maven with
>>> github is concerned, the setup I did for you is quite simple. I don't
>>> use maven at all, I use sbt for all my projects, so I am not that
>>> familiar with how maven works but setting you up only took about 10
>>> mins (I did it because I was mostly just curious to see if there was
>>> something out there that would do it). The only problem I had with it
>>> was that it did not automatically fall back to using my own git and
>>> ssh config, requiring a settings.xml file that defined where my ssh
>>> private key was.
>> 
>> This may be minor to you. But it could be major to someone who just wants to write, and document, good Scala
code. I've configured ssh in the past, so at least knew what it was, but ssh, while simple to use once you are
in the know, is not a trivial learning curve. To paraphrase a quote from I don't know where, "Things should
be just simple enough, and no simpler." In my opinion, the successful documenter should need to know the following:
>>        1) The source documentation of the scaladoc (in the build tree).
>>        2) The destination doc of the scaladoc (in the WWW).
>> No further knowledge should be necessary. If it is, it should be prompted for during the installation process.
>> 
>> I can remember the days when I used to love to catch the original knowledge of bits and bytes. (Yep, kids, in
those days, we really were learning bits and bytes. Why, I almost remember the knickname I had for 6502
register...no, no, let's not go there. Bye the way, did you know the 6502 was a simply awesome bit of
computing. If I recall, it was the only Asych logic CPU on the market. Ah, how times move on.) But as people
get older, other things assume more important roles. Wife, family, social life. Time becomes
precious--too precious to be spent learning and relearning bits of hackery.
>>> 
>>> I would like to have a plugin for sbt that would do the same thing,
>>> maybe it is out there, but again it wouldn't be something I'd want to
>>> be there automatically with every project as not every project will be
>>> using git and stored at github, so there would have to be some amount
>>> of setup and config.
>>> 
>>> Or do you mean something like Ruby gems where the code and
>>> documentation are rolled up into one easy-to-install package? The
>>> problem with that is maven repositories seem to be the standard way of
>>> hosting dependencies for the jvm. For my own projects that would mean
>>> any dependency not hosted in a maven friendly way would have to be
>>> manually included.
>> 
>> I think the latter. I note the following of your implementation (nothing to do with you)
>> 
>> http://derekjw.github.com/rex/ does not exactly provide a clear path to the docs. People, docs are
CRUCIAL. They shouldn't be buried a couple of links deep.
>> 
>> I provided a __README__ class whose only function was to serve as the "central" document. Clicking on
this produces an error. Not sure why. Maybe the pages were too large?
>>> 
>>> On Sat, Oct 30, 2010 at 1:24 PM, Kenneth McDonald
>>> <kenneth.m.mcdonald@...> wrote:
>>>> A little bit of background:
>>>> I have a site on github which promises to make regular expressions easier to
>>>> use.
>>>> However, I'm not a programmer, I'm a tech writer by nature.
>>>> After reading about gh-pages, I decided it was too complex for the likes of
>>>> me.
>>>> 
>>>> Derek has very kindly offered the following:
>>>> I forked your repo and added changes to pom.xml to allow you to use
>>>> wagon-gitsite to auto publish. Tested it and the results can be seen
>>>> at: http://derekjw.github.com/rex
>>>> 
>>>> Just make sure you have a settings.xml in your maven directory
>>>> (~/.m2/settings.xml in linux) that looks something like this:
>>>> 
>>>> <settings>
>>>>  <servers>
>>>>    <server>
>>>>      <id>rexsite</id>
>>>>      <username>git</username>
>>>>      <privateKey>/home/derek/.ssh/id_rsa</privateKey>
>>>>    </server>
>>>>  </servers>
>>>> </settings>
>>>> 
>>>> But why should users be required to have settings in their Maven directory?
>>>> Why should they even need to know what a Maven directory is? (I certainly
>>>> didn't want to)
>>>> 
>>>> I've been in the tech writing business for a while now, and it is one of the
>>>> most overlooked ways of getting attention to projects. People will much more
>>>> readily read a well done document that explains the pluses and minuses of
>>>> cases, than they will go to the trouble of dl'ing a distro, compiling it,
>>>> running with all the attendant errors, and finally deploying it. Let's make
>>>> sense, which is more cost effective. A well-reasoned appeal to a highly
>>>> intelligent, but discerning, audience, or a piece of (effectively) raw code.
>>>> So here's the gauntlet I'm throwing down. Come up with a way that
>>>> distributes, IN TANDEM, the documentation that accompanies a piece of code,
>>>> an easy way of installing and reading that code, and one that doesn't
>>>> require complex file setups, absurd config files, etc. etc. Because I
>>>> guarantee you, if you can do that, you'll get a lot more eyeballs.
>>>> Now maybe this already exists and I just haven't seen it. But I've been
>>>> looking for awhile...
>>>> Cheers,
>>>> Ken
>>> 
>>> 
>>> 
>>> --
>>> Derek
>> 
>> Thanks,
>> Ken
>> 
>> 

Philippe Lhoste | 1 Nov 18:23 2010
Picon
Picon

Re: using something like (GNU) Make to build scala projects

For those that might be interested, I managed to compile a small Scala 
project with Gradle, while using only local Scala install and local 
libraries (outside of the project, as I plan to share them among various 
of these micro-projects).

I grouped my thoughts at:
Building and running Scala programs with Gradle
http://phi.lho.free.fr/serendipity/index.php?/archives/27-Building-and-running-Scala-programs-with-Gradle.html
http://bit.ly/dcygA9 for those with e-mail clients not liking wrapped 
URLs...

I suppose my need is quite atypical in modern development environment, 
but well, if somebody else has the same, they might have now a starting 
point. :-)

--

-- 
Philippe Lhoste
--  (near) Paris -- France
--  http://Phi.Lho.free.fr
--  --  --  --  --  --  --  --  --  --  --  --  --  --

Jens Tinz | 1 Nov 18:44 2010
Picon

Re: Some more thoughts on what makes documentation valuable

Hello!

Scala now features package objects. These could be a good location for 
an overview over a module. At the moment, we're internally using a mix 
of comments and scaladoc plus a google doc for high level documentation. 
But I consider changing that so that the documentation is under revision 
control and is in one place with the code.

Cheers,
Jens

Kenneth McDonald | 1 Nov 20:37 2010
Picon
Picon

Re: Some more thoughts on what makes documentation valuable

Cool! Yes, I'd definitely consider changing things so docs under RCS and at one place with the code. Could
you refer me to a ref page? (I'll probably find it by googling, but just in case I don't.)

Thanks,
Ken

On Nov 1, 2010, at 12:44 PM, Jens Tinz wrote:

> Hello!
> 
> Scala now features package objects. These could be a good location for an overview over a module. At the
moment, we're internally using a mix of comments and scaladoc plus a google doc for high level
documentation. But I consider changing that so that the documentation is under revision control and is in
one place with the code.
> 
> 
> Cheers,
> Jens
> 

Naftoli Gugenheim | 1 Nov 21:27 2010
Picon

Re: Call for better (easier) documentation standards

By the way, you know that github has a button to generate a starter github pages page?


On Mon, Nov 1, 2010 at 12:57 PM, Kenneth McDonald <kenneth.m.mcdonald-rphTv4pjVZMJGwgDXS7ZQA@public.gmane.org> wrote:
Tim,

Thanks for the interest. I'll take a look at it if I get a chance, but as I've mentioned, I'm really not a programmer, I'm a tech writer with an inclination towards programming. I would need to do a lot of coming up to speed to be able to do that, I suspect.

Basically, I like doing rather small projects, and then giving them the docs they need to be useful. Twenty years ago, I would've been in there hacking with the best (or at least most enthusiastic), but my tastes have changed over time.

Cheers,
Ken


On Oct 31, 2010, at 2:56 PM, Tim Perrett wrote:

> Ken,
>
> Interesting discussion. Have you considered making some kind of SBT
> plugin to assist in this process?  I mean, perhaps a gh-pages
> plugin... it could automate the setup of the steps your originally
> outlined. Just spitballing :-)
>
> Cheers, Tim
>
> On 31 October 2010 17:50, Kenneth McDonald
> <kenneth.m.mcdonald-rphTv4pjVZMJGwgDXS7ZQA@public.gmane.org> wrote:
>>
>> On Oct 30, 2010, at 4:54 PM, Derek Williams wrote:
>>
>>> Are you talking about a complete build system / web hosting setup, or
>>> using an existing build system and web host?
>>
>> The second. Bye the way thanks for all the help!
>>> As far as maven with
>>> github is concerned, the setup I did for you is quite simple. I don't
>>> use maven at all, I use sbt for all my projects, so I am not that
>>> familiar with how maven works but setting you up only took about 10
>>> mins (I did it because I was mostly just curious to see if there was
>>> something out there that would do it). The only problem I had with it
>>> was that it did not automatically fall back to using my own git and
>>> ssh config, requiring a settings.xml file that defined where my ssh
>>> private key was.
>>
>> This may be minor to you. But it could be major to someone who just wants to write, and document, good Scala code. I've configured ssh in the past, so at least knew what it was, but ssh, while simple to use once you are in the know, is not a trivial learning curve. To paraphrase a quote from I don't know where, "Things should be just simple enough, and no simpler." In my opinion, the successful documenter should need to know the following:
>>        1) The source documentation of the scaladoc (in the build tree).
>>        2) The destination doc of the scaladoc (in the WWW).
>> No further knowledge should be necessary. If it is, it should be prompted for during the installation process.
>>
>> I can remember the days when I used to love to catch the original knowledge of bits and bytes. (Yep, kids, in those days, we really were learning bits and bytes. Why, I almost remember the knickname I had for 6502 register...no, no, let's not go there. Bye the way, did you know the 6502 was a simply awesome bit of computing. If I recall, it was the only Asych logic CPU on the market. Ah, how times move on.) But as people get older, other things assume more important roles. Wife, family, social life. Time becomes precious--too precious to be spent learning and relearning bits of hackery.
>>>
>>> I would like to have a plugin for sbt that would do the same thing,
>>> maybe it is out there, but again it wouldn't be something I'd want to
>>> be there automatically with every project as not every project will be
>>> using git and stored at github, so there would have to be some amount
>>> of setup and config.
>>>
>>> Or do you mean something like Ruby gems where the code and
>>> documentation are rolled up into one easy-to-install package? The
>>> problem with that is maven repositories seem to be the standard way of
>>> hosting dependencies for the jvm. For my own projects that would mean
>>> any dependency not hosted in a maven friendly way would have to be
>>> manually included.
>>
>> I think the latter. I note the following of your implementation (nothing to do with you)
>>
>> http://derekjw.github.com/rex/ does not exactly provide a clear path to the docs. People, docs are CRUCIAL. They shouldn't be buried a couple of links deep.
>>
>> I provided a __README__ class whose only function was to serve as the "central" document. Clicking on this produces an error. Not sure why. Maybe the pages were too large?
>>>
>>> On Sat, Oct 30, 2010 at 1:24 PM, Kenneth McDonald
>>> <kenneth.m.mcdonald-rphTv4pjVZMJGwgDXS7ZQA@public.gmane.org> wrote:
>>>> A little bit of background:
>>>> I have a site on github which promises to make regular expressions easier to
>>>> use.
>>>> However, I'm not a programmer, I'm a tech writer by nature.
>>>> After reading about gh-pages, I decided it was too complex for the likes of
>>>> me.
>>>>
>>>> Derek has very kindly offered the following:
>>>> I forked your repo and added changes to pom.xml to allow you to use
>>>> wagon-gitsite to auto publish. Tested it and the results can be seen
>>>> at: http://derekjw.github.com/rex
>>>>
>>>> Just make sure you have a settings.xml in your maven directory
>>>> (~/.m2/settings.xml in linux) that looks something like this:
>>>>
>>>> <settings>
>>>>  <servers>
>>>>    <server>
>>>>      <id>rexsite</id>
>>>>      <username>git</username>
>>>>      <privateKey>/home/derek/.ssh/id_rsa</privateKey>
>>>>    </server>
>>>>  </servers>
>>>> </settings>
>>>>
>>>> But why should users be required to have settings in their Maven directory?
>>>> Why should they even need to know what a Maven directory is? (I certainly
>>>> didn't want to)
>>>>
>>>> I've been in the tech writing business for a while now, and it is one of the
>>>> most overlooked ways of getting attention to projects. People will much more
>>>> readily read a well done document that explains the pluses and minuses of
>>>> cases, than they will go to the trouble of dl'ing a distro, compiling it,
>>>> running with all the attendant errors, and finally deploying it. Let's make
>>>> sense, which is more cost effective. A well-reasoned appeal to a highly
>>>> intelligent, but discerning, audience, or a piece of (effectively) raw code.
>>>> So here's the gauntlet I'm throwing down. Come up with a way that
>>>> distributes, IN TANDEM, the documentation that accompanies a piece of code,
>>>> an easy way of installing and reading that code, and one that doesn't
>>>> require complex file setups, absurd config files, etc. etc. Because I
>>>> guarantee you, if you can do that, you'll get a lot more eyeballs.
>>>> Now maybe this already exists and I just haven't seen it. But I've been
>>>> looking for awhile...
>>>> Cheers,
>>>> Ken
>>>
>>>
>>>
>>> --
>>> Derek
>>
>> Thanks,
>> Ken
>>
>>



Gmane