Mohan Radhakrishnan | 5 Jan 2011 14:52
Picon

Data structure concurrency question

Hi,

     Ascii representation of UML multiplicity :

 

Logger Name 1---------* Logger 1----------1 Levels

 

 

I have a requirement to associate one logger name with multiple Loggers each of which is in turn associated with a single level. What is a concurrent data structure recommendation for this ?

 

      It is basically a map of maps. The accesses are not going to be highly concurrent but the log messages should not be interleaved.

 

 

Thanks,

Mohan

_______________________________________________
Concurrency-interest mailing list
Concurrency-interest <at> cs.oswego.edu
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Mohan Radhakrishnan | 6 Jan 2011 08:25
Picon

Re: Race condition on singleton

Hi,
     What about this ? Is this a similar singleton ? Since the map is static we haven't anticipated any problem.

Should the method be synchronized ? There could be a problem with a partially created object ? Logically
even if threads get interleaved the map will anyway have the right object.

We run this code with about 10-15 threads with no problem.

	private static Test test = null;

	private static Hashtable<String, Test> logMap = new Hashtable<String, Test >();

	public static Test getInstance( String logName ) {

		if (null == logMap.get(logName)) {

			test = new Test (logName);

			logMap.put(logName, test);

		} else {

			test = logMap.get(logName);

		}

		return test;

	 }

Thanks,
Mohan
-----Original Message-----
From: concurrency-interest-bounces <at> cs.oswego.edu
[mailto:concurrency-interest-bounces <at> cs.oswego.edu] On Behalf Of Marco Villalobos
Sent: Monday, December 13, 2010 10:58 AM
To: concurrency-interest
Subject: Re: [concurrency-interest] Race condition on singleton

By simply reading his code, which I assume is an abridged version of
the original, I didn't notice anything that would cause an NPE.

However, if his original code uses static variables, then order of
those declarations will matter.

For example:

public class WillNPE {

    private final static WillNPE instance = new WillNPE();
    private final static Set<Integer> set = new HashSet<Integer>();

    public static WillNPE getInstance() {
        return instance;
    }

    public WillNPE() {
        set.add(1);
    }

    public static void main(String args[]) {
        WillNPE local = WillNPE.getInstance();
    }
}

On Sun, Dec 12, 2010 at 9:10 PM, Justin T. Sampson <justin <at> krasama.com> wrote:
> I'm surprised with all the replies that no one commented on the lack of
> synchronization on accessing the contents of the map. That can easily cause
> NPEs from within map.put(k, v), or almost any other arbitrary behavior for
> that matter.
> Cheers,
> Justin
>
> On Thu, Dec 9, 2010 at 4:44 AM, Guy Korland <gkorland <at> gmail.com> wrote:
>>
>> Hi,
>> We found a very strange pattern that seems like contradicting the Java
>> Memory Model.
>> It seems like a class that is maintained as singleton doesn't have its
>> constructor fully initialized!
>> See the code example bellow.
>> public class MyClass{
>>   private static final MyClass = new MyClass();
>>
>>   private final HashMap map;
>>   private MyClass(){
>>       map = new HashMap();
>>   }
>>   public void put(Object k, Object v){
>>      map.put(k,v);
>>   }
>>   static public getMyClass(){
>>     return myClass;
>>   }
>> }
>>
>> And when we invoke the following:
>> MyClass.getMyClass().put("a","b");
>> We get a NullPointerException on the "map.put(k,v);", meaning the
>> map==null !?!?
>> Any ideas?
>> Thanks,
>> Guy Korland
>>
>> _______________________________________________
>> Concurrency-interest mailing list
>> Concurrency-interest <at> cs.oswego.edu
>> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>>
>
>
> _______________________________________________
> Concurrency-interest mailing list
> Concurrency-interest <at> cs.oswego.edu
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
>

_______________________________________________
Concurrency-interest mailing list
Concurrency-interest <at> cs.oswego.edu
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Mark Thornton | 6 Jan 2011 09:26
Picon
Favicon

Re: Race condition on singleton

On 06/01/2011 07:25, Mohan Radhakrishnan wrote:
> Hi,
>       What about this ? Is this a similar singleton ? Since the map is static we haven't anticipated any problem.
>
> Should the method be synchronized ? There could be a problem with a partially created object ? Logically
even if threads get interleaved the map will anyway have the right object.
>
> We run this code with about 10-15 threads with no problem.

You got lucky, that code is badly broken. The simplest fix would be to 
synchronize the getInstance method.

Mark Thornton

>
> 	private static Test test = null;
>
> 	private static Hashtable<String, Test>  logMap = new Hashtable<String, Test>();
>
> 	public static Test getInstance( String logName ) {
>
> 		if (null == logMap.get(logName)) {
>
> 			test = new Test (logName);
>
> 			logMap.put(logName, test);
>
>
> 		} else {
>
> 			test = logMap.get(logName);
>
> 		}
>
> 		return test;
>
> 	 }
>
> Thanks,
> Mohan
> -----Original Message-----
> From: concurrency-interest-bounces <at> cs.oswego.edu
[mailto:concurrency-interest-bounces <at> cs.oswego.edu] On Behalf Of Marco Villalobos
> Sent: Monday, December 13, 2010 10:58 AM
> To: concurrency-interest
> Subject: Re: [concurrency-interest] Race condition on singleton
>
> By simply reading his code, which I assume is an abridged version of
> the original, I didn't notice anything that would cause an NPE.
>
> However, if his original code uses static variables, then order of
> those declarations will matter.
>
> For example:
>
> public class WillNPE {
>
>      private final static WillNPE instance = new WillNPE();
>      private final static Set<Integer>  set = new HashSet<Integer>();
>
>      public static WillNPE getInstance() {
>          return instance;
>      }
>
>      public WillNPE() {
>          set.add(1);
>      }
>
>      public static void main(String args[]) {
>          WillNPE local = WillNPE.getInstance();
>      }
> }
>
>
> On Sun, Dec 12, 2010 at 9:10 PM, Justin T. Sampson<justin <at> krasama.com>  wrote:
>> I'm surprised with all the replies that no one commented on the lack of
>> synchronization on accessing the contents of the map. That can easily cause
>> NPEs from within map.put(k, v), or almost any other arbitrary behavior for
>> that matter.
>> Cheers,
>> Justin
>>
>> On Thu, Dec 9, 2010 at 4:44 AM, Guy Korland<gkorland <at> gmail.com>  wrote:
>>> Hi,
>>> We found a very strange pattern that seems like contradicting the Java
>>> Memory Model.
>>> It seems like a class that is maintained as singleton doesn't have its
>>> constructor fully initialized!
>>> See the code example bellow.
>>> public class MyClass{
>>>    private static final MyClass = new MyClass();
>>>
>>>    private final HashMap map;
>>>    private MyClass(){
>>>        map = new HashMap();
>>>    }
>>>    public void put(Object k, Object v){
>>>       map.put(k,v);
>>>    }
>>>    static public getMyClass(){
>>>      return myClass;
>>>    }
>>> }
>>>
>>> And when we invoke the following:
>>> MyClass.getMyClass().put("a","b");
>>> We get a NullPointerException on the "map.put(k,v);", meaning the
>>> map==null !?!?
>>> Any ideas?
>>> Thanks,
>>> Guy Korland
>>>
>>> _______________________________________________
>>> Concurrency-interest mailing list
>>> Concurrency-interest <at> cs.oswego.edu
>>> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>>>
>>
>> _______________________________________________
>> Concurrency-interest mailing list
>> Concurrency-interest <at> cs.oswego.edu
>> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>>
>>
> _______________________________________________
> Concurrency-interest mailing list
> Concurrency-interest <at> cs.oswego.edu
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
> _______________________________________________
> Concurrency-interest mailing list
> Concurrency-interest <at> cs.oswego.edu
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Sébastien Bocq | 7 Jan 2011 16:01
Picon
Gravatar

Concurrent sequential access to a mutable field without the volatile modifier

Hello,

I made a simple test (see below) to verify my assumption that a variable mutated sequentially by multiple threads must be marked as volatile to have mutations visible across these threads. How comes my test succeeds even though I omitted the volatile keyword?

Thanks,
Sébastien

import java.util.concurrent.*;

public class VolatileTest {

    static class CountTask implements Runnable {
        public int count = 0;             // volatile omitted
        public void run() { count++; }
        <at> Override
        public String toString() {return String.valueOf(count);}
    };

    static class SimpleExecutor implements Runnable {

       LinkedBlockingQueue<Runnable> queue = new LinkedBlockingQueue<Runnable>();
      
       public void submit(Runnable runnable) { queue.offer(runnable); }

       volatile Thread thread = null;

       public void run() {

          thread = Thread.currentThread();
         
          while (thread != null) {

            try {
                Runnable task = queue.take();
                task.run();
            } catch (InterruptedException e) {}
          }
       }
   
       public void shutdown() {
         Thread t = thread;
         thread = null;
         t.interrupt();
       }
      
       public SimpleExecutor() {
           new Thread(this).start();
       }
    }
   
    static class LoopTask implements Runnable {
        CountDownLatch latch;
        int count;
        Runnable task;
        SimpleExecutor[] executors;

        /** Repeat a task sequentially using different executors
         *
         * <at> param executors the executors
         * <at> param count     the number of iterations
         * <at> param latch     latch decremented after the last iteration
         * <at> param task      the task invoked repeatedly
         */
        public LoopTask(SimpleExecutor[] executors, int count, CountDownLatch latch, Runnable task) {
            this.executors = executors;
            this.latch = latch;
            this.count = count;
            this.task  = task;
        }

        public void run() {
            if (count == 0)
                latch.countDown();
            else {
                task.run();
                executors[count % executors.length].submit(new LoopTask(executors, count - 1, latch, task));
            }
        }
    }

     public static void main(String[] args) throws Exception {
       int loops = 1000000;
       int nbExecs = 10;
      
       CountDownLatch latch = new CountDownLatch(1);
       SimpleExecutor[] executors = new SimpleExecutor[nbExecs];
      
       for (int i = 0; i < executors.length; i++)
         executors[i] = new SimpleExecutor();
      
       CountTask task = new CountTask();
      
       try {
           new LoopTask(executors, loops, latch, task).run();

           latch.await();

           System.out.println(task);
           assert(task.count == loops); 
       } finally {
           for (int i = 0; i < executors.length; i++)
               executors[i].shutdown();
       }
     }
}

_______________________________________________
Concurrency-interest mailing list
Concurrency-interest <at> cs.oswego.edu
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Mark Thornton | 7 Jan 2011 16:17
Picon
Favicon

Re: Concurrent sequential access to a mutable field without the volatile modifier

On 07/01/2011 15:01, Sébastien Bocq wrote:
> Hello,
>
> I made a simple test (see below) to verify my assumption that a 
> variable mutated sequentially by multiple threads must be marked as 
> volatile to have mutations visible across these threads. How comes my 
> test succeeds even though I omitted the volatile keyword?

The absence of volatile merely means the mutations may not be visible, 
it doesn't guarantee that they won't be visible. So in some 
implementations your mutations may be visible while in others they may 
not. Adding volatile gives you a guarantee of visibility.

Mark
Sébastien Bocq | 7 Jan 2011 16:32
Picon
Gravatar

Re: Concurrent sequential access to a mutable field without the volatile modifier

2011/1/7 Mark Thornton <mthornton <at> optrak.co.uk>

On 07/01/2011 15:01, Sébastien Bocq wrote:
Hello,

I made a simple test (see below) to verify my assumption that a variable mutated sequentially by multiple threads must be marked as volatile to have mutations visible across these threads. How comes my test succeeds even though I omitted the volatile keyword?

The absence of volatile merely means the mutations may not be visible, it doesn't guarantee that they won't be visible. So in some implementations your mutations may be visible while in others they may not. Adding volatile gives you a guarantee of visibility.


Ok, I needed to be sure.

Thanks for the swift answer,
Sébastien
_______________________________________________
Concurrency-interest mailing list
Concurrency-interest <at> cs.oswego.edu
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Holger Hoffstätte | 7 Jan 2011 16:48
Gravatar

Re: Concurrent sequential access to a mutable field without the volatile modifier

On 07.01.2011 16:01, Sébastien Bocq wrote:
> I made a simple test (see below) to verify my assumption that a variable
> mutated sequentially by multiple threads must be marked as volatile to
> have mutations visible across these threads. How comes my test succeeds
> even though I omitted the volatile keyword?

http://en.wikipedia.org/wiki/Memory_ordering
In short: you got lucky.

Holger
Sébastien Bocq | 7 Jan 2011 17:48
Picon
Gravatar

Re: Concurrent sequential access to a mutable field without the volatile modifier

2011/1/7 Holger Hoffstätte <holger.hoffstaette <at> googlemail.com>

On 07.01.2011 16:01, Sébastien Bocq wrote:
> I made a simple test (see below) to verify my assumption that a variable
> mutated sequentially by multiple threads must be marked as volatile to
> have mutations visible across these threads. How comes my test succeeds
> even though I omitted the volatile keyword?

http://en.wikipedia.org/wiki/Memory_ordering
In short: you got lucky.

What worries me is that I get lucky each time I run the test which means I can't rely on regression tests for catching these kinds of mistakes, at least on my current test setup (Intel Core Duo, Windows and Oracle JDK 1.6). I'd feel better if I could get my hands on a test platform less tolerant towards these kinds bugs. Any advice?

Thanks,
Sébastien
_______________________________________________
Concurrency-interest mailing list
Concurrency-interest <at> cs.oswego.edu
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Rémi Forax | 7 Jan 2011 18:04
Picon

Re: Concurrent sequential access to a mutable field without the volatile modifier

On 01/07/2011 05:48 PM, Sébastien Bocq wrote:
2011/1/7 Holger Hoffstätte <holger.hoffstaette <at> googlemail.com>
On 07.01.2011 16:01, Sébastien Bocq wrote:
> I made a simple test (see below) to verify my assumption that a variable
> mutated sequentially by multiple threads must be marked as volatile to
> have mutations visible across these threads. How comes my test succeeds
> even though I omitted the volatile keyword?

http://en.wikipedia.org/wiki/Memory_ordering
In short: you got lucky.

What worries me is that I get lucky each time I run the test which means I can't rely on regression tests for catching these kinds of mistakes, at least on my current test setup (Intel Core Duo, Windows and Oracle JDK 1.6). I'd feel better if I could get my hands on a test platform less tolerant towards these kinds bugs. Any advice?

I love this test:
public class MutablePoint {
  int x;
  int y;
 
  public static void main(String[] args) {
    final MutablePoint point = new MutablePoint();
    for(int i=0; i<2; i++) {
      final int id = i;
      new Thread(new Runnable() {
        <at> Override
        public void run() {
          for(;;) {
            point.x = point.y = id;
            if (point.x != point.y) {
              System.out.println("zorg !");
            }
          }
        }
      }).start();
    }
  }
}

which prints a finite number of "zorg !",
at least if you don't use -Xint.
Also -server usually prints more "zorg !" than -client :)


Thanks,
Sébastien

Rémi
_______________________________________________
Concurrency-interest mailing list
Concurrency-interest <at> cs.oswego.edu
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Sangjin Lee | 7 Jan 2011 18:12
Picon

Re: Concurrent sequential access to a mutable field without the volatile modifier

It's really the handoff using the blocking queue that is providing memory barrier. And the way you wrote, you have essentially a serial execution (using multiple threads).


If you allowed full fledged concurrent access at the counter, you'll find that the test fails. Actually making the count volatile does not make the program correct in this situation. Volatile will not protect you from the data race for the ++ operation. You'll need a strong mechanism than that to get a thread safe concurrent counter, possibly using something like an AtomicInteger.

Sangjin

2011/1/7 Sébastien Bocq <sebastien.bocq <at> gmail.com>
2011/1/7 Holger Hoffstätte <holger.hoffstaette <at> googlemail.com>

On 07.01.2011 16:01, Sébastien Bocq wrote:
> I made a simple test (see below) to verify my assumption that a variable
> mutated sequentially by multiple threads must be marked as volatile to
> have mutations visible across these threads. How comes my test succeeds
> even though I omitted the volatile keyword?

http://en.wikipedia.org/wiki/Memory_ordering
In short: you got lucky.

What worries me is that I get lucky each time I run the test which means I can't rely on regression tests for catching these kinds of mistakes, at least on my current test setup (Intel Core Duo, Windows and Oracle JDK 1.6). I'd feel better if I could get my hands on a test platform less tolerant towards these kinds bugs. Any advice?

Thanks,
Sébastien

_______________________________________________
Concurrency-interest mailing list
Concurrency-interest <at> cs.oswego.edu
http://cs.oswego.edu/mailman/listinfo/concurrency-interest


_______________________________________________
Concurrency-interest mailing list
Concurrency-interest <at> cs.oswego.edu
http://cs.oswego.edu/mailman/listinfo/concurrency-interest

Gmane