Greg Young | 26 May 17:36 2016
Picon
Gravatar

TCP connects

In looking through a recent issue, when a node disappears we end up with:

"Threadpool worker" at <unknown> <0xffffffff>
at (wrapper managed-to-native)
System.Net.Sockets.Socket.Connect_internal
(intptr,System.Net.SocketAddress,int&) <0xffffffff>
at System.Net.Sockets.Socket.Connect (System.Net.EndPoint) <0x00135>
at System.Net.WebConnection.Connect (System.Net.HttpWebRequest) <0x00615>
at System.Net.WebConnection.InitConnection (object) <0x0031a>
at System.Net.WebConnection.<WebConnection>m__0 (object) <0x00024>
at (wrapper runtime-invoke)
<Module>.runtime_invoke_void__this___object
(object,intptr,intptr,intptr) <0xffffffff>

"Threadpool worker" at <unknown> <0xffffffff>
at (wrapper managed-to-native)
System.Net.Sockets.Socket.Connect_internal
(intptr,System.Net.SocketAddress,int&) <0xffffffff>
at System.Net.Sockets.Socket.Connect (System.Net.EndPoint) <0x00135>
at System.Net.WebConnection.Connect (System.Net.HttpWebRequest) <0x00615>
at System.Net.WebConnection.InitConnection (object) <0x0031a>
at System.Net.WebConnection.<WebConnection>m__0 (object) <0x00024>
at (wrapper runtime-invoke)
<Module>.runtime_invoke_void__this___object
(object,intptr,intptr,intptr) <0xffffffff>

On every thread from our async code. As the connection is timing out
the entire ThreadPool very quickly blocks.

https://github.com/mono/mono/blob/master/mcs/class/System/System.Net/WebConnection.cs#L195
(Continue reading)

Pascal Haakmat | 26 May 09:27 2016
Picon

Patches for LINQ to SQL?

Dear list,

I am porting a project that makes use of WCF and LINQ to SQL to Mono.
So far so good. Most of the stuff works or can be made to work. Many
thanks for making that happen!

At this point there are a few LINQ to SQL issues and it seems most
productive to solve them in LINQ to SQL rather than the application
code. In particular I am considering some improvements to sqlmetal.exe
(e.g. --member-attributes does not seem to work, some other stuff) and
adding a few string.* QueryMethods[1].

But since LINQ to SQL is no longer supported and DbLinq is dead, I
wonder if there is any chance of patches being folded back into
Mono. Please advise.

Thanks,
Pascal

[1] See AnalyzeStringCall in https://github.com/mono/mono/blob/master/mcs/class/System.Data.Linq/src/DbLinq/Data/Linq/Sugar/Implementation/ExpressionDispatcher.Analyzer.cs
Chris Swiedler | 25 May 20:29 2016

High threadpool CPU usage

We have a server app which is periodically going into a mode where all threadpool threads start running at
very high CPU. I've run pstack when it's in this mode, and every time I do it, nearly all the threadpool
threads have this stack:

#0  pthread_cond_timedwait <at>  <at> GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238
#1  0x0000000000618817 in mono_cond_timedwait_ms (cond=cond <at> entry=0x7fe7ee1fddc0,
mutex=0x241eb78, timeout_ms=<optimized out>) at mono-mutex.c:181
#2  0x0000000000586f28 in worker_park () at threadpool-ms.c:509
#3  worker_thread (data=<optimized out>) at threadpool-ms.c:607
#4  0x00000000005841e9 in start_wrapper_internal (data=<optimized out>) at threads.c:725
#5  start_wrapper (data=<optimized out>) at threads.c:772
#6  0x0000000000621026 in inner_start_thread (arg=0x7fe831df4650) at mono-threads-posix.c:97
#7  0x00007fe88a55edf5 in start_thread (arg=0x7fe7ee1fe700) at pthread_create.c:308
#8  0x00007fe88a28c1ad in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113

Usually one thread will have a stack like this:

#0  sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
#1  0x000000000061aa37 in mono_sem_wait (sem=0x9542c0 <suspend_ack_semaphore>,
alertable=alertable <at> entry=0) at mono-semaphore.c:107
#2  0x00000000005c77bd in sgen_wait_for_suspend_ack (count=count <at> entry=18) at sgen-os-posix.c:188
#3  0x00000000005c78f9 in sgen_thread_handshake (suspend=suspend <at> entry=1) at sgen-os-posix.c:224
#4  0x00000000005c7e92 in sgen_client_stop_world (generation=generation <at> entry=0) at sgen-stw.c:234
#5  0x00000000005e6aca in sgen_stop_world (generation=0) at sgen-gc.c:3389
#6  0x00000000005e6c29 in sgen_perform_collection (requested_size=4096, generation_to_collect=0,
reason=0x6d9595 "Nursery full", wait_to_finish=0) at sgen-gc.c:2322#7  0x00000000005da62a in
sgen_alloc_obj_nolock (vtable=vtable <at> entry=0x7fe85c0343c0, size=size <at> entry=128) at sgen-alloc.c:291
#8  0x00000000005da913 in sgen_alloc_obj (vtable=vtable <at> entry=0x7fe85c0343c0, size=128) at sgen-alloc.c:457
#9  0x00000000005c86e9 in mono_gc_alloc_obj (vtable=vtable <at> entry=0x7fe85c0343c0, size=<optimized
out>) at sgen-mono.c:936
(Continue reading)

Chris Swiedler | 23 May 23:53 2016

Reading threadpool performance counters

Per https://bugzilla.xamarin.com/show_bug.cgi?id=1772 I've been trying to get Mono threadpool
metrics with lines like this:

    m_workItemsAdded = new PerformanceCounter("Mono Threadpool", "Work Items Added", Process.GetCurrentProcess().Id.ToString());

However, the counter I get back always has its value set to zero. Are these threadpool counters still
supported? Am I doing something wrong? This is using mono 4.2.3.

Thanks,
chris
Burkhard Linke | 19 May 16:30 2016
Picon
Picon

Re: FW: Random hangs while running mono app

Hi,

On 04/29/2016 04:12 PM, Rodrigo Kumpera wrote:
> This looks like a shutdown bug in mono.
>
> Do you have a reliable way to reproduce it?
> How loaded are the machines running your workload?

We have encountered the same(?) bug on our compute cluster. Applications 
process data, write output files, but do not terminate.

(gdb) info threads
   Id   Target Id         Frame
   6    Thread 0x2b1f83200700 (LWP 63141) "mono" 
pthread_cond_wait <at>  <at> GLIBC_2.3.2 () at 
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
   5    Thread 0x2b1f84cf3700 (LWP 63142) "Finalizer" sem_wait () at 
../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
   4    Thread 0x2b1f87ee1700 (LWP 63143) "mono" 
pthread_cond_timedwait <at>  <at> GLIBC_2.3.2 () at 
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238
   3    Thread 0x2b1f8c81d700 (LWP 63148) "Timer-Scheduler" 
pthread_cond_wait <at>  <at> GLIBC_2.3.2 () at 
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
   2    Thread 0x2b1fe1133700 (LWP 63248) "mono" 
pthread_cond_wait <at>  <at> GLIBC_2.3.2 () at 
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
* 1    Thread 0x2b1f81c98580 (LWP 63140) "mono" 
pthread_cond_wait <at>  <at> GLIBC_2.3.2 () at 
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
(Continue reading)

Zhanxing Ding | 12 May 11:47 2016

Parameter on the stack overlap

Hi all,

 

Now I am porting mono to aix,

I met a problem that is,

When convert from decimal to flaot, I found the parameter d of the mono_decimal_to_float (d=...)

function overlapped, so I can’t get the correct content of the parameter.

 

Prologue of the function:

 

mono_decimal_to_float (d=...) at decimal-ms.c:2369(which is an Internal Call)

2369    {

=> 0x1021925c <mono_decimal_to_float+0>:        7c 08 02 a6     mflr    r0

   0x10219260 <mono_decimal_to_float+4>:        90 01 00 08     stw     r0,8(r1)

   0x10219264 <mono_decimal_to_float+8>:        93 e1 ff fc     stw     r31,-4(r1)

   0x10219268 <mono_decimal_to_float+12>:       94 21 ff a0     stwu    r1,-96(r1)

   0x1021926c <mono_decimal_to_float+16>:       7c 3f 0b 78     mr      r31,r1

   0x10219270 <mono_decimal_to_float+20>:       90 7f 00 78     stw     r3,120(r31)

   0x10219274 <mono_decimal_to_float+24>:       90 9f 00 7c     stw     r4,124(r31)

   0x10219278 <mono_decimal_to_float+28>:       90 bf 00 80     stw     r5,128(r31)

   0x1021927c <mono_decimal_to_float+32>:       90 df 00 84     stw     r6,132(r31)

 

Parameter d is the MonoDecimal structure,

The address is 0x2ff22700, but when run the instruction stw     r3,120(r31), one filed of  0x2ff22700  populated with

The value 0x2ff22700.

 

(gdb) info reg r31

r31            0x2ff22690       804398736

 

Anyone know why this happened?

Why it save the parameter where will overlap the value?

 

Thank you in advance!

================================
Rocket Software, Inc. and subsidiaries ■ 77 Fourth Avenue, Waltham MA 02451 ■ +1 877.328.2932 ■ +1 781.577.4321
Unsubscribe From Commercial Email – unsubscribe <at> rocketsoftware.com
Manage Your Subscription Preferences - http://info.rocketsoftware.com/GlobalSubscriptionManagementEmailFooter_SubscriptionCenter.html
Privacy Policy - http://www.rocketsoftware.com/company/legal/privacy-policy
================================

This communication and any attachments may contain confidential information of Rocket Software, Inc. All unauthorized use, disclosure or distribution is prohibited. If you are not the intended recipient, please notify Rocket Software immediately and destroy all copies of this communication. Thank you.

_______________________________________________
Mono-devel-list mailing list
Mono-devel-list <at> lists.ximian.com
http://lists.ximian.com/mailman/listinfo/mono-devel-list
Gary Briggs | 12 May 06:52 2016

Diagnosing "Invalid IL code"

I have some code that, when compiled with monodevelop [latest in Ubuntu
15.10], throws an invalid IL code exception when run with mono. This same
code, when compiled with VS2015, does not throw said exception when run
with mono.

Of course, the kicker is that this is VB.NET code at my place of work;
I can't provide binaries or source, and I've been unable to create a
minimal test case because the whole thing clocks in around 20k SLOC.

So... how do I diagnose what's going on? I want to provide a decent bug
report, and obviously "VB.NET code that I can't show you, when compiled
with mono, has invalid bytecode" doesn't cut it.

And advice as to how I can proceed would be appreciated.

Thanks,
Gary
Alexander Köplinger | 11 May 16:21 2016
Gravatar

Re: Build error

It's not fixed in master yet, I sent a tentative fix with https://github.com/mono/mono/pull/2986, if you could try that one it'd be awesome.

- Alex

2016-05-11 16:15 GMT+02:00 Neale Ferguson <neale <at> sinenomine.net>:
Ah, great. Thanks!

Does that mean you’re getting through to both jenkins servers now without error?

Neale

From: Alexander Köplinger <alexander.koeplinger <at> xamarin.com>
Date: Wednesday, May 11, 2016 at 8:23 AM
To: Neale Ferguson <neale <at> sinenomine.net>
Cc: Mono-Devel <mono-devel-list <at> lists.ximian.com>
Subject: Re: [Mono-dev] Build error

Btw. we're seeing the same issue on the s390x builders on jenkins. We're bootstrapping with Mono 3.10 there so maybe it's an issue with mcs in that version. I'll talk to Marek.

- Alex

2016-05-11 3:34 GMTכֿ:00 Neale Ferguson <neale <at> sinenomine.net>:
Did the git clean –xdf and rebuilt. Same error. I’ll see if I can clone a copy and try again.


Oid was recently imported from the MS referencesources: https://github.com/mono/mono/commit/4a6e5d24e174b9d2860300503b4a79396d7ae96d

I don't see why that would happen for you, it is building fine on our bots. Maybe try a "git clean -xdf" to ensure a clean repo?



_______________________________________________
Mono-devel-list mailing list
Mono-devel-list <at> lists.ximian.com
http://lists.ximian.com/mailman/listinfo/mono-devel-list
Alexander Köplinger | 11 May 14:23 2016
Gravatar

Re: Build error

Btw. we're seeing the same issue on the s390x builders on jenkins. We're bootstrapping with Mono 3.10 there so maybe it's an issue with mcs in that version. I'll talk to Marek.

- Alex

2016-05-11 3:34 GMT+02:00 Neale Ferguson <neale <at> sinenomine.net>:
Did the git clean –xdf and rebuilt. Same error. I’ll see if I can clone a copy and try again.


Oid was recently imported from the MS referencesources: https://github.com/mono/mono/commit/4a6e5d24e174b9d2860300503b4a79396d7ae96d

I don't see why that would happen for you, it is building fine on our bots. Maybe try a "git clean -xdf" to ensure a clean repo?


_______________________________________________
Mono-devel-list mailing list
Mono-devel-list <at> lists.ximian.com
http://lists.ximian.com/mailman/listinfo/mono-devel-list
Neale Ferguson | 10 May 16:56 2016
Picon

Build error

Over the last couple of days days I have been encountering an error
building from master. I did a make clean but it persists:

System.Net.Sockets/SocketAsyncEventArgs.cs(203,34): warning CS0420:
`System.Net.Sockets.SocketAsyncEventArgs.in_progress': A volatile field
references will not be treated as volatile
System.Security.Cryptography.X509Certificates/X509EnhancedKeyUsageExtension
.cs(71,20): error CS0135: `oid' conflicts with a declaration in a child
block
System.Security.Cryptography.X509Certificates/X509EnhancedKeyUsageExtension
.cs(74,17): (Location of the symbol related to previous error)
Compilation failed: 1 error(s), 38 warnings

I have no local changes in this area.

Neale
Zinkevicius, Matt | 4 May 04:48 2016

Re: Using valgrind with Mono

To close on this, most of the valgrind leaks reported have now been plugged in the Mono 4.4 branch. Unfortunately, these leaks turned out to be very small portion of the overall leak we’re experiencing.

 

It turns out that Mono is leaking almost 100 bytes every time a System.Reflection.Emit.DynamicMethod is created, or any method that uses DynamicMethod (Like Expression.DynamicInvoke). Dynamic methods are used at a high-frequency in libraries like NHibernate, and so our service runs out of memory in short order.

 

https://bugzilla.xamarin.com/show_bug.cgi?id=40691

 

I would be very grateful for any assistance fixing this issue, and am willing to provide any additional info required.

 

--Matt

 

From: mono-devel-list-bounces <at> lists.ximian.com [mailto:mono-devel-list-bounces <at> lists.ximian.com] On Behalf Of Zinkevicius, Matt
Sent: Wednesday, March 30, 2016 2:06 AM
To: Rodrigo Kumpera <kumpera <at> gmail.com>
Cc: Straw, David (Storage) <david.straw <at> hpe.com>; mono-devel-list <at> lists.ximian.com
Subject: Re: [Mono-dev] Using valgrind with Mono

 

I have backported the following fixes from master into 4.2.3: https://github.com/mono/mono/commit/b97ac0023256bf7d915552f5f24a7742b28c32b7

https://github.com/mono/mono/commit/8c52b398c5eb962bba5985e8bc01445ac5f027a5

https://github.com/mono/mono/pull/2781

https://github.com/mono/mono/pull/2783

https://github.com/mono/mono/pull/2785

 

This has helped tremendously. I am now down to 659 leak occurrences, of which 640 have one of the following signatures:

 

1) "mono_metadata_type_dup" x 327 occurrences x 12-36 bytes each

 

==31699== 36 bytes in 3 blocks are definitely lost in loss record 7,355 of 13,872

==31699==    at 0x4C2828A: malloc (vg_replace_malloc.c:299)

==31699==    by 0x62D1E1: monoeg_malloc (in /usr/bin/mono-sgen)

==31699==    by 0x55B97F: mono_metadata_type_dup (in /usr/bin/mono-sgen)

==31699==    by 0x49FD0B: get_shared_gparam (in /usr/bin/mono-sgen)

==31699==    by 0x49FF30: get_shared_inst (in /usr/bin/mono-sgen)

==31699==    by 0x4A07FA: mini_get_shared_method_full (in /usr/bin/mono-sgen)

==31699==    by 0x414723: lookup_method (in /usr/bin/mono-sgen)

==31699==    by 0x4147FA: mono_jit_compile_method_with_opt (in /usr/bin/mono-sgen)

==31699==    by 0x414B9A: mono_jit_compile_method (in /usr/bin/mono-sgen)

==31699==    by 0x498E64: common_call_trampoline_inner (in /usr/bin/mono-sgen)

 

2) "mono_method_get_header" x 313 occurrences x 32-192 bytes each

Note: PR 2781 brought this down from 5800 occurrences.

Would https://github.com/mono/mono/pull/2705 help potentially?

 

==31699== 32 bytes in 1 blocks are definitely lost in loss record 7,047 of 13,872

==31699==    at 0x4C2828A: malloc (vg_replace_malloc.c:299)

==31699==    by 0x62D1E1: monoeg_malloc (in /usr/bin/mono-sgen)

==31699==    by 0x62D237: monoeg_g_memdup (in /usr/bin/mono-sgen)

==31699==    by 0x53CB47: mono_method_get_header (in /usr/bin/mono-sgen)

==31699==    by 0x4F8EA0: mini_method_compile (in /usr/bin/mono-sgen)

==31699==    by 0x4FA788: mono_jit_compile_method_inner (in /usr/bin/mono-sgen)

==31699==    by 0x414A01: mono_jit_compile_method_with_opt (in /usr/bin/mono-sgen)

==31699==    by 0x414B9A: mono_jit_compile_method (in /usr/bin/mono-sgen)

==31699==    by 0x498E64: common_call_trampoline_inner (in /usr/bin/mono-sgen)

or

==31699== 192 bytes in 3 blocks are definitely lost in loss record 11,517 of 13,872

==31699==    at 0x4C2828A: malloc (vg_replace_malloc.c:299)

==31699==    by 0x62D1E1: monoeg_malloc (in /usr/bin/mono-sgen)

==31699==    by 0x62D237: monoeg_g_memdup (in /usr/bin/mono-sgen)

==31699==    by 0x53CB47: mono_method_get_header (in /usr/bin/mono-sgen)

==31699==    by 0x430691: mono_method_to_ir (in /usr/bin/mono-sgen)

==31699==    by 0x4F94F5: mini_method_compile (in /usr/bin/mono-sgen)

==31699==    by 0x4FA788: mono_jit_compile_method_inner (in /usr/bin/mono-sgen)

==31699==    by 0x414A01: mono_jit_compile_method_with_opt (in /usr/bin/mono-sgen)

==31699==    by 0x414B9A: mono_jit_compile_method (in /usr/bin/mono-sgen)

==31699==    by 0x498E64: common_call_trampoline_inner (in /usr/bin/mono-sgen)

 

Getting close! Thanks again for any help anyone can provide,

Matt

 

From: mono-devel-list-bounces <at> lists.ximian.com [mailto:mono-devel-list-bounces <at> lists.ximian.com] On Behalf Of Zinkevicius, Matt
Sent: Tuesday, March 29, 2016 8:26 PM
To: Rodrigo Kumpera <kumpera <at> gmail.com>
Cc: Straw, David (Storage) <david.straw <at> hpe.com>; mono-devel-list <at> lists.ximian.com
Subject: Re: [Mono-dev] Using valgrind with Mono

 

Unfortunately, PR 2783 did not have any noticeable effect. We still see thousands of leaks like the following:

 

==12142== 89,860 (89,704 direct, 156 indirect) bytes in 2,800 blocks are definitely lost in loss record 19,763 of 19,792

==12142==    at 0x4C26FEF: calloc (vg_replace_malloc.c:711)

==12142==    by 0x62D269: monoeg_malloc0 (in /usr/bin/mono-sgen)

==12142==    by 0x53CA32: mono_method_get_header (in /usr/bin/mono-sgen)

==12142==    by 0x56CCEA: mono_basic_block_split (in /usr/bin/mono-sgen)

==12142==    by 0x4323B3: mono_method_to_ir (in /usr/bin/mono-sgen)

==12142==    by 0x45FC8B: inline_method (in /usr/bin/mono-sgen)

==12142==    by 0x44C2F4: mono_method_to_ir (in /usr/bin/mono-sgen)

==12142==    by 0x4F94A5: mini_method_compile (in /usr/bin/mono-sgen)

==12142==    by 0x4FA738: mono_jit_compile_method_inner (in /usr/bin/mono-sgen)

==12142==    by 0x414A01: mono_jit_compile_method_with_opt (in /usr/bin/mono-sgen)

==12142==    by 0x414B9A: mono_jit_compile_method (in /usr/bin/mono-sgen)

==12142==    by 0x498E44: common_call_trampoline_inner (in /usr/bin/mono-sgen)

 

Looks like https://github.com/mono/mono/pull/2781/ may address this leak? I’ll attempt to backport it and report back.

 

Matt

 

From: mono-devel-list-bounces <at> lists.ximian.com [mailto:mono-devel-list-bounces <at> lists.ximian.com] On Behalf Of Zinkevicius, Matt
Sent: Tuesday, March 29, 2016 7:22 PM
To: Rodrigo Kumpera <
kumpera <at> gmail.com>
Cc: Straw, David (Storage) <
david.straw <at> hpe.com>; mono-devel-list <at> lists.ximian.com
Subject: Re: [Mono-dev] Using valgrind with Mono

 

Thanks, Rodrigo!

 

I’ve ported this to 4.2 to test, though these changes seem to only address AOT, and we’re seeing this leak using the normal JIT runtime.

 

Matt

 

From: Rodrigo Kumpera [mailto:kumpera <at> gmail.com]
Sent: Tuesday, March 29, 2016 6:15 PM
To: Zinkevicius, Matt <
matt.zinkevicius <at> hpe.com>
Cc: Straw, David (Storage) <
david.straw <at> hpe.com>; mono-devel-list <at> lists.ximian.com
Subject: Re: [Mono-dev] Using valgrind with Mono

 

This is the PR in question: https://github.com/mono/mono/pull/2783

 

It probably won't make into 4.2, but should definitely be in 4.4.

 

_______________________________________________
Mono-devel-list mailing list
Mono-devel-list <at> lists.ximian.com
http://lists.ximian.com/mailman/listinfo/mono-devel-list

Gmane