headers
Fabian Koch | 1 Sep 2009 08:21
Picon

Re: Reassemble fragmented TCP packets


> The thing to bear in mind is that lwIP provides TCP chuncks to the
> application as soon as it have been received. Any applications must
> be aware of that feature!

... with the RAW-API.
Sockets and netconns behave very much as expected.

regards,
Fabian
_______________________________________________
lwip-users mailing list
lwip-users@...
http://lists.nongnu.org/mailman/listinfo/lwip-users
Permalink | Reply | 1 comment |
headers
Martin Velek | 1 Sep 2009 09:44
Picon

pbuf_realloc in ip.c problem

Hello,

I am using lwip 1.3.1 with FreeRTOS and LM3S9B92. Because there is a
memory manager in the OS I have defined
#define MEM_LIBC_MALLOC                 1
#define MEMP_MEM_MALLOC                 1
#define MEM_USE_POOLS                   0
with bget implementation of memory manager. All incomming packets are PBUF_RAM.

The lwip running fine until a tcp packet to port 80 is received(http
server by netconn). The lwip will cause a invalid memory  access.

During debuging I have discovered that in file ip.c is called
pbuf_realloc(p, iphdr_len) and inside this function is called
mem_realloc(brel). Unfortunately bget reallocate memory by creating a
new space and than will free the old memory.
However this new memory is "lost" because the new pointer is not
stored. (q = mem_realloc(q, (u8_t *)q->payload - (u8_t *)q +
rem_len);) and q is not stored back to "p".
So, p is still pointing to the deallocated memory.
I have thought that the bget implementation is wrong but the standard
C's realloc returns:
"A pointer to the reallocated memory block, which may be either the
same as the ptr argument or a new location.
The type of this pointer is void*, which can be cast to the desired
type of data pointer in order to be dereferenceable.
If the function failed to allocate the requested block of memory, a
NULL pointer is returned."

Have anyone next similar problem?

Thank you for your reply.

Regards,
Martin Velek
Permalink | Reply | 4 comments |
headers
Simon Goldschmidt | 1 Sep 2009 10:39
Picon
Picon

Re: pbuf_realloc in ip.c problem

Which version of lwIP are you using? This was a bug in 1.3.0 that has been fixed in 1.3.1, if I remember
correctly. Try upgrading to the latest version to fix this.

Simon

-------- Original-Nachricht --------
> Datum: Tue, 1 Sep 2009 09:44:09 +0200
> Von: Martin Velek <martin.velek@...>
> An: lwip-users@...
> Betreff: [lwip-users] pbuf_realloc in ip.c problem

> Hello,
> 
> I am using lwip 1.3.1 with FreeRTOS and LM3S9B92. Because there is a
> memory manager in the OS I have defined
> #define MEM_LIBC_MALLOC                 1
> #define MEMP_MEM_MALLOC                 1
> #define MEM_USE_POOLS                   0
> with bget implementation of memory manager. All incomming packets are
> PBUF_RAM.
> 
> The lwip running fine until a tcp packet to port 80 is received(http
> server by netconn). The lwip will cause a invalid memory  access.
> 
> During debuging I have discovered that in file ip.c is called
> pbuf_realloc(p, iphdr_len) and inside this function is called
> mem_realloc(brel). Unfortunately bget reallocate memory by creating a
> new space and than will free the old memory.
> However this new memory is "lost" because the new pointer is not
> stored. (q = mem_realloc(q, (u8_t *)q->payload - (u8_t *)q +
> rem_len);) and q is not stored back to "p".
> So, p is still pointing to the deallocated memory.
> I have thought that the bget implementation is wrong but the standard
> C's realloc returns:
> "A pointer to the reallocated memory block, which may be either the
> same as the ptr argument or a new location.
> The type of this pointer is void*, which can be cast to the desired
> type of data pointer in order to be dereferenceable.
> If the function failed to allocate the requested block of memory, a
> NULL pointer is returned."
> 
> Have anyone next similar problem?
> 
> Thank you for your reply.
> 
> Regards,
> Martin Velek
> 
> 
> _______________________________________________
> lwip-users mailing list
> lwip-users@...
> http://lists.nongnu.org/mailman/listinfo/lwip-users

--

-- 
Jetzt kostenlos herunterladen: Internet Explorer 8 und Mozilla Firefox 3 -
sicherer, schneller und einfacher! http://portal.gmx.net/de/go/chbrowser
Permalink | Reply | 3 comments |
headers
Martin Velek | 1 Sep 2009 10:53
Picon

Re: pbuf_realloc in ip.c problem

Hi,

I am using http://download.savannah.gnu.org/releases-noredirect/lwip/lwip-1.3.1.zip

md5
af8c4bf938e3ca48af3d54e29d57ab54 *ip.c
1a1ec514a862fb1d931f4eb4c236cb21 *pbuf.c

Martin

2009/9/1 Simon Goldschmidt <goldsimon@...>:
> Which version of lwIP are you using? This was a bug in 1.3.0 that has been fixed in 1.3.1, if I remember
correctly. Try upgrading to the latest version to fix this.
>
> Simon
>
>
> -------- Original-Nachricht --------
>> Datum: Tue, 1 Sep 2009 09:44:09 +0200
>> Von: Martin Velek <martin.velek@...>
>> An: lwip-users@...
>> Betreff: [lwip-users] pbuf_realloc in ip.c problem
>
>> Hello,
>>
>> I am using lwip 1.3.1 with FreeRTOS and LM3S9B92. Because there is a
>> memory manager in the OS I have defined
>> #define MEM_LIBC_MALLOC                 1
>> #define MEMP_MEM_MALLOC                 1
>> #define MEM_USE_POOLS                   0
>> with bget implementation of memory manager. All incomming packets are
>> PBUF_RAM.
>>
>> The lwip running fine until a tcp packet to port 80 is received(http
>> server by netconn). The lwip will cause a invalid memory  access.
>>
>> During debuging I have discovered that in file ip.c is called
>> pbuf_realloc(p, iphdr_len) and inside this function is called
>> mem_realloc(brel). Unfortunately bget reallocate memory by creating a
>> new space and than will free the old memory.
>> However this new memory is "lost" because the new pointer is not
>> stored. (q = mem_realloc(q, (u8_t *)q->payload - (u8_t *)q +
>> rem_len);) and q is not stored back to "p".
>> So, p is still pointing to the deallocated memory.
>> I have thought that the bget implementation is wrong but the standard
>> C's realloc returns:
>> "A pointer to the reallocated memory block, which may be either the
>> same as the ptr argument or a new location.
>> The type of this pointer is void*, which can be cast to the desired
>> type of data pointer in order to be dereferenceable.
>> If the function failed to allocate the requested block of memory, a
>> NULL pointer is returned."
>>
>> Have anyone next similar problem?
>>
>> Thank you for your reply.
>>
>> Regards,
>> Martin Velek
>>
>>
>> _______________________________________________
>> lwip-users mailing list
>> lwip-users@...
>> http://lists.nongnu.org/mailman/listinfo/lwip-users
>
> --
> Jetzt kostenlos herunterladen: Internet Explorer 8 und Mozilla Firefox 3 -
> sicherer, schneller und einfacher! http://portal.gmx.net/de/go/chbrowser
>
>
> _______________________________________________
> lwip-users mailing list
> lwip-users@...
> http://lists.nongnu.org/mailman/listinfo/lwip-users
>
Permalink | Reply | 2 comments |
headers
Simon Goldschmidt | 1 Sep 2009 12:40
Picon
Picon

Re: pbuf_realloc in ip.c problem

This was bug 27105, fixed in 1.3.1RC3 (was still included in RC1 and RC2).

Link to the bug:
http://cvs.savannah.gnu.org/viewvc/lwip/src/include/lwip/mem.h?revision=1.23&root=lwip&view=markup

This has nothing to do with ip.c or pbuf.c. Please check your mem.h file against the file in this link (which
is the current and 1.3.1 version):

http://cvs.savannah.gnu.org/viewvc/lwip/src/include/lwip/mem.h?revision=1.23&root=lwip&view=markup

Especially these lines, which fix the bug:

>>>
#ifndef mem_realloc
static void *mem_realloc(void *mem, mem_size_t size)
{
  LWIP_UNUSED_ARG(size);
  return mem;
}
#endif
<<<

they were like this before:

>>>
#ifndef mem_realloc
#define mem_realloc realloc
#endif
<<<

I just checked the download link you gave and memp.h is fixed in the zip. Please re-check the version you are
using is really 1.3.1 stable, not an RC.

You can either upgrade or change the memp.h file manually if upgrading is not an option.

Simon

> Hi,
> 
> I am using
> http://download.savannah.gnu.org/releases-noredirect/lwip/lwip-1.3.1.zip
> 
> md5
> af8c4bf938e3ca48af3d54e29d57ab54 *ip.c
> 1a1ec514a862fb1d931f4eb4c236cb21 *pbuf.c
> 
> Martin
> 
> 2009/9/1 Simon Goldschmidt <goldsimon@...>:
> > Which version of lwIP are you using? This was a bug in 1.3.0 that has
> been fixed in 1.3.1, if I remember correctly. Try upgrading to the latest
> version to fix this.
> >
> > Simon
> >
> >
> > -------- Original-Nachricht --------
> >> Datum: Tue, 1 Sep 2009 09:44:09 +0200
> >> Von: Martin Velek <martin.velek@...>
> >> An: lwip-users@...
> >> Betreff: [lwip-users] pbuf_realloc in ip.c problem
> >
> >> Hello,
> >>
> >> I am using lwip 1.3.1 with FreeRTOS and LM3S9B92. Because there is a
> >> memory manager in the OS I have defined
> >> #define MEM_LIBC_MALLOC                 1
> >> #define MEMP_MEM_MALLOC                 1
> >> #define MEM_USE_POOLS                   0
> >> with bget implementation of memory manager. All incomming packets are
> >> PBUF_RAM.
> >>
> >> The lwip running fine until a tcp packet to port 80 is received(http
> >> server by netconn). The lwip will cause a invalid memory  access.
> >>
> >> During debuging I have discovered that in file ip.c is called
> >> pbuf_realloc(p, iphdr_len) and inside this function is called
> >> mem_realloc(brel). Unfortunately bget reallocate memory by creating a
> >> new space and than will free the old memory.
> >> However this new memory is "lost" because the new pointer is not
> >> stored. (q = mem_realloc(q, (u8_t *)q->payload - (u8_t *)q +
> >> rem_len);) and q is not stored back to "p".
> >> So, p is still pointing to the deallocated memory.
> >> I have thought that the bget implementation is wrong but the standard
> >> C's realloc returns:
> >> "A pointer to the reallocated memory block, which may be either the
> >> same as the ptr argument or a new location.
> >> The type of this pointer is void*, which can be cast to the desired
> >> type of data pointer in order to be dereferenceable.
> >> If the function failed to allocate the requested block of memory, a
> >> NULL pointer is returned."
> >>
> >> Have anyone next similar problem?
> >>
> >> Thank you for your reply.
> >>
> >> Regards,
> >> Martin Velek
> >>
> >>
> >> _______________________________________________
> >> lwip-users mailing list
> >> lwip-users@...
> >> http://lists.nongnu.org/mailman/listinfo/lwip-users
> >
> > --
> > Jetzt kostenlos herunterladen: Internet Explorer 8 und Mozilla Firefox 3
> -
> > sicherer, schneller und einfacher! http://portal.gmx.net/de/go/chbrowser
> >
> >
> > _______________________________________________
> > lwip-users mailing list
> > lwip-users@...
> > http://lists.nongnu.org/mailman/listinfo/lwip-users
> >
> 
> 
> _______________________________________________
> lwip-users mailing list
> lwip-users@...
> http://lists.nongnu.org/mailman/listinfo/lwip-users

--

-- 
GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT!
Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01
Permalink | Reply | 1 comment |
headers
JM | 1 Sep 2009 13:51
Picon
Favicon

Freeing all pbufs on tcp_abort()

What is the proper way to close a connection and free all resources, namely pbufs, associated with it?  In my application, I have to use tcp_abort to end the connection.  (I do indeed have to send a RST; Winamp does the same thing with streaming audio and I'm emulating that). 

I can connect to to a host and receive data fine, but when I end that connection and initiate a different connection, my debug output occasionally indicates I'm out of MEMP_POOL.  However, if I reset the micro and connect to any host, I do not get any out of memory errors.  This tells me one or more pbufs are not being freed when I end a connection.  Is there a way to free all pbufs, or check which ones are being used?  




_______________________________________________
lwip-users mailing list
lwip-users@...
http://lists.nongnu.org/mailman/listinfo/lwip-users
Permalink | Reply | 6 comments |
headers
Martin Velek | 1 Sep 2009 14:06
Picon

Re: pbuf_realloc in ip.c problem

Hi Simon,

thank you for links. There was a misunderstanding. I can see that in
case of MEM_LIBC_MALLOC I should not use my realloc function or do
nothing if defined.

Regards,
Martin

2009/9/1 Simon Goldschmidt <goldsimon@...>:
> This was bug 27105, fixed in 1.3.1RC3 (was still included in RC1 and RC2).
>
> Link to the bug:
> http://cvs.savannah.gnu.org/viewvc/lwip/src/include/lwip/mem.h?revision=1.23&root=lwip&view=markup
>
> This has nothing to do with ip.c or pbuf.c. Please check your mem.h file against the file in this link (which
is the current and 1.3.1 version):
>
> http://cvs.savannah.gnu.org/viewvc/lwip/src/include/lwip/mem.h?revision=1.23&root=lwip&view=markup
>
> Especially these lines, which fix the bug:
>
>>>>
> #ifndef mem_realloc
> static void *mem_realloc(void *mem, mem_size_t size)
> {
>  LWIP_UNUSED_ARG(size);
>  return mem;
> }
> #endif
> <<<
>
> they were like this before:
>
>>>>
> #ifndef mem_realloc
> #define mem_realloc realloc
> #endif
> <<<
>
> I just checked the download link you gave and memp.h is fixed in the zip. Please re-check the version you are
using is really 1.3.1 stable, not an RC.
>
> You can either upgrade or change the memp.h file manually if upgrading is not an option.
>
> Simon
>
>
>
>> Hi,
>>
>> I am using
>> http://download.savannah.gnu.org/releases-noredirect/lwip/lwip-1.3.1.zip
>>
>> md5
>> af8c4bf938e3ca48af3d54e29d57ab54 *ip.c
>> 1a1ec514a862fb1d931f4eb4c236cb21 *pbuf.c
>>
>> Martin
>>
>> 2009/9/1 Simon Goldschmidt <goldsimon@...>:
>> > Which version of lwIP are you using? This was a bug in 1.3.0 that has
>> been fixed in 1.3.1, if I remember correctly. Try upgrading to the latest
>> version to fix this.
>> >
>> > Simon
>> >
>> >
>> > -------- Original-Nachricht --------
>> >> Datum: Tue, 1 Sep 2009 09:44:09 +0200
>> >> Von: Martin Velek <martin.velek@...>
>> >> An: lwip-users@...
>> >> Betreff: [lwip-users] pbuf_realloc in ip.c problem
>> >
>> >> Hello,
>> >>
>> >> I am using lwip 1.3.1 with FreeRTOS and LM3S9B92. Because there is a
>> >> memory manager in the OS I have defined
>> >> #define MEM_LIBC_MALLOC                 1
>> >> #define MEMP_MEM_MALLOC                 1
>> >> #define MEM_USE_POOLS                   0
>> >> with bget implementation of memory manager. All incomming packets are
>> >> PBUF_RAM.
>> >>
>> >> The lwip running fine until a tcp packet to port 80 is received(http
>> >> server by netconn). The lwip will cause a invalid memory  access.
>> >>
>> >> During debuging I have discovered that in file ip.c is called
>> >> pbuf_realloc(p, iphdr_len) and inside this function is called
>> >> mem_realloc(brel). Unfortunately bget reallocate memory by creating a
>> >> new space and than will free the old memory.
>> >> However this new memory is "lost" because the new pointer is not
>> >> stored. (q = mem_realloc(q, (u8_t *)q->payload - (u8_t *)q +
>> >> rem_len);) and q is not stored back to "p".
>> >> So, p is still pointing to the deallocated memory.
>> >> I have thought that the bget implementation is wrong but the standard
>> >> C's realloc returns:
>> >> "A pointer to the reallocated memory block, which may be either the
>> >> same as the ptr argument or a new location.
>> >> The type of this pointer is void*, which can be cast to the desired
>> >> type of data pointer in order to be dereferenceable.
>> >> If the function failed to allocate the requested block of memory, a
>> >> NULL pointer is returned."
>> >>
>> >> Have anyone next similar problem?
>> >>
>> >> Thank you for your reply.
>> >>
>> >> Regards,
>> >> Martin Velek
>> >>
>> >>
>> >> _______________________________________________
>> >> lwip-users mailing list
>> >> lwip-users@...
>> >> http://lists.nongnu.org/mailman/listinfo/lwip-users
>> >
>> > --
>> > Jetzt kostenlos herunterladen: Internet Explorer 8 und Mozilla Firefox 3
>> -
>> > sicherer, schneller und einfacher! http://portal.gmx.net/de/go/chbrowser
>> >
>> >
>> > _______________________________________________
>> > lwip-users mailing list
>> > lwip-users@...
>> > http://lists.nongnu.org/mailman/listinfo/lwip-users
>> >
>>
>>
>> _______________________________________________
>> lwip-users mailing list
>> lwip-users@...
>> http://lists.nongnu.org/mailman/listinfo/lwip-users
>
> --
> GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT!
> Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01
>
>
> _______________________________________________
> lwip-users mailing list
> lwip-users@...
> http://lists.nongnu.org/mailman/listinfo/lwip-users
>
Permalink | Reply | 0 comments |
headers
goldsimon-Mmb7MZpHnFY@public.gmane.org | 1 Sep 2009 17:45
Picon
Picon

Re: Freeing all pbufs on tcp_abort()

JM wrote:
> What is the proper way to close a connection and free all resources, 
> namely pbufs, associated with it?  In my application, I have to use 
> tcp_abort to end the connection.  (I do indeed have to send a RST; 
> Winamp does the same thing with streaming audio and I'm emulating that). 
>
> I can connect to to a host and receive data fine, but when I end that 
> connection and initiate a different connection, my debug output 
> occasionally indicates I'm out of MEMP_POOL.
>
"Out of MEMP_POOL" can mean anything. You need to know exactly which 
pool and see if it is really the pbufs you are running out of. But I'd 
rather guess it's the tcp_pcb-pool: aborted pcbs are put on the 
time-wait list. If you then open a new connection, you will eventually 
run out of tcp_pcbs. However, that's not a fatal warning as the tcp code 
then frees the oldest pcb in time-wait and uses it. You should check the 
stats (or use the prin_stats function) to see which memp pool was emtpy.

Simon
Permalink | Reply | 5 comments |
headers
JM | 1 Sep 2009 17:52
Picon
Favicon

Re: Freeing all pbufs on tcp_abort()

I don't recall the exact error, but I have seen the error indicating it's out of tcp_pcbs and it's not that one.  I do know however that it is complaining about being out of pbufs, and it appears occasionally and randomly while it's receiving data.  The stats also show I exceed pbufs in this circumstance. 

--- On Tue, 9/1/09, goldsimon-Mmb7MZpHnFY@public.gmane.org <goldsimon-Mmb7MZpHnFY@public.gmane.org> wrote:

From: goldsimon-Mmb7MZpHnFY@public.gmane.org <goldsimon-Mmb7MZpHnFY@public.gmane.org>
Subject: Re: [lwip-users] Freeing all pbufs on tcp_abort()
To: "Mailing list for lwIP users" <lwip-users-qX2TKyscuCcdnm+yROfE0A@public.gmane.org>
Date: Tuesday, September 1, 2009, 11:45 AM

JM wrote:
> What is the proper way to close a connection and free all resources, namely pbufs, associated with it?  In my application, I have to use tcp_abort to end the connection.  (I do indeed have to send a RST; Winamp does the same thing with streaming audio and I'm emulating that).
> I can connect to to a host and receive data fine, but when I end that connection and initiate a different connection, my debug output occasionally indicates I'm out of MEMP_POOL.
>
"Out of MEMP_POOL" can mean anything. You need to know exactly which pool and see if it is really the pbufs you are running out of. But I'd rather guess it's the tcp_pcb-pool: aborted pcbs are put on the time-wait list. If you then open a new connection, you will eventually run out of tcp_pcbs. However, that's not a fatal warning as the tcp code then frees the oldest pcb in time-wait and uses it. You should check the stats (or use the prin_stats function) to see which memp pool was emtpy.

Simon


_______________________________________________
lwip-users mailing list
lwip-users-qX2TKyscuCcdnm+yROfE0A@public.gmane.org
http://lists.nongnu.org/mailman/listinfo/lwip-users

_______________________________________________
lwip-users mailing list
lwip-users@...
http://lists.nongnu.org/mailman/listinfo/lwip-users
Permalink | Reply | 4 comments |
headers
Mathieu Schneider | 2 Sep 2009 01:17

LwIP Data Stream Memory Leak

I have posted this to the Luminarymicro.com forum as well.



LwIP Data Stream Memory Leak

I have created an application based on enet_ptpd for the LM3S8962.

My application is different in that it is not a web server but rather streams data from the embedded device to a PC. The data is streamed over TCP/IP. The network is simply the a cable between to static IP address.

I am having a memory leak problem. I identified the problem when after some time the embedded device no longer can allocate pbuf memory to send another packet to the PC. I enabled lwip_stats and can see that every thing is fine for 15 minutes or so and then suddenly the lwip_stats.mem.used starts to slowly creep up until finally after another 15 minutes or so it finally runs out of available memory.

I have only found 2 items that appear to affect the memory leak rate.

1) if the device streams at 5 packets per second the above memory leak occurs. If the same size packet is sent 50 times per second it can take over 12 hours before the leak becomes present. 

2) if MEMP_SANITY_CHECK is enabled this can also improve the system for maybe around 4-5 hours before failure.

Has anyone seen a problem like this before? Any ideas where I should look?


--
Regards,

Mathieu Schneider
Firmware Developer
______________________________________________________
Pacific Design Engineering,
8505 Eastlake Drive, Burnaby, BC, CANADA, V5A 4T7
Phone: 1 (604) 421-1311 Ext: 39
Toll free: 1 (800) 561-3322
Fax: 1 (604) 421-9202
______________________________________________________
_______________________________________________
lwip-users mailing list
lwip-users@...
http://lists.nongnu.org/mailman/listinfo/lwip-users
Permalink | Reply | 0 comments |

Gmane