headers
Matt Wilmas | 1 May 2009 02:55

Re: [PHP-DEV] Re: [PATCH] Scanner "diet" with fixes, etc.

Hi Dmitry,

----- Original Message -----
From: "Dmitry Stogov"
Sent: Thursday, April 30, 2009

> Hi Matt,
>
> Does this patch fix EOF handling issues related to mmap()? (e.g. parsing 
> of files with size 4096, 8192, ...). Now we have two dirty fixes to handle 
> them correctly.

I'm not really sure about the mmap() stuff, and haven't followed the 
workarounds that you or Brian have done...  But as far as scanning 
strings/comments, there shouldn't be an EOF problem with them after my 
changes.  Those were the ones that could contain NULL and got the EOF 
NULL(s) confused and scan too far.  I guess that's the ZEND_MMAP_AHEAD 
padding...?  So I don't know if reverting the "dirty fixes" would then still 
cause any problems with re2c scanning other tokens (that can't contain 
NULL).

Brian would know more I guess, sooner than I could look into what to revert, 
when I don't fully understand that part.

> The patch is quite big to understand it quickly. I'll probably take a look 
> on weekend.

OK.  I think the patch looks more complicated and is larger because of so 
many removed lines. :-)
(Continue reading)

Permalink | Reply | 0 comments |
headers
Matt Wilmas | 1 May 2009 03:00

Re: [PHP-DEV] [PATCH] Scanner "diet" with fixes, etc.

Hi guys,

----- Original Message -----
From: "Nuno Lopes"
Sent: Thursday, April 30, 2009

>>> The patch looks generally ok. However I'll need a few more days to 
>>> review it carefully and throughly. (you can merge it in the meantime  if 
>>> you want).
>>> I'm just slighty concern with the amount of parsing we are now doing  by 
>>> hand, and with the possible (local) security bugs we might be 
>>> introducing..
>>
>>
>> Am I understanding this properly, that this addresses the re2c EOF  bug? 
>> So we have an RC planned for next week (freeze Monday evening).  Can you 
>> get this fixed and released by then as Marcus is unable to do  this 
>> himself?
>
> So this addresses some of the re2c EOF problems, but I don't know if it 
> addresses all of them or not. I haven't had the time yet for a full 
> review.
> Anyway, Matt can surelly comment on this.

Yes, it addresses the re2c EOF issues for strings and comments, as they were 
the problem ones that allowed NULL bytes, and scanned past the EOF NULL.  As 
I said to Dmitry, I'm not sure if it's now possible to remove the temporary 
mmap() fixes that he wanted removed before the next RC (??), or if there 
would still be problems with re2c scanning other tokens, even though they 
can't contain NULLs.  I didn't attempt to make any changes there, since I'm
(Continue reading)

Permalink | Reply | 12 comments |
headers
Jani Taskinen | 1 May 2009 19:52
Picon

Re: [PHP-DEV] rfc1867.c question

Andrei Zmievski kirjoitti:
> Jani Taskinen wrote:
>> What branch and what lines? I checked PHP_5_2 and as far as I can 
>> tell, there is no such function that simply returns when passed 
>> track_vars_array is NULL..
>>
>> Maybe HEAD is different, but that branch doesn't even compile so I 
>> don't really have time for it.. :)
> 
> I was talking about HEAD. And it compiles just fine.

Ah..and my build problems were caused by ext/intl..disabling that made the build 
work.

--Jani

--

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Permalink | Reply | 0 comments |
headers
Lukas Kahwe Smith | 3 May 2009 22:22

Re: [PHP-DEV] PHP 5.3.0RC2


On 28.04.2009, at 12:51, Lukas Kahwe Smith wrote:

> So seriously .. Thursday next week we will release RC2. This means a  
> commit freeze for all but build fixes and README commits starting  
> Monday evening. Also with RC2 any feature additions, regardless of  
> how small, will no longer be allowed (when in doubt if a patch is a  
> feature addition or a bug fix, ask one of the RMs). We are looking  
> to really wrap things up quickly now. So if necessary expect an RC3  
> within 2 weeks after RC2. And we will continue with this cycle until  
> we have a final release.

Ok, so what is missing before we can make a commit freeze starting  
Tuesday morning?

regards,
Lukas Kahwe Smith
mls <at> pooteeweet.org

--

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Permalink | Reply | 3 comments |
headers
Pierre Joye | 4 May 2009 00:59
Picon
Gravatar

Re: [PHP-DEV] PHP 5.3.0RC2

hi Lukas,

The re3c bug (8k bug) for one.

Cheers,

On Sun, May 3, 2009 at 10:22 PM, Lukas Kahwe Smith <mls <at> pooteeweet.org> wrote:
>
> On 28.04.2009, at 12:51, Lukas Kahwe Smith wrote:
>
>> So seriously .. Thursday next week we will release RC2. This means a
>> commit freeze for all but build fixes and README commits starting Monday
>> evening. Also with RC2 any feature additions, regardless of how small, will
>> no longer be allowed (when in doubt if a patch is a feature addition or a
>> bug fix, ask one of the RMs). We are looking to really wrap things up
>> quickly now. So if necessary expect an RC3 within 2 weeks after RC2. And we
>> will continue with this cycle until we have a final release.
>
>
> Ok, so what is missing before we can make a commit freeze starting Tuesday
> morning?
>
> regards,
> Lukas Kahwe Smith
> mls <at> pooteeweet.org
>
>
>
>
> --
(Continue reading)

Permalink | Reply | 2 comments |
headers
Greg Beaver | 4 May 2009 06:25
Favicon

[PHP-DEV] major memory - can't make short test

Hi,

I have just found a significant memory leak, but cannot get a truly 
short reproducing script.  I have opened a bug, but I wanted to make 
sure this doesn't slip through the cracks since RC2 is slated for Tuesday.

http://bugs.php.net/bug.php?id=48141

is the bug.  I don't have a clear idea of why this is happening, but I 
can reproduce it every time.

Greg

--

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Permalink | Reply | 0 comments |
headers
shire | 4 May 2009 09:36
Picon

Re: [PHP-DEV] [PATCH] Scanner "diet" with fixes, etc.


Hey Matt,

Thanks for posting, sorry for not having a chance to reply to this sooner.  Maybe couple things from the patch,

>> +/* To save initial string length after scanning to first variable, CG(doc_comment_len) can be reused */
>> +#define double_quotes_scanned_len CG(doc_comment_len)
>> +

(minor) Maybe we should rename this var if we're going to use it for other purposes, this doesn't really save
any typing.  Also if we do want the define maybe we should upper case it so it's more obvious?

>> +	while (YYCURSOR < YYLIMIT) {
>> +		switch (*YYCURSOR++) {

In the example above, which we have a couple examples of here, we don't obey the YYFILL macro to detect if we
have exceeded our EOF.  This *might* be a problem, but only really depends on if we intend to use the YYFILL as
a solution for exceeding our mmap bounds.

Regarding the ZEND_MMAP_AHEAD issue and the temp. fix that Dmitry put in we need to find a solution to that,
perhaps I can play with that this week too as I think I'm seeing some related issues in my testing of 5.3. 
Essentially we abuse ZEND_MMAP_AHEAD by adding it to the file size and passing it to the mmap call which
isn't at all valid and only really works up to PAGESIZE.  We could possibly use YYFILL to re-allocate more
space as necessary past the end of file to fix this.

I don't see anything glaring in the patch that's a major issue, I can probably test more on a larger code base
in the next 2-3 days.  As I've said before this seems to be crossing the line of us writing a scanner by hand
rather than letting re2c do the heavy lifting, but without a modification to re2c to handle EOF I don't have
an alternative solution currently.  (If we had some way to detect which regex we where matching against in
the YYFILL that would likely be able to handle these bugs, but I didn't see a way to do that easily).
(Continue reading)

Permalink | Reply | 9 comments |
headers
Dmitry Stogov | 4 May 2009 09:40
Favicon
Gravatar

Re: [PHP-DEV] [PATCH] Scanner "diet" with fixes, etc.

Hi Matt,

I wasn't able to look into all details of the patch, but in general I 
like it, as it fixes bugs and makes scanner smaller. I think you can 
commit it.

Although this patch doesn't fix the EOF handling related to mmap().

Thanks. Dmitry.

Matt Wilmas wrote:
> Hi guys,
> 
> ----- Original Message -----
> From: "Nuno Lopes"
> Sent: Thursday, April 30, 2009
> 
>>>> The patch looks generally ok. However I'll need a few more days to 
>>>> review it carefully and throughly. (you can merge it in the 
>>>> meantime  if you want).
>>>> I'm just slighty concern with the amount of parsing we are now 
>>>> doing  by hand, and with the possible (local) security bugs we might 
>>>> be introducing..
>>>
>>>
>>> Am I understanding this properly, that this addresses the re2c EOF  
>>> bug? So we have an RC planned for next week (freeze Monday evening).  
>>> Can you get this fixed and released by then as Marcus is unable to 
>>> do  this himself?
>>
(Continue reading)

Permalink | Reply | 1 comment |
headers
internals | 4 May 2009 11:30
Picon

[PHP-DEV] PHP 5 Bug Summary Report

 PHP 5 Bug Database summary - http://bugs.php.net/

 Num Status     Summary (1327 total -- which includes 862 feature requests)
===============================================[*XML functions]===============
48095 Verified   Load RDF Format Error
===============================================[Apache2 related]==============
32220 Assigned   [PATCH] thread_resources for thread not getting freed when apache kills thread
47675 Open       File descriptor leaked due to HAVE_BROKEN_GETCWD
47681 Open       System TMP dir ignored in file uploads
48094 Feedback   Two graceful restarts are needed to enable PHP
===============================================[Arrays related]===============
47221 Open       no result from array_diff()
===============================================[BC math related]==============
44995 Open       bcpowmod() using a scale function always returns 0
46564 Verified   bcmod( '1071', '357.5' ) returns '0'
===============================================[Bzip2 Related]================
29521 Assigned   compress.bzip2 wrapper
===============================================[Calendar related]=============
40213 Suspended  easter_date() returns wrong timestamp if ...
===============================================[CGI related]==================
45217 Open       crash if -z and -m are used together
47042 Open       cgi sapi is incorrectly removing the SCRIPT_FILENAME for non apache
47412 Open       PHP_MSHUTDOWN_FUNCTION not being called under FastCGI
47540 Open       CLI can go into an infinite write() loop when ignore_user_abort(true)
47605 Open       CGI SAPI can not send HTTP 200 header
47627 Open       "No input file specified" causing crash
47766 Assigned   php-cgi.exe crashes
48104 Feedback   FD-Leak - FastCGI + auto_prepend
===============================================[Class/Object related]=========
41461 Verified   E_STRICT notice when overriding methods not defined by an Interface in hierarchy
(Continue reading)

Permalink | Reply | 0 comments |
headers
internals | 4 May 2009 12:00
Picon

[PHP-DEV] PHP 6 Bug Summary Report

 PHP 6 Bug Database summary - http://bugs.php.net/

 Num Status     Summary (75 total -- which includes 34 feature requests)
===============================================[Apache related]===============
47061 Open       User not logged under Apache
===============================================[Apache2 related]==============
44083 Open       virtual() not outputting results if zlib.output_compression = On
===============================================[Arrays related]===============
35277 Suspended  incorrect recursion detection
41758 Assigned   SORT_LOCALE_STRING broken for sort() in PHP6
43109 Open       array_intersect() emits unexpected no of notices when 2d array is passed as arg
===============================================[COM related]==================
45836 Open       cannot use com 
46909 Open       COM object not allowing calls to methods
===============================================[Compile Failure]==============
42606 Open       unicode/constants.c relies on ICU draft api
44502 Suspended  Compiling ok with MySQL 5.0
===============================================[Date/time related]============
46948 Assigned   ext/date/lib/parse_tz.c:99: Memory leak: buffer
===============================================[Filesystem function related]==
42110 Open       fgetcsv doesn't handle ""\n correctly in multiline csv record
44034 Open       FILE_IGNORE_NEW_LINES in FILE does not work as expected when lines end in \r\n
46688 Open       Return values differ from 5.3 and are also inconsistent
46689 Open       Downcoded notices suggest unfinished code in file system?
===============================================[GD related]===================
34670 Assigned   imageTTFText for Indian scripts (Devanagari)
34992 Assigned   imageconvolution does not respect alpha
===============================================[I18N and L10N related]========
42471 Open       locale_set_default returns true on invalid locales
===============================================[mcrypt related]===============
(Continue reading)

Permalink | Reply | 0 comments |

Gmane