headers
Jeff Janes | 1 Feb 03:39
Picon

Re: CLOG contention, part 2

On Mon, Jan 30, 2012 at 12:24 PM, Robert Haas <robertmhaas <at> gmail.com> wrote:
> On Fri, Jan 27, 2012 at 8:21 PM, Jeff Janes <jeff.janes <at> gmail.com> wrote:
>> On Fri, Jan 27, 2012 at 3:16 PM, Merlin Moncure <mmoncure <at> gmail.com> wrote:
>>> On Fri, Jan 27, 2012 at 4:05 PM, Jeff Janes <jeff.janes <at> gmail.com> wrote:
>>>> Also, I think the general approach is wrong.  The only reason to have
>>>> these pages in shared memory is that we can control access to them to
>>>> prevent write/write and read/write corruption.  Since these pages are
>>>> never written, they don't need to be in shared memory.   Just read
>>>> each page into backend-local memory as it is needed, either
>>>> palloc/pfree each time or using a single reserved block for the
>>>> lifetime of the session.  Let the kernel worry about caching them so
>>>> that the above mentioned reads are cheap.
>>>
>>> right -- exactly.  but why stop at one page?
>>
>> If you have more than one, you need code to decide which one to evict
>> (just free) every time you need a new one.  And every process needs to
>> be running this code, while the kernel is still going to need make its
>> own decisions for the entire system.  It seems simpler to just let the
>> kernel do the job for everyone.  Are you worried that a read syscall
>> is going to be slow even when the data is presumably cached in the OS?
>
> I think that would be a very legitimate worry.  You're talking about
> copying 8kB of data because you need two bits.  Even if the
> user/kernel mode context switch is lightning-fast, that's a lot of
> extra data copying.

I guess the most radical step in the direction I am advocating would
be to simply read the one single byte with the data you want.  Very
little copying, but then the odds of the next thing you want being on
(Continue reading)

Permalink | Reply | 0 comments |
headers
Robert Haas | 1 Feb 03:56
Picon

Re: Should I implement DROP INDEX CONCURRENTLY?

On Sun, Jan 29, 2012 at 5:01 PM, Simon Riggs <simon <at> 2ndquadrant.com> wrote:
> I can't see any way that situation can occur. The patch *explicitly*
> waits until all people that can see the index as usable have dropped
> their lock. So I don't think this is necessary. Having said that,
> since we are talking about the index and not the whole table, if I
> believe the above statement then I can't have any reasonable objection
> to doing as you suggest.
>
> Patch now locks index in AccessExclusiveLock in final stage of drop.
>
> v3 attached.
>
> If you have suggestions to improve grammar issues, they;re most
> welcome. Otherwise this seems good to go.

I improved the grammar issues in the attached version of the patch -
the syntax is now simpler and more consistent, IF EXISTS now works,
and RESTRICT is accepted (without changing the behavior) while CASCADE
fails with a nicer error message.  I also fixed a bug in
RangeVarCallbackForDropRelation.

However, testing reveals that this doesn't really work.  I found that
by playing around with a couple of concurrent sessions, I could get a
SELECT statement to hang waiting on the AccessExclusiveLock in the
second phase, because we're really still opening the relation (even
though it's invalid) and thus still locking it, and thus still waiting
for the AccessExclusiveLock.  And I managed to get this:

rhaas=# select * from foo where a = 1;
ERROR:  could not open relation with OID 16388
(Continue reading)

Permalink | Reply | 1 comment |
headers
Andrew Dunstan | 1 Feb 05:10
Gravatar

Re: [GENERAL] pg_dump -s dumps data?!


On 01/31/2012 05:48 PM, Tom Lane wrote:
> Andrew Dunstan<andrew <at> dunslane.net>  writes:
>> On 01/30/2012 11:18 PM, Tom Lane wrote:
>>> As I suspected, the behavioral change from 9.1 to HEAD is not
>>> intentional.  It is an artifact of commit
>>> 7b070e896ca835318c90b02c830a5c4844413b64, which is almost, but not
>>> quite, entirely broken.  I won't enumerate its shortcomings here,
>> I'm perplexed about what you thing the patch does wrong or how it affects this. If I've broken something
I'd like to know how, exactly, so I have a chance to fix it.
> Well, it adds a new field to all instances of DumpableObject and then
> leaves it uninitialized in most cases, which is bad style (and unlike in
> the backend, there is no forced zeroing to ensure a consistent value);
> but the proximate cause of the bug is that you put the filtering in the
> wrong place.  The way this is supposed to work, or at least used to
> work, is that dump-the-data-or-not is determined by whether
> a TableDataInfo DumpableObject gets created --- see the callers of
> makeTableDataInfo.  You didn't follow that convention but instead
> inserted an extra filter test in dumpTableData.  The reason that
> depesz's example is not dumping the unwanted config table is that the
> code path in which getExtensionMembership calls makeTableDataInfo isn't
> ever setting the dumpdata flag.  Unfortunately that means that config
> table data won't get dumped when it *is* wanted, either.  Or worse,
> it means that the data might or might not get dumped depending on
> whether the pg_malloc in makeTableDataInfo is allocating new or recycled
> memory and what happens to be in that memory in the latter case.
>
> Now I'm not entirely sure that we ought to preserve the old code
> structure as-is; it might be more future-proof if we always made
> TableDataInfo objects and then used their dump flags to control whether
(Continue reading)

Permalink | Reply | 5 comments |
headers
Andrew Dunstan | 1 Feb 06:46
Gravatar

Re: JSON for PG 9.2


On 01/31/2012 01:48 PM, Andrew Dunstan wrote:
>
>
> On 01/31/2012 01:32 PM, Merlin Moncure wrote:
>> On Tue, Jan 31, 2012 at 12:15 PM, Josh Berkus<josh <at> agliodbs.com>  wrote:
>>> Andrew,
>>>
>>>> based on Abhijit's feeling and some discussion offline, the consensus
>>>> seems to be to remove query_to_json.
>>> If we do that, what would getting complete query results back from a
>>> query look like?  It's important to make this as simple for developers
>>> as possible.
>> two options:
>> 1. row_to_json(rowvar)
>> SELECT row_to_json(foo) from foo;
>> SELECT row_to_json(row(a,b,c)) from foo;
>>
>> 2. array_to_json(array_agg()/array())
>> SELECT array_to_json(array(select foo from foo));
>> SELECT array_to_json(array[1,2,3]);
>>
>> #1 I expect will be the more used version -- most json handling client
>> side api (for example node.js drivers) are optimized for row by row
>> processing, but via #2 you can stuff a whole query into single json
>> object if you're so inclined.
>>
>
> You could also write a wrapper something like this:
>
(Continue reading)

Permalink | Reply | 1 comment |
headers
Pavel Stehule | 1 Feb 07:17
Picon
Gravatar

feature request - datum_compute_size and datum write_should be public

Hello

I looked to sources and I found a some useful routines for people who
write extensions and probably PL too.

There are datum_compute_size and datum_write from range_types.c. These
routines can be used in PL libs and maybe in other places.

Should be these routines moved to varlena.c and be public?

Regards

Pavel

--

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers <at> postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Permalink | Reply | 5 comments |
headers
Tom Lane | 1 Feb 07:29
Picon

Re: feature request - datum_compute_size and datum write_should be public

Pavel Stehule <pavel.stehule <at> gmail.com> writes:
> I looked to sources and I found a some useful routines for people who
> write extensions and probably PL too.

> There are datum_compute_size and datum_write from range_types.c. These
> routines can be used in PL libs and maybe in other places.

> Should be these routines moved to varlena.c and be public?

Why?  It is not common for types to contain other types, and it
certainly isn't likely to happen without needing lots of other
infrastructure --- the existing examples are arrays, records, and
rangetypes, and all of those come with lots of baggage.  And there
are a number of choices in those functions that are pretty specific to
rangetypes, as illustrated by the fact that they're not already sharing
code with either arrays or records.

			regards, tom lane

--

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers <at> postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Permalink | Reply | 4 comments |
headers
Pavel Stehule | 1 Feb 07:45
Picon
Gravatar

Re: feature request - datum_compute_size and datum write_should be public

2012/2/1 Tom Lane <tgl <at> sss.pgh.pa.us>:
> Pavel Stehule <pavel.stehule <at> gmail.com> writes:
>> I looked to sources and I found a some useful routines for people who
>> write extensions and probably PL too.
>
>> There are datum_compute_size and datum_write from range_types.c. These
>> routines can be used in PL libs and maybe in other places.
>
>> Should be these routines moved to varlena.c and be public?
>
> Why?  It is not common for types to contain other types, and it
> certainly isn't likely to happen without needing lots of other
> infrastructure --- the existing examples are arrays, records, and
> rangetypes, and all of those come with lots of baggage.  And there
> are a number of choices in those functions that are pretty specific to
> rangetypes, as illustrated by the fact that they're not already sharing
> code with either arrays or records.

For example I can use this code in my implementation of set of enum
(enumset datatype) because I have to wrap a array sometimes (I reuse a
array infrastructure).

In orafce I can use this code for serialisation and deserialisation
Datums - it is used more times there

Pavel

>
>                        regards, tom lane
(Continue reading)

Permalink | Reply | 3 comments |
headers
Kyotaro HORIGUCHI | 1 Feb 09:32
Picon

Re: Speed dblink using alternate libpq tuple storage

Hello,

> Another thing: if realloc() fails, the old pointer stays valid.
> So it needs to be assigned to temp variable first, before
> overwriting old pointer.

 mmm. I've misunderstood of the realloc.. I'll fix there in the
next patch.

> And seems malloc() is preferable to palloc() to avoid
> exceptions inside row processor.  Although latter
> one could be made to work, it might be unnecessary
> complexity.  (store current pqresult into remoteConn?)

Hmm.. palloc may throw ERRCODE_OUT_OF_MEMORY so I must catch it
and return NULL. That seems there is no difference to using
malloc after all.. However, the inhibition of throwing exceptions
in RowProcessor is not based on any certain fact, so palloc here
may make sense if we can do that.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center

--

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers <at> postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
(Continue reading)

Permalink | Reply | 16 comments |
headers
Dean Rasheed | 1 Feb 10:09
Picon

Re: Index-only scan performance regression

On 31 January 2012 23:04, Tom Lane <tgl <at> sss.pgh.pa.us> wrote:
> Dean Rasheed <dean.a.rasheed <at> gmail.com> writes:
>> The thing I'm unsure about is whether sending sinval
>> messages when the visibility map is extended is a good idea.
>
> Seems perfectly reasonable to me.  They'd occur so seldom as to be
> more than repaid if we can scrape some cost out of the mainline paths.
>

OK, thanks. That's good.

> The real objection to this probably is that if it only saves anything
> for tables that don't have a VM yet, it's dubious whether it's worth
> doing.  But if we can avoid wasted checks for VM extension as well,
> then I think it's probably a no-brainer.
>
>                        regards, tom lane

Yes it applies in the same way to VM extension - if the table has
grown and the VM has not yet been extended, but I don't see why that
is any worse than the case of not having a VM yet.

Actually I think that this is not such an uncommon case - for a table
which has only had data inserted - no deletes or updates - it is
tempting to think that ANALYSE is sufficient, and that there is no
need to VACUUM. If it were simply the case that this caused an
index-only scan to have no real benefit, you might be willing to live
with normal index scan performance. But actually it causes a very
significant performance regression beyond that, to well below 9.1
performance.
(Continue reading)

Permalink | Reply | 3 comments |
headers
Marko Kreen | 1 Feb 10:52
Picon

Re: Speed dblink using alternate libpq tuple storage

On Wed, Feb 1, 2012 at 10:32 AM, Kyotaro HORIGUCHI
<horiguchi.kyotaro <at> oss.ntt.co.jp> wrote:
>> Another thing: if realloc() fails, the old pointer stays valid.
>> So it needs to be assigned to temp variable first, before
>> overwriting old pointer.
>
>  mmm. I've misunderstood of the realloc.. I'll fix there in the
> next patch.

Please wait a moment, I started doing small cleanups,
and now have some larger ones too.  I'll send it soon.

OTOH, if you have already done something, you can send it,
I have various states in GIT so it should not be hard
to merge things.

>> And seems malloc() is preferable to palloc() to avoid
>> exceptions inside row processor.  Although latter
>> one could be made to work, it might be unnecessary
>> complexity.  (store current pqresult into remoteConn?)
>
> Hmm.. palloc may throw ERRCODE_OUT_OF_MEMORY so I must catch it
> and return NULL. That seems there is no difference to using
> malloc after all.. However, the inhibition of throwing exceptions
> in RowProcessor is not based on any certain fact, so palloc here
> may make sense if we can do that.

No, I was thinking about storing the result in connection
struct and then letting the exception pass, as the
PGresult can be cleaned later.  Thus we could get rid
(Continue reading)

Permalink | Reply | 1 comment |

Gmane