Marko Kreen | 26 Aug 22:36 2015

PL/Proxy 2.6

New version of PL/Proxy is out.

Includes support for language validator, Postgres 9.5.  Several
query canceling related bugfixes.



About PL/Proxy

PL/Proxy is database partitioning system implemented as PL language.



Sent via pgsql-announce mailing list (pgsql-announce <at>
To make changes to your subscription:

Satoshi Nagayasu | 25 Aug 04:08 2015

sql_firewall 0.8 released


I'm proud to announce the first release of sql_firewall.

sql_firewall is a PostgreSQL extension which is intended to
protect database from SQL injections or unexpected queries.

sql_firewall module learns queries allowed to be executed,
and prevents/warns on executing queries which are not found
in the learned firewall rule.

For more information, please visit the github repo.

NAGAYASU Satoshi <snaga <at>>


Sent via pgsql-announce mailing list (pgsql-announce <at>
To make changes to your subscription:

Terry Erisman | 24 Aug 17:25 2015

pg_shard 1.2 for Scalable PostgreSQL and cstore_fdw 1.3 for Columnar PostgreSQL Released

Citus Data is pleased to announce the release of the open source pg_shard 1.2 extension for creating a scalable PostgreSQL cluster on commodity physical or virtual servers. Release notes can be found on the Citus Data blog. A recent blog post describes how to scale PostgreSQL on Amazon RDS using pg_shard. Significant enhancements in this release include:


·        Full CitusDB integration — Distribution metadata is always in sync with CitusDB

·        Better type support — Partition by enumeration or composite types

·        Planning improvements — Improved internal metadata locking and function checking

·        Usability enhancements — Better validations and error messages during use



We are also pleased to announce the release of the open source cstore_fdw 1.3 extension for creating columnar PostgreSQL. Release notes can be found on the Citus Data blog. Significant enhancements in this release include:


·        ALTER FOREIGN TABLE ADD/DROP COLUMN ... support. You can now add/drop columns on existing cstore tables. Default values on newly added columns are also supported with some restrictions.

·        Improved COPY FROM support. Column references in the COPY command is now supported. You can now specify list of columns to be copied from a file to a cstore table.

·        Query performance improvements. Table row count estimation works better which allows the query planner to better prepare query plans.

·        (Fix) Whole row references. You can now use row references in function calls like SELECT to_json(*) FROM cstore_table.



We are also pleased to announce the release of CitusDB 4.1. Release notes are available on the Citus Data blog.





Terry Erisman

Citus Data



David Fetter | 24 Aug 01:05 2015

== PostgreSQL Weekly News - August 23 2015 ==

== PostgreSQL Weekly News - August 23 2015 ==

PgCUBA will take place October 19-23, 2015 in Havana.  Spanish
language information below:

== PostgreSQL Product News ==

Further enhancements to 2UDA (Alpha 2 release), a platform for data
analytics, available now.

2ndQuadrant now offers production support on IBM z Series mainframe

== PostgreSQL Jobs for August ==

== PostgreSQL Local ==

PostgresOpen 2015 will being held in Dallas, Texas September 16-18.

PostgreSQL Session #7, will be held September 24th, 2015 in Paris,

PGDay.IT 2015 will take place in Prato on October 23, 2015.

PostgreSQL Conference Europe 2015 will be held on October 27-30 in the
Vienna Marriott Hotel, in Vienna, Austria.

PGConf Silicon Valley 2015 is November 17-18 at the South San
Francisco Convention Center.

PGBR2015 (The Brazilian PostgreSQL Conference) will take place in Porto
Alegre, Rio Grande do Sul, on November 18, 19 and 20.  The CfP is open
until August 31.

== PostgreSQL in the News ==

Planet PostgreSQL:

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david <at>, German language
to pwn <at>, Italian language to pwn <at>  Spanish language
to pwn <at>

== Applied Patches ==

Heikki Linnakangas pushed:

- Fix reporting of skipped transactions in pgbench.  Broken by commit
  1bc90f7a.  Fabien Coelho.

- Add hint to run "pgbench -i", if test tables don't exist.  Fabien
  Coelho, reviewed by Julien Rouhaud

Andres Freund pushed:

- Improve configure test for the sse4.2 crc instruction.  With
  optimizations enabled at least one compiler, clang 3.7, optimized
  away the crc intrinsics knowing that the result went on unused and
  has no side effects. That can trigger errors in code generation when
  the intrinsic is used, as we chose to use the intrinsics without any
  additional compiler flag. Return the computed value to prevent that.
  With some more pedantic warning flags (-Wold-style-definition) the
  configure test failed to recognize the existence of _mm_crc32_u*
  intrinsics due to an independent warning in the test because the
  test turned on -Werror, but that's not actually needed here.
  Discussion: 20150814092039.GH4955 <at> Backpatch: 9.5,
  where the use of crc intrinsics was integrated.

- docs: Fix "typo" introduced in 3f811c2d.  Reported-By: Michael
  Paquier Discussion:
  CAB7nPqSco+RFw9C-VgbCpyurQB3OocS-fuTOa_gFnUy1EE-pyQ <at>

Tom Lane pushed:

- Fix performance bug from conflict between two previous improvements.
  My expanded-objects patch (commit 1dc5ebc9077ab742) included code to
  make plpgsql pass expanded-object variables as R/W pointers to
  certain functions that are trusted for modifying such variables
  in-place.  However, that optimization got broken by commit
  6c82d8d1fdb1f126, which arranged to share a single ParamListInfo
  across most expressions evaluated by a plpgsql function.  We don't
  want a R/W pointer to be passed to other functions just because we
  decided one function was safe!  Fortunately, the breakage was in the
  other direction, of never passing a R/W pointer at all, because we'd
  always have pre-initialized the shared array slot with a R/O
  pointer.  So it was still functionally correct, but we were back to
  O(N^2) performance for repeated use of "a := a || x".  To fix, force
  an unshared param array to be used when the R/W param optimization
  is active.  Commit 6c82d8d1fdb1f126 is in HEAD only, so no need for
  a back-patch.

- Fix a few bogus statement type names in plpgsql error messages.
  plpgsql's error location context messages ("PL/pgSQL function
  fn-name line line-no at stmt-type") would misreport a CONTINUE
  statement as being an EXIT, and misreport a MOVE statement as being
  a FETCH.  These are clear bugs that have been there a long time, so
  back-patch to all supported branches.  In addition, in 9.5 and HEAD,
  change the description of EXECUTE from "EXECUTE statement" to just
  plain EXECUTE; there seems no good reason why this statement type
  should be described differently from others that have a well-defined
  head keyword.  And distinguish GET STACKED DIAGNOSTICS from plain
  GET DIAGNOSTICS.  These are a bit more of a judgment call, and also
  affect existing regression-test outputs, so I did not back-patch
  into stable branches.  Pavel Stehule and Tom Lane

- Remove xpath namespace-handling change from 9.5 release notes.
  Although commit 79af9a1d2 was initially applied to HEAD only, we
  later back-patched the change into all branches (commits 6bbf75192
  et al).  So it's not a new behavior in 9.5 and should not be
  release-noted here.

- Allow record_in() and record_recv() to work for transient record
  types.  If we have the typmod that identifies a registered record
  type, there's no reason that record_in() should refuse to perform
  input conversion for it.  Now, in direct SQL usage, record_in() will
  always be passed typmod = -1 with type OID RECORDOID, because no
  typmodin exists for type RECORD, so the case can't arise.  However,
  some InputFunctionCall users such as PLs may be able to supply the
  right typmod, so we should allow this to support them.  Note: the
  previous coding and comment here predate commit 59c016aa9f490b53.
  There has been no case since 8.1 in which the passed type OID
  wouldn't be valid; and if it weren't, this error message wouldn't be
  apropos anyway.  Better to let lookup_rowtype_tupdesc complain about
  it.  Back-patch to 9.1, as this is necessary for my upcoming
  plpython fix.  I'm committing it separately just to make it a bit
  more visible in the commit history.

- Fix plpython crash when returning string representation of a RECORD
  result.  PLyString_ToComposite() blithely overwrote
  proc->result.out.d, even though for a composite result type the
  other union variant proc->result.out.r is the one that should be
  valid.  This could result in a crash if out.r had in fact been
  filled in (proc->result.is_rowtype == 1) and then somebody later
  attempted to use that data; as per bug #13579 from Paweł Michalak.
  Just to add insult to injury, it didn't work for RECORD results
  anyway, because record_in() would refuse the case.  Fix by doing the
  I/O function lookup in a local PLyTypeInfo variable, as we were
  doing already in PLyObject_ToComposite().  This is not a great
  technique because any fn_extra data allocated by the input function
  will be leaked permanently (thanks to using TopMemoryContext as
  fn_mcxt).  But that's a pre-existing issue that is much less serious
  than a crash, so leave it to be fixed separately.  This bug would be
  a potential security issue, except that plpython is only available
  to superusers and the crash requires coding the function in a way
  that didn't work before today's patches.  Add regression test cases
  covering all the supported methods of converting composite results.
  Back-patch to 9.1 where the faulty coding was introduced.

- Detect mismatched CONTINUE and EXIT statements at plpgsql compile
  time.  With a bit of tweaking of the compile namestack data
  structure, we can verify at compile time whether a CONTINUE or EXIT
  is legal.  This is surely better than leaving it to runtime, both
  because earlier is better and because we can issue a proper error
  pointer.  Also, we can get rid of the ad-hoc old way of detecting
  the problem, which only took care of CONTINUE not EXIT.  Jim Nasby,
  adjusted a bit by me

- Avoid O(N^2) behavior when enlarging SPI tuple table in
  spi_printtup().  For no obvious reason, spi_printtup() was coded to
  enlarge the tuple pointer table by just 256 slots at a time, rather
  than doubling the size at each reallocation, as is our usual habit.
  For very large SPI results, this makes for O(N^2) time spent in
  repalloc(), which of course soon comes to dominate the runtime.  Use
  the standard doubling approach instead.  This is a longstanding
  performance bug, so back-patch to all active branches.  Neil Conway

- Avoid use of float arithmetic in bipartite_match.c.  Since the
  distances used in this algorithm are small integers (not more than
  the size of the U set, in fact), there is no good reason to use
  float arithmetic for them.  Use short ints instead: they're smaller,
  faster, and require no special portability assumptions.  Per testing
  by Greg Stark, which disclosed that the code got into an infinite
  loop on VAX for lack of IEEE-style float infinities.  We don't
  really care all that much whether Postgres can run on a VAX anymore,
  but there seems sufficient reason to change this code anyway.  In
  passing, make a few other small adjustments to make the code match
  usual Postgres coding style a bit better.

- Reduce number of bytes examined by convert_one_string_to_scalar().
  Previously, convert_one_string_to_scalar() would examine up to 20
  bytes of the input string, producing a scalar conversion with
  theoretical precision far greater than is of any possible use
  considering the other limitations on the accuracy of the resulting
  selectivity estimate.  (I think this choice might pre-date the
  caller-level logic that strips any common prefix of the strings;
  before that, there could have been value in scanning the strings far
  enough to use all the precision available in a double.) Aside from
  wasting cycles to little purpose, this choice meant that the "denom"
  variable could grow to as much as 256^21 = 3.74e50, which could
  overflow in some non-IEEE float arithmetics.  While we don't really
  support any machines with non-IEEE arithmetic anymore, this still
  seems like quite an unnecessary platform dependency.  Limit the scan
  to 12 bytes instead, thus limiting "denom" to 256^13 = 2.03e31, a
  value more likely to be computable everywhere.  Per testing by Greg
  Stark, which showed overflow failures in our standard regression
  tests on VAX.

Robert Haas pushed:

- psql: Make EXECUTE PROCEDURE tab completion a bit narrower.  If the
  user has typed GRANT EXECUTE, the correct completion is "ON", not
  "PROCEDURE".  Daniel Verite

Kevin Grittner pushed:

- Fix bug in calculations of hash join buckets.  Commit
  8cce08f168481c5fc5be4e7e29b968e314f1b41e used a left-shift on a
  literal of 1 that could (in large allocations) be shifted by 31 or
  more bits.  This was assigned to a local variable that was already
  declared to be a long to protect against overruns of int, but the
  literal in this shift needs to be declared long to allow it to work
  correctly in some compilers.  Backpatch to 9.5, where the bug was
  introduced.  Report and patch by KaiGai Kohei, slighly modified
  based on discussion.

- Fix typo in C comment.  Merlin Moncure Backpatch to 9.5, where the
  misspelling was introduced

Peter Eisentraut pushed:

- Update config.guess and config.sub

- doc: Whitespace and formatting fixes

- Improve whitespace

- Improve spelling

Stephen Frost pushed:

- Rename 'cmd' to 'cmd_name' in CreatePolicyStmt.  To avoid confusion,
  rename CreatePolicyStmt's 'cmd' to 'cmd_name',
  parse_policy_command's 'cmd' to 'polcmd', and AlterPolicy's
  'cmd_datum' to 'polcmd_datum', per discussion with Noah and as a
  follow-up to his correction of copynodes/equalnodes handling of the
  CreatePolicyStmt 'cmd' field.  Back-patch to 9.5 where the
  CreatePolicyStmt was introduced, as we are still only in alpha.

- In AlterRole, make bypassrls an int.  When reworking bypassrls in
  AlterRole to operate the same way the other attribute handling is
  done, I missed that the variable was incorrectly a bool rather than
  an int.  This meant that on platforms with an unsigned char, we
  could end up with incorrect behavior during ALTER ROLE.  Pointed out
  by Andres thanks to tests he did changing our bool to be the one
  from stdbool.h which showed this and a number of other issues.  Add
  regression tests to test CREATE/ALTER role for the various role
  attributes.  Arrange to leave roles behind for testing pg_dumpall,
  but none which have the LOGIN attribute.  Back-patch to 9.5 where
  the AlterRole bug exists.

- Clean up roles from roleattributes test.  Having the roles remain
  after the test ends up causing repeated 'make installcheck' runs to
  fail and may be risky from a security perspective also, so remove
  them at the end of the test.

Álvaro Herrera pushed:

- Remove ExecGetScanType function.  This became unused in

- Do not allow *timestamp to be passed as NULL.  The code had bugs
  that would cause crashes if NULL was passed as that argument
  (originally intended to mean not to bother returning its value), and
  after inspection it turns out that nothing seems interested in the
  case that *ts is NULL anyway.  Therefore, remove the partial checks
  intended to support that case.  Author: Michael Paquier though I
  didn't include a proposed Assert.  Backpatch to 9.5.

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Mark Johnston sent in a patch to fix a bug in the make rules when
DTrace is enabled.

Fabien COELHO sent in another revision of a patch to add pgbench
statistics per script, etc.

Robert Haas sent in a flock of patches to allow multiple isolation
tester sessions to wait, specify permutations for isolation tests with
"invalid" permutations, and aAdd simple isolation tests for deadlock

Ewan Higgs sent in a patch to add stdatomic.h and support for same to
the thread_test.c.

Michael Paquier sent in another revision of a patch to document the
fact that partial and incomplete WALs can be archived upon promotion
for debugging purposes.

Michael Paquier sent in another revision of a patch to implement SCRAM

Dmitry Dolgov sent in a patch to add jsonb array-style subscripting
and auto-vivification.

Jeff Janes sent in a patch which takes a ShareUpdateExclusiveLock lock
on the index in order to clean the GIN pending list.

Fabien COELHO sent in two more revisions of a patch to implement
checkpointer continuous flushing.

Marko Tiikkaja sent in another revision of a patch to add support for
RADIUS passwords longer than 16 octets.

Kyotaro HORIGUCHI sent in a patch to export the psql parser for use in

Paul A Jungwirth sent in a patch to add GiST support for UUIDs.

Kaigai Kouhei sent in a patch to fix an issue where the number of hash
buckets can overflow its type.

Qingqing Zhou sent in a patch to substitute CTEs with subqueries,
resulting in a significant performance boost.

SAWADA Masahiko sent in another revision of a patch to add a "frozen"
bit to the visibility map.

Jeff Janes and Tom Lane traded patches to make HeapTupleSatisfiesMVCC
more concurrent.

John Naylor and Pavel Stěhule traded patches for CONTINUE in PL/pgsql.

Tomas Vondra sent in another revision of a patch to use foreign keys
to improve join estimates.

Peter Geoghegan sent in a patch to use quicksort for every external
sort run.

Tomas Vondra sent in two revisions of a patch to fix hash bucket

Alexander Shulgin sent in two more revisions of a patch to add
deparsing utility commands.

Tom Lane sent in a patch to summarize memory context stats.

Joe Conway sent in another revision of a patch to add pg_controldata
and pg_config as functions.

Tomas Vondra and Fabien COELHO traded patches to allow numeric
timestamp in log_line_prefix.

Fabien COELHO sent in a patch to add pgbench progress with timestamps.

Michael Paquier sent in two more revisions of a patch to add a
function for SSL extension information in sslinfo and some sanity
checks in sslinfo.

Pavel Stěhule sent in another revision of a patch to add a
--strict-names mode to pg_dump.

Pavel Stěhule sent in a patch to add a parse_ident() function.


Sent via pgsql-announce mailing list (pgsql-announce <at>
To make changes to your subscription:

David Fetter | 17 Aug 02:47 2015

== PostgreSQL Weekly News - August 16 2015 ==

== PostgreSQL Weekly News - August 16 2015 ==

== PostgreSQL Jobs for August ==

== PostgreSQL Local ==

PostgresOpen 2015 will being held in Dallas, Texas September 16-18.

PostgreSQL Session #7, will be held September 24th, 2015 in Paris,

PGDay.IT 2015 will take place in Prato on October 23, 2015.

PostgreSQL Conference Europe 2015 will be held on October 27-30 in the
Vienna Marriott Hotel, in Vienna, Austria.

PGConf Silicon Valley 2015 is November 17-18 at the South San
Francisco Convention Center.

PGBR2015 (The Brazilian PostgreSQL Conference) will take place in Porto
Alegre, Rio Grande do Sul, on November 18, 19 and 20.  The CfP is open
until August 31.

== PostgreSQL in the News ==

Planet PostgreSQL:

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david <at>, German language
to pwn <at>, Italian language to pwn <at>  Spanish language
to pwn <at>

== Applied Patches ==

Andres Freund pushed:

- Fix copy & paste mistake in pg_get_replication_slots().  XLogRecPtr
  was compared with InvalidTransactionId instead of InvalidXLogRecPtr.
  As both are defined to the same value this doesn't cause any actual
  problems, but it's still wrong.  Backpatch: 9.4-master, bug was
  introduced in 9.4

- Don't start to stream after pg_receivexlog --create-slot.
  Immediately starting to stream after --create-slot is inconvenient
  in a number of situations (e.g. when configuring a slot for use in
  recovery.conf) and it's easy to just call pg_receivexlog twice in
  the rest of the cases.  Author: Michael Paquier Discussion:
  CAB7nPqQ9qEtuDiKY3OpNzHcz5iUA+DUX9FcN9K8GUkCZvG7+Ew <at>
  Backpatch: 9.5, where the option was introduced

- Add confirmed_flush column to pg_replication_slots.  There's no
  reason not to expose both restart_lsn and confirmed_flush since they
  have rather distinct meanings. The former is the oldest WAL still
  required and valid for both physical and logical slots, whereas the
  latter is the location up to which a logical slot's consumer has
  confirmed receiving data. Most of the time a slot will require older
  WAL (i.e. restart_lsn) than the confirmed position (i.e.
  confirmed_flush_lsn).  Author: Marko Tiikkaja, editorialized by me
  Discussion: 559D110B.1020109 <at>

- Minor cleanups in slot related code.  Fix a bunch of typos, and
  remove two superflous includes.  Author: Gurjeet Singh Discussion:
  CABwTF4Wh_dBCzTU=49pFXR6coR4NW1ynb+vBqT+Po=7fuq5iCw <at>
  Backpatch: 9.4

- Introduce macros determining if a replication slot is physical or
  logical.  These make the code a bit easier to read, and make it
  easier to add a more explicit notion of a slot's type at some point
  in the future.  Author: Gurjeet Singh Discussion:
  CABwTF4Wh_dBCzTU=49pFXR6coR4NW1ynb+vBqT+Po=7fuq5iCw <at>

- Handle PQresultErrorField(PG_DIAG_SQLSTATE) returning NULL in
  streamutil.c.  In ff27db5d I missed that PQresultErrorField() may
  return NULL if there's no sqlstate associated with an error.
  Spotted-By: Coverity Reported-By: Michael Paquier Discussion:
  CAB7nPqQ3o10SY6NVdU4pjq85GQTN5tbbkq2gnNUh2fBNU3rKyQ <at>
  Backpatch: 9.5, like ff27db5d

- Fix two off-by-one errors in bufmgr.c.  In 4b4b680c I passed a
  buffer index number (starting from 0) instead of a proper Buffer id
  (which start from 1 for shared buffers) in two places.  This wasn't
  noticed so far as one of those locations isn't compiled at all
  (PrintPinnedBufs) and the other one (InvalidBuffer) requires a
  unlikely, but possible, set of circumstances to trigger a symptom.
  To reduce the likelihood of such incidents a bit also convert
  existing open coded mappings from buffer descriptors to buffer ids
  with BufferDescriptorGetBuffer().  Author: Qingqing Zhou
  Reported-By: Qingqing Zhou Discussion:
  CAJjS0u2ai9ooUisKtkV8cuVUtEkMTsbK8c7juNAjv8K11zeCQg <at>
  Backpatch: 9.5 where the private ref count infrastructure was

- Remove duplicated assignment in pg_create_physical_replication_slot.
  Reported-By: Gurjeet Singh

- Allow pg_create_physical_replication_slot() to reserve WAL.  When
  creating a physical slot it's often useful to immediately reserve
  the current WAL position instead of only doing after the first
  feedback message arrives. That e.g. allows slots to guarantee that
  all the WAL for a base backup will be available afterwards.  Logical
  slots already have to reserve WAL during creation, so generalize
  that logic into being usable for both physical and logical slots.
  Catversion bump because of the new parameter.  Author: Gurjeet Singh
  Reviewed-By: Andres Freund Discussion:
  CABwTF4Wh_dBCzTU=49pFXR6coR4NW1ynb+vBqT+Po=7fuq5iCw <at>

- Use the correct type for TableInfo->relreplident.  Mistakenly
  relreplident was stored as a bool. That works today as c.h typedefs
  bool to a char, but isn't very future proof.  Discussion:
  20150812084351.GD8470 <at> Backpatch: 9.4 where
  replica identity was introduced.

- Don't use 'bool' as a struct member name in help_config.c.  Doing so
  doesn't work if bool is a macro rather than a typedef.  Although c.h
  spends some effort to support configurations where bool is a
  preexisting macro, help_config.c has existed this way since 2003
  (b700a6), and there have not been any reports of problems. Backpatch
  anyway since this is as riskless as it gets.  Discussion:
  20150812084351.GD8470 <at> Backpatch: 9.0-master

- vacuumdb: Don't assign negative values to a boolean.  Since
  a17923204736 (vacuumdb: enable parallel mode) -1 has been assigned
  to a boolean. That can, justifiedly, trigger compiler warnings.
  There's also no need for ternary logic, result was only ever set to
  0 or -1. So don't.  Discussion:
  20150812084351.GD8470 <at> Backpatch: 9.5

- Correct type of waitMode variable in ExecInsertIndexTuples().  It
  was a bool, even though it should be CEOUC_WAIT_MODE. That's
  unlikely to have a negative effect with the current definition of
  bool (char), but it's definitely wrong.  Discussion:
  20150812084351.GD8470 <at> Backpatch: 9.5, where ON
  CONFLICT was merged

- Don't use function definitions looking like old-style ones.  This
  fixes a bunch of somewhat pedantic warnings with new compilers.
  Since by far the majority of other functions definitions use the
  (void) style it just seems to be consistent to do so as well in the
  remaining few places.

Andrew Dunstan pushed:

- Work around an apparent bug in the Msys DTK perl's regex engine.
  Several versions of the perl that comes with the Msys DTK have been
  found to have a bug that fails to recognize a ' before a multiline $
  in some circumstances. To work around the problem, use a character
  class for the '. Another solution would have been to use \n instead
  of $, but that would have changed the test semantics very slightly.

- More fixes to allow pg_rewind tests to run on Msys.

Tom Lane pushed:

- Temporarily(?) remove BRIN isolation test.  Commit 2834855cb added a
  not-very-carefully-thought-out isolation test to check a BRIN index
  bug fix.  The test depended on the availability of the pageinspect
  contrib module, which meant it did not work in several common
  testing scenarios such as "make check-world".  It's not clear
  whether we want a core test depending on a contrib module like that,
  but in any case, failing to deal with the possibility that the
  module isn't present in the installation-under-test is not
  acceptable.  Remove that test pending some better solution.

- Further mucking with PlaceHolderVar-related restrictions on join
  order.  Commit 85e5e222b1dd02f135a8c3bf387d0d6d88e669bd turns out
  not to have taken care of all cases of the
  partially-evaluatable-PlaceHolderVar problem found by Andreas
  Seltenreich's fuzz testing.  I had set it up to check for risky PHVs
  only in the event that we were making a star-schema-based exception
  to the param_source_rels join ordering heuristic.  However, it turns
  out that the problem can occur even in joins that satisfy the
  param_source_rels heuristic, in which case allow_star_schema_join()
  isn't consulted.  Refactor so that we check for risky PHVs whenever
  the proposed join has any remaining parameterization.  Back-patch to
  9.2, like the previous patch (except for the regression test case,
  which only works back to 9.3 because it uses LATERAL).  Note that
  this discovery implies that problems of this sort could've occurred
  in 9.2 and up even before the star-schema patch; though I've not
  tried to prove that experimentally.

- Accept alternate spellings of __sparcv7 and __sparcv8.  Apparently
  some versions of gcc prefer __sparc_v7__ and __sparc_v8__.  Per
  report from Waldemar Brodkorb.

- Fix privilege dumping from servers too old to have that type of
  privilege.  pg_dump produced fairly silly GRANT/REVOKE commands when
  dumping types from pre-9.2 servers, and when dumping functions or
  procedural languages from pre-7.3 servers.  Those server versions
  lack the typacl, proacl, and/or lanacl columns respectively, and
  pg_dump substituted default values that were in fact incorrect.  We
  ended up revoking all the owner's own privileges for the object
  while granting all privileges to PUBLIC.  Of course the owner would
  then have those privileges again via PUBLIC, so long as she did not
  try to revoke PUBLIC's privileges; which may explain the lack of
  field reports.  Nonetheless this is pretty silly behavior.  The
  stakes were raised by my recent patch to make pg_dump dump shell
  types, because 9.2 and up pg_dump would proceed to emit bogus
  GRANT/REVOKE commands for a shell type if dumping from a pre-9.2
  server; and the server will not accept GRANT/REVOKE commands for a
  shell type.  (Perhaps it should, but that's a topic for another
  day.)  So the resulting dump script wouldn't load without errors.
  The right thing to do is to act as though these objects have default
  privileges (null ACL entries), which causes pg_dump to print no
  GRANT/REVOKE commands at all for them.  That fixes the silly results
  and also dodges the problem with shell types.  In passing, modify
  getProcLangs() to be less creatively different about how to handle
  missing columns when dumping from older server versions.  Every
  other data-acquisition function in pg_dump does that by substituting
  appropriate default values in the version-specific SQL commands, and
  I see no reason why this one should march to its own drummer.  Its
  use of "SELECT *" was likewise not conformant with anyplace else,
  not to mention it's not considered good SQL style for production
  queries.  Back-patch to all supported versions.  Although 9.0 and
  9.1 pg_dump don't have the issue with typacl, they are more likely
  than newer versions to be used to dump from ancient servers, so we
  ought to fix the proacl/lanacl issues all the way back.

- Fix broken markup, and copy-edit a bit.  Fix docs build failure
  introduced by commit 6fcd88511f8e69e3.  I failed to resist the
  temptation to rearrange the description of
  pg_create_physical_replication_slot(), too.

- Postpone extParam/allParam calculations until the very end of
  planning.  Until now we computed these Param ID sets at the end of
  subquery_planner, but that approach depends on subquery_planner
  returning a concrete Plan tree.  We would like to switch over to
  returning one or more Paths for a subquery, and in that
  representation the necessary details aren't fully fleshed out (not
  to mention that we don't really want to do this work for Paths that
  end up getting discarded).  Hence, refactor so that we can compute
  the param ID sets at the end of planning, just before
  set_plan_references is run.  The main change necessary to make this
  work is that we need to capture the set of outer-level Param IDs
  available to the current query level before exiting
  subquery_planner, since the outer levels' plan_params lists are
  transient.  (That's not going to pose a problem for returning Paths,
  since all the work involved in producing that data is part of
  expression preprocessing, which will continue to happen before Paths
  are produced.) On the plus side, this change gets rid of several
  existing kluges.  Eventually I'd like to get rid of SS_finalize_plan
  altogether in favor of doing this work during set_plan_references,
  but that will require some complex rejiggering because
  SS_finalize_plan needs to visit subplans and initplans before the
  main plan.  So leave that idea for another day.

- Fix some possible low-memory failures in regexp compilation.
  newnfa() failed to set the regex error state when malloc() fails.
  Several places in regcomp.c failed to check for an error after
  calling subre().  Each of these mistakes could lead to
  null-pointer-dereference crashes in memory-starved backends.  Report
  and patch by Andreas Seltenreich.  Back-patch to all branches.

- Undo mistaken tightening in join_is_legal().  One of the changes I
  made in commit 8703059c6b55c427 turns out not to have been such a
  good idea: we still need the exception in join_is_legal() that
  allows a join if both inputs already overlap the RHS of the special
  join we're checking.  Otherwise we can miss valid plans, and might
  indeed fail to find a plan at all, as in recent report from Andreas
  Seltenreich.  That code was added way back in commit
  c17117649b9ae23d, but I failed to include a regression test case
  then; my bad.  Put it back with a better explanation, and a test
  this time.  The logic does end up a bit different than before
  though: I now believe it's appropriate to make this check first,
  thereby allowing such a case whether or not we'd consider the
  previous SJ(s) to commute with this one.  (Presumably, we already
  decided they did; but it was confusing to have this consideration in
  the middle of the code that was handling the other case.) Back-patch
  to all active branches, like the previous patch.

- Improve regression test case to avoid depending on system catalog
  stats.  In commit 95f4e59c32866716 I added a regression test case
  that examined the plan of a query on system catalogs.  That isn't a
  terribly great idea because the catalogs tend to change from version
  to version, or even within a version if someone makes an unrelated
  regression-test change that populates the catalogs a bit
  differently.  Usually I try to make planner test cases rely on test
  tables that have not changed since Berkeley days, but I got sloppy
  in this case because the submitted crasher example queried the
  catalogs and I didn't spend enough time on rewriting it.  But it was
  a problem waiting to happen, as I was rudely reminded when I tried
  to port that patch into Salesforce's Postgres variant :-(.  So spend
  a little more effort and rewrite the query to not use any system
  catalogs.  I verified that this version still provokes the Assert if
  95f4e59c32866716's code fix is reverted.  I also removed the EXPLAIN
  output from the test, as it turns out that the assertion occurs
  while considering a plan that isn't the one ultimately selected
  anyway; so there's no value in risking any cross-platform variation
  in that printout.  Back-patch to 9.2, like the previous patch.

- Repair unsafe use of shared typecast-lookup table in plpgsql DO
  blocks.  DO blocks use private simple_eval_estates to avoid
  intra-transaction memory leakage, cf commit c7b849a89.  I had
  forgotten about that while writing commit 0fc94a5ba, but it means
  that expression execution trees created within a DO block disappear
  immediately on exiting the DO block, and hence can't safely be
  linked into plpgsql's session-wide cast hash table.  To fix, give a
  DO block a private cast hash table to go with its private
  simple_eval_estate.  This is less efficient than one could wish,
  since DO blocks can no longer share any cast lookup work with other
  plpgsql execution, but it shouldn't be too bad; in any case it's no
  worse than what happened in DO blocks before commit 0fc94a5ba.  Per
  bug #13571 from Feike Steenbergen.  Preliminary analysis by
  Oleksandr Shulgin.

- Improve documentation about MVCC-unsafe utility commands.  The
  table-rewriting forms of ALTER TABLE are MVCC-unsafe, in much the
  same way as TRUNCATE, because they replace all rows of the table
  with newly-made rows with a new xmin.  (Ideally, concurrent
  transactions with old snapshots would continue to see the old table
  contents, but the data is not there anymore --- and if it were
  there, it would be inconsistent with the table's updated rowtype, so
  there would be serious implementation problems to fix.) This was
  nowhere documented though, and the problem was only documented for
  TRUNCATE in a note in the TRUNCATE reference page.  Create a new
  "Caveats" section in the MVCC chapter that can be home to this and
  other limitations on serializable consistency.  In passing, fix a
  mistaken statement that VACUUM and CLUSTER would reclaim space
  occupied by a dropped column.  They don't reconstruct existing
  tuples so they couldn't do that.  Back-patch to all supported

- Add docs about postgres_fdw's setting of search_path and other GUCs.
  This behavior wasn't documented, but it should be because it's
  user-visible in triggers and other functions executed on the remote
  server.  Per question from Adam Fuchs.  Back-patch to 9.3 where
  postgres_fdw was added.

Álvaro Herrera pushed:

- Don't include rel.h when relcache.h is sufficient.  Trivial change
  to reduce exposure of rel.h.

- Close some holes in BRIN page assignment.  In some corner cases, it
  is possible for the BRIN index relation to be extended by
  brin_getinsertbuffer but the new page not be used immediately for
  anything by its callers; when this happens, the page is initialized
  and the FSM is updated (by brin_getinsertbuffer) with the info about
  that page, but these actions are not WAL-logged.  A later index
  insert/update can use the page, but since the page is already
  initialized, the initialization itself is not WAL-logged then
  either.  Replay of this sequence of events causes recovery to fail
  altogether.  There is a related corner case within
  brin_getinsertbuffer itself, in which we extend the relation to put
  a new index tuple there, but later find out that we cannot do so,
  and do not return the buffer; the page obtained from extension is
  not even initialized.  The resulting page is lost forever.  To fix,
  shuffle the code so that initialization is not the responsibility of
  brin_getinsertbuffer anymore, in normal cases; instead, the
  initialization is done by its callers (brin_doinsert and
  brin_doupdate) once they're certain that the page is going to be
  used.  When either those functions determine that the new page
  cannot be used, before bailing out they initialize the page as an
  empty regular page, enter it in FSM and WAL-log all this.  This way,
  the page is usable for future index insertions, and WAL replay
  doesn't find trying to insert tuples in pages whose initialization
  didn't make it to the WAL.  The same strategy is used in
  brin_getinsertbuffer when it cannot return the new page.
  Additionally, add a new step to vacuuming so that all pages of the
  index are scanned; whenever an uninitialized page is found, it is
  initialized as empty and WAL-logged.  This closes the hole that the
  relation is extended but the system crashes before anything is
  WAL-logged about it.  We also take this opportunity to update the
  FSM, in case it has gotten out of date.  Thanks to Heikki
  Linnakangas for finding the problem that kicked some additional
  analysis of BRIN page assignment code.  Backpatch to 9.5, where BRIN
  was introduced.  Discussion: <at>

- Fix unitialized variables.  As complained by clang, reported by
  Andres Freund.  Brown paper bag bug in ccc4c074994d.  Add some
  comments, too.  Backpatch to 9.5, like that one.

- Use materialize SRF mode in brin_page_items.  This function was
  using the single-value-per-call mechanism, but the code relied on a
  relcache entry that wasn't kept open across calls.  This manifested
  as weird errors in buildfarm during the short time that the "brin-1"
  isolation test lived.  Backpatch to 9.5, where it was introduced.

- Re-add BRIN isolation test.  This time, instead of using a core
  isolation test, put it on its own test module; this way it can
  require the pageinspect module to be present before running.  The
  module's Makefile is loosely modeled after test_decoding's, so that
  it's easy to add further tests for either pg_regress or
  isolationtester later.  Backpatch to 9.5.

- MSVC: Exclude 'brin' contrib module.  The build script is not able
  to parse the Makefile, so remove it.

Michael Meskes pushed:

- Fix declaration of isarray variable.  Found and fixed by Andres

Heikki Linnakangas pushed:

- Run autoheader to add a few missing #defines to
  These are emitted by the new ax_pthread.m4 script version. They are
  not used for anything in PostgreSQL, but let's keep the generated
  header file up-to-date.  Andres Freund

Peter Eisentraut pushed:

- PL/Python: Make tests pass with Python 3.5.  The error message
  wording for AttributeError has changed in Python 3.5.  For the
  plpython_error test, add a new expected file.  In the
  plpython_subtransaction test, we didn't really care what the
  exception is, only that it is something coming from Python.  So use
  a generic exception instead, which has a message that doesn't vary
  across versions.

- Update key words table for 9.5

Noah Misch pushed:

- Encoding PG_UHC is code page 949.  This fixes presentation of
  non-ASCII messages to the Windows event log and console in rare
  cases involving Korean locale.  Processes like the postmaster and
  checkpointer, but not processes attached to databases, were
  affected.  Back-patch to 9.4, where MessageEncoding was introduced.
  The problem exists in all supported versions, but this change has no
  effect in the absence of the code recognizing PG_UHC
  MessageEncoding.  Noticed while investigating bug #13427 from Dmitri

- Restore old pgwin32_message_to_UTF16() behavior outside
  transactions.  Commit 49c817eab78c6f0ce8c3bf46766b73d6cf3190b7
  replaced with a hard error the dubious pg_do_encoding_conversion()
  behavior when outside a transaction.  Reintroduce the historic soft
  failure locally within pgwin32_message_to_UTF16().  This fixes
  errors when writing messages in less-common encodings to the Windows
  event log or console.  Back-patch to 9.4, where the aforementioned
  commit first appeared.  Per bug #13427 from Dmitri Bourlatchkov.

Robert Haas pushed:

- Reject isolation test specifications with duplicate step names.
  alter-table-1.spec has such a case, so change one instance of step
  rx1 to rx3 instead.

- Remove bogus step from test_decoding isolation tests.  Commit
  43b4a16817c8b5568cec72f3b0e1c8209f5ac7f7 made the isolation tester
  reject duplicate step names, and it turns out that the test_decoding
  module's concurrent_ddl_dml isolation test has a duplicate name.  I
  think the second definition isn't actually getting used, so just
  remove it.  Per buildfarm.

- Remove unused expected-output file.

Simon Riggs pushed:

- Reduce lock levels for ALTER TABLE SET autovacuum storage options.
  Reduce lock levels down to ShareUpdateExclusiveLock for all
  autovacuum-related relation options when setting them using ALTER
  TABLE.  Add infrastructure to allow varying lock levels for relation
  options in later patches. Setting multiple options together uses the
  highest lock level required for any option. Works for both main and
  toast tables.  Fabrízio Mello, reviewed by Michael Paquier, mild
  edit and additional regression tests from myself

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Franck Verrot sent in a patch to mention column name in error

Jeff Janes sent in a patch to adjust the documentation of
gin_leafpage_items in the pageinspect extension to specify that only
compressed pages are supported.

Bruce Momjian sent in a patch to fix a bug where pg_upgrade fails when
postgres/template1 isn't in default tablespace.

Jeff Janes sent in a patch to fix the FSM versus GIN pending list
bloat issue.

Fabien COELHO sent in three more revisions of a patch to add a
checkpoint_sort GUC.

Jeff Janes sent in another revision of a patch to make more sane
default statistics for array types.

Fabien COELHO sent in a patch to fix a bug in pgbench where gbench
does not report skipped transactions (-L) counts properly.

Michael Paquier sent in a patch to fix a situation with
TransactionIdGetCommitTsData and its dereferenced pointers.

Ashutosh Bapat sent in two more revisions of a patch to add a contrib
module to launch foreign transaction resolver to resolve unresolved
transactions prepared on foreign servers.

Jeff Janes sent in two revisions of a patch to make GIN pending clean
up interruptible.

Michael Paquier sent in another revision of a patch to allow using
SCRAM authentication instead of MD5 password hashing.

Andres Freund sent in a patch to convert fastgetattr() and
heap_getattr() into inline functions.  This is in service of upping
the compiler requirements for 9.6 and later.

Etsuro Fujita and Kaigai Kouhei traded patches for evaluating qual
pushdowns to foreign servers and fixing their relationship to

Andres Freund sent in a patch to change some buffer and page related
macros to inline functions.

Jeff Janes sent in a patch to expose GIN pending list clean up to SQL.

Pavel Stěhule sent in another revision of a patch to add a libpq
context filter.

Marko Tiikkaja sent in a patch to add a count_nulls(VARIADIC "any").

Taiki Kondo sent in a patch to implement a hash join pushdown for
tables partitioned by hash.

Pavel Stěhule sent in another revision of a patch to add a
--strict-names option to pg_dump.

Haribabu Kommi sent in a PoC patch to implement more isolated
multi-tenancy via RLS.

Michael Paquier sent in two more revisions of a patch to add
regression tests for recovery.

Andrew Dunstan sent in two more revisions of a patch to fix TAP tests
on Windows.

Robert Haas sent in a patch to allow the isolation tester to run with
more than one waiting process.

Dickson S. Guedes sent in a patch to allow, "semester" in date_part,
etc., which turned out to be local.  Current trends go towards "half,"
which is less ambiguous in its meaning.

Andres Freund sent in a WIP patch to decrease usage of lock.h further.
This is part of the drive toward requiring more recent compilers for

Marko Tiikkaja sent in a patch to add support for RADIUS passwords
longer than 16 octets.

David Rowley sent in a patch to improve how get_base_rel_indexes()
works by changing its return type and adding a parameter.


Sent via pgsql-announce mailing list (pgsql-announce <at>
To make changes to your subscription:

David Fetter | 10 Aug 06:21 2015

== PostgreSQL Weekly News - August 09 2015 ==

== PostgreSQL Weekly News - August 09 2015 ==

PostgreSQL 9.5 Alpha 2 Released.  Test!

== PostgreSQL Product News ==

cstore_fdw 1.3, a columnar store extension for PostgreSQL, released.

pgbouncer 1.6, a light-weight connection pooler for PostgreSQL, released.

pgCluu 2.4, a Perl program to audit PostgreSQL performance, released.

PostgreSQL Replication, 2nd Edition, is out.

2UDA, a platform for data analytics, released.

== PostgreSQL Jobs for August ==

== PostgreSQL Local ==

PostgresOpen 2015 will being held in Dallas, Texas September 16-18.

PostgreSQL Session #7, will be held September 24th, 2015 in Paris,

PGDay.IT 2015 will take place in Prato on October 23, 2015.

PostgreSQL Conference Europe 2015 will be held on October 27-30 in the
Vienna Marriott Hotel, in Vienna, Austria.

PGConf Silicon Valley 2015 is November 17-18 at the South San
Francisco Convention Center.

PGBR2015 (The Brazilian PostgreSQL Conference) will take place in Porto
Alegre, Rio Grande do Sul, on November 18, 19 and 20.  The CfP is open
until August 31.

== PostgreSQL in the News ==

Planet PostgreSQL:

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david <at>, German language
to pwn <at>, Italian language to pwn <at>  Spanish language
to pwn <at>

== Applied Patches ==

Andrew Dunstan pushed:

- Allow TAP tests to run under Msys.  The Msys DTK perl, which is
  required to run TAP tests under Msys as a native perl won't
  recognize the correct virtual paths, has its osname recorded in the
  Config module as 'msys' instead of 'MSWin32'. To avoid having to
  repeat the test a variable is created that is true iff the osname is
  either of these values, and is then used everywhere that matters.

- Remove carriage returns from certain tap test output under Msys.
  These were causing spurious test failures.

- Allow pg_rewind tap tests to run with older File::Path versions.
  Older versions have rmtree but not remove_tree. The one-argument
  forms of these are equivalent, so replace remove_tree with rmtree.
  This allows the tests to be run on oldish Msys systems.

Tom Lane pushed:

- Fix a number of places that produced XX000 errors in the regression
  tests.  It's against project policy to use elog() for user-facing
  errors, or to omit an errcode() selection for errors that aren't
  supposed to be "can't happen" cases.  Fix all the violations of this
  policy that result in ERRCODE_INTERNAL_ERROR log entries during the
  standard regression tests, as errors that can reliably be triggered
  from SQL surely should be considered user-facing.  I also looked
  through all the files touched by this commit and fixed other nearby
  problems of the same ilk.  I do not claim to have fixed all
  violations of the policy, just the ones in these files.  In a few
  places I also changed existing ERRCODE choices that didn't seem
  particularly appropriate; mainly replacing ERRCODE_SYNTAX_ERROR by
  something more specific.  Back-patch to 9.5, but no further;
  changing ERRCODE assignments in stable branches doesn't seem like a
  good idea.

- contrib/isn now needs a .gitignore file.  Oversight in commit
  cb3384a0cb4cf900622b77865f60e31259923079.  Back-patch to 9.1, like
  that commit.

- Make modules/test_ddl_deparse/.gitignore match its siblings.  Not
  sure why /tmp_check/ was omitted from this one, but even if it isn't
  really needed right now, it's inconsistent not to include it.

- Update 9.5 release notes through today.

- Fix a PlaceHolderVar-related oversight in star-schema planning
  patch.  In commit b514a7460d9127ddda6598307272c701cbb133b7, I
  changed the planner so that it would allow nestloop paths to remain
  partially parameterized, ie the inner relation might need parameters
  from both the current outer relation and some upper-level outer
  relation.  That's fine so long as we're talking about distinct
  parameters; but the patch also allowed creation of nestloop paths
  for cases where the inner relation's parameter was a PlaceHolderVar
  whose eval_at set included the current outer relation and some
  upper-level one.  That does *not* work.  In principle we could allow
  such a PlaceHolderVar to be evaluated at the lower join node using
  values passed down from the upper relation along with values from
  the join's own outer relation.  However, nodeNestloop.c only
  supports simple Vars not arbitrary expressions as nestloop
  parameters.  createplan.c is also a few bricks shy of being able to
  handle such cases; it misplaces the PlaceHolderVar parameters in the
  plan tree, which is why the visible symptoms of this bug are "plan
  should not reference subplan's variable" and "failed to assign all
  NestLoopParams to plan nodes" planner errors.  Adding the necessary
  complexity to make this work doesn't seem like it would be repaid in
  significantly better plans, because in cases where such a PHV
  exists, there is probably a corresponding join order constraint that
  would allow a good plan to be found without using the star-schema
  exception.  Furthermore, adding complexity to nodeNestloop.c would
  create a run-time penalty even for plans where this whole
  consideration is irrelevant.  So let's just reject such paths
  instead.  Per fuzz testing by Andreas Seltenreich; the added
  regression test is based on his example query.  Back-patch to 9.2,
  like the previous patch.

- Fix bogus "out of memory" reports in tuplestore.c.  The
  tuplesort/tuplestore memory management logic assumed that the chunk
  allocation overhead for its memtuples array could not increase when
  increasing the array size.  This is and always was true for
  tuplesort, but we (I, I think) blindly copied that logic into
  tuplestore.c without noticing that the assumption failed to hold for
  the much smaller array elements used by tuplestore.  Given rather
  small work_mem, this could result in an improper complaint about
  "unexpected out-of-memory situation", as reported by Brent DeSpain
  in bug #13530.  The easiest way to fix this is just to increase
  tuplestore's initial array size so that the assumption holds.
  Rather than relying on magic constants, though, let's export a
  #define from aset.c that represents the safe allocation threshold,
  and make tuplestore's calculation depend on that.  Do the same in
  tuplesort.c to keep the logic looking parallel, even though
  tuplesort.c isn't actually at risk at present.  This will keep us
  from breaking it if we ever muck with the allocation parameters in
  aset.c.  Back-patch to all supported versions.  The error message
  doesn't occur pre-9.3, not so much because the problem can't happen
  as because the pre-9.3 tuplestore code neglected to check for it.
  (The chance of trouble is a great deal larger as of 9.3, though, due
  to changes in the array-size-increasing strategy.)  However,
  allowing LACKMEM() to become true unexpectedly could still result in
  less-than-desirable behavior, so let's patch it all the way back.

- Fix pg_dump to dump shell types.  Per discussion, it really ought to
  do this.  The original choice to exclude shell types was probably
  made in the dark ages before we made it harder to accidentally
  create shell types; but that was in 7.3.  Also, cause the standard
  regression tests to leave a shell type behind, for convenience in
  testing the case in pg_dump and pg_upgrade.  Back-patch to all
  supported branches.

- Docs: add an explicit example about controlling overall greediness
  of REs.  Per discussion of bug #13538.

- Make real sure we don't reassociate joins into or out of SEMI/ANTI
  joins.  Per the discussion in optimizer/README, it's unsafe to
  reassociate anything into or out of the RHS of a SEMI or ANTI join.
  An example from Piotr Stefaniak showed that join_is_legal() wasn't
  sufficiently enforcing this rule, so lock it down a little harder.
  I couldn't find a reasonably simple example of the optimizer trying
  to do this, so no new regression test.  (Piotr's example involved
  the random search in GEQO accidentally trying an invalid case and
  triggering a sanity check way downstream in clause selectivity
  estimation, which did not seem like a sequence of events that would
  be useful to memorialize in a regression test as-is.) Back-patch to
  all active branches.

- Further fixes for degenerate outer join clauses.  Further testing
  revealed that commit f69b4b9495269cc4 was still a few bricks shy of
  a load: minor tweaking of the previous test cases resulted in the
  same wrong-outer-join-order problem coming back.  After study I
  concluded that my previous changes in make_outerjoininfo() were just
  accidentally masking the problem, and should be reverted in favor of
  forcing syntactic join order whenever an upper outer join's
  predicate doesn't mention a lower outer join's LHS.  This still
  allows the chained-outer-joins style that is the normally
  optimizable case.  I also tightened things up some more in
  join_is_legal().  It seems to me on review that what's really
  happening in the exception case where we ignore a mismatched special
  join is that we're allowing the proposed join to associate into the
  RHS of the outer join we're comparing it to.  As such, we should
  *always* insist that the proposed join be a left join, which
  eliminates a bunch of rather dubious argumentation.  The case where
  we weren't enforcing that was the one that was already known buggy
  anyway (it had a violatable Assert before the aforesaid commit) so
  it hardly deserves a lot of deference.  Back-patch to all active
  branches, like the previous patch.  The added regression test case
  failed in all branches back to 9.1, and I think it's only an
  unrelated change in costing calculations that kept 9.0 from choosing
  a broken plan.

- Fix eclass_useful_for_merging to give valid results for appendrel
  children.  Formerly, this function would always return "true" for an
  appendrel child relation, because it would think that the appendrel
  parent was a potential join target for the child.  In principle that
  should only lead to some inefficiency in planning, but fuzz testing
  by Andreas Seltenreich disclosed that it could lead to "could not
  find pathkey item to sort" planner errors in odd corner cases.
  Specifically, we would think that all columns of a child table's
  multicolumn index were interesting pathkeys, causing us to generate
  a MergeAppend path that sorts by all the columns.  However, if any
  of those columns weren't actually used above the level of the
  appendrel, they would not get added to that rel's targetlist, which
  would result in being unable to resolve the MergeAppend's sort keys
  against its targetlist during createplan.c.  Backpatch to 9.3.  In
  older versions, columns of an appendrel get added to its targetlist
  even if they're not mentioned above the scan level, so that the
  failure doesn't occur.  It might be worth back-patching this fix to
  older versions anyway, but I'll refrain for the moment.

- Fix old oversight in join removal logic.  Commit
  9e7e29c75ad441450f9b8287bd51c13521641e3b introduced an Assert that
  join removal didn't reduce the eval_at set of any PlaceHolderVar to
  empty.  At first glance it looks like join_is_removable ensures
  that's true --- but actually, the loop in join_is_removable skips
  PlaceHolderVars that are not referenced above the join due to be
  removed.  So, if we don't want any empty eval_at sets, the right
  thing to do is to delete any now-unreferenced PlaceHolderVars from
  the data structure entirely.  Per fuzz testing by Andreas
  Seltenreich.  Back-patch to 9.3 where the aforesaid Assert was

- Further adjustments to PlaceHolderVar removal.  A new test case from
  Andreas Seltenreich showed that we were still a bit confused about
  removing PlaceHolderVars during join removal.  Specifically,
  remove_rel_from_query would remove a PHV that was used only
  underneath the removable join, even if the place where it's used was
  the join partner relation and not the join clause being deleted.
  This would lead to a "too late to create a new PlaceHolderInfo"
  error later on.  We can defend against that by checking ph_eval_at
  to see if the PHV could possibly be getting used at some partner
  rel.  Also improve some nearby LATERAL-related logic.  I decided
  that the check on ph_lateral needed to take precedence over the
  check on ph_needed, in case there's a lateral reference underneath
  the join being considered.  (That may be impossible, but I'm not
  convinced of it, and it's easy enough to defend against the case.)
  Also, I realized that remove_rel_from_query's logic for updating
  LateralJoinInfos is dead code, because we don't build those at all
  until after join removal.  Back-patch to 9.3.  Previous versions
  didn't have the LATERAL issues, of course, and they also didn't
  attempt to remove PlaceHolderInfos during join removal.  (I'm
  starting to wonder if changing that was really such a great idea.)

- Remove gram.y's precedence declaration for OVERLAPS.  The allowed
  syntax for OVERLAPS, viz "row OVERLAPS row", is sufficiently
  constrained that we don't actually need a precedence declaration for
  OVERLAPS; indeed removing this declaration does not change the
  generated gram.c file at all.  Let's remove it to avoid confusion
  about whether OVERLAPS has precedence or not.  If we ever generalize
  what we allow for OVERLAPS, we might need to put back a precedence
  declaration for it, but we might want some other level than what it
  has today --- and leaving the declaration there would just risk
  confusion about whether that would be an incompatible change.
  Likewise, remove OVERLAPS from the documentation's precedence table.
  Per discussion with Noah Misch.  Back-patch to 9.5 where we hacked
  up some nearby precedence decisions.

Heikki Linnakangas pushed:

- Clean up pg_rewind regression test script.  Since commit 01f6bb4b2, has exported path to tmp_check directory, so let's use
  that also for the pg_rewind test clusters etc.  Also, in master, the
  $tempdir_short variable has not been used since commit 13d856e17,
  which moved the initdb-running code to  Backpatch to

- Share transition state between different aggregates when possible.
  If there are two different aggregates in the query with same inputs,
  and the aggregates have the same initial condition and transition
  function, only calculate the state value once, and only call the
  final functions separately. For example, AVG(x) and SUM(x)
  aggregates have the same transition function, which accumulates the
  sum and number of input tuples.  For a query like "SELECT AVG(x),
  SUM(x) FROM x", we can therefore accumulate the state function only
  once, which gives a nice speedup.  David Rowley, reviewed and edited
  by me.

- Fix pg_rewind when pg_xlog is a symlink.  pg_xlog is often a
  symlink, typically to a different filesystem. Don't get confused and
  comlain about by that, and just always pretend that it's a normal
  directory, even if it's really a symlink.  Also add a test case for
  this.  Backpatch to 9.5.

Fujii Masao pushed:

- Make recovery rename tablespace_map to *.old if backup_label is not
  present.  If tablespace_map file is present without backup_label
  file, there is no use of such file.  There is no harm in retaining
  it, but it is better to get rid of the map file so that we don't
  have any redundant file in data directory and it will avoid any sort
  of confusion. It seems prudent though to just rename the file out of
  the way rather than delete it completely, also we ignore any error
  that occurs in rename operation as even if map file is present
  without backup_label file, it is harmless.  Back-patch to 9.5 where
  tablespace_map file was introduced.  Amit Kapila, reviewed by Robert
  Haas, Alvaro Herrera and me.

Joe Conway pushed:

- Fix psql \d output of policies.  psql neglected to wrap parenthesis
  around USING and WITH CHECK expressions -- fixed. Back-patched to
  9.5 where RLS policies were introduced.

Stephen Frost pushed:

- RLS: Keep deny policy when only restrictive exist.  Only remove the
  default deny policy when a permissive policy exists (either from the
  hook or defined by the user).  If only restrictive policies exist
  then no rows will be visible, as restrictive policies shouldn't make
  rows visible.  To address this requirement, a single "USING (true)"
  permissive policy can be created.  Update the test_rls_hooks
  regression tests to create the necessary "USING (true)" permissive
  policy.  Back-patch to 9.5 where RLS was added.  Per discussion with

Robert Haas pushed:

- Update comment to match behavior of latest code.  Peter Geoghegan

- Tab completion for CREATE SEQUENCE.  Vik Fearing, reviewed by
  Brendan Jurd, Michael Paquier, and myself

- Cap wal_buffers to avoid a server crash when it's set very large.
  It must be possible to multiply wal_buffers by XLOG_BLCKSZ without
  overflowing int, or calculations in StartupXLOG will go badly wrong
  and crash the server.  Avoid that by imposing a maximum value on
  wal_buffers.  This will be just under 2GB, assuming the usual value
  for XLOG_BLCKSZ.  Josh Berkus, per an analysis by Andrew Gierth.

- Reduce ProcArrayLock contention by removing backends in batches.
  When a write transaction commits, it must clear its XID advertised
  via the ProcArray, which requires that we hold ProcArrayLock in
  exclusive mode in order to prevent concurrent processes running
  GetSnapshotData from seeing inconsistent results.  When many
  processes try to commit at once, ProcArrayLock must change hands
  repeatedly, with each concurrent process trying to commit waking up
  to acquire the lock in turn.  To make things more efficient, when
  more than one backend is trying to commit a write transaction at the
  same time, have just one of them acquire ProcArrayLock in exclusive
  mode and clear the XIDs of all processes in the group.  Benchmarking
  reveals that this is much more efficient at very high client counts.
  Amit Kapila, heavily revised by me, with some review also from Pavan

- Fix incorrect calculation in shm_mq_receive.  If some, but not all,
  of the length word has already been read, and the next attempt to
  read sees exactly the number of bytes needed to complete the length
  word, or fewer, then we'll incorrectly read less than all of the
  available data.  Antonin Houska

- Fix attach-related race condition in shm_mq_send_bytes.  Spotted by
  Antonin Houska.

Andres Freund pushed:

- Fix comment atomics.h.  I appear to accidentally have switched the
  comments for pg_atomic_write_u32 and pg_atomic_read_u32 around. Also
  fix some minor typos I found while fixing.  Noticed-By: Amit Kapila
  Backpatch: 9.5

- Fix debug message output when connecting to a logical slot.
  Previously the message erroneously printed the same LSN twice as the
  assignment to the start_lsn variable was before the message. Correct
  that.  Reported-By: Marko Tiikkaja Author: Marko Tiikkaja Backpatch:
  9.5, where logical decoding was introduced

- Rely on inline functions even if that causes warnings in older
  compilers.  So far we have worked around the fact that some very old
  compilers do not support 'inline' functions by only using inline
  functions conditionally (or not at all). Since such compilers are
  very rare by now, we have decided to rely on inline functions from
  9.6 onwards.  To avoid breaking these old compilers inline is
  defined away when not supported. That'll cause "function x defined
  but not used" type of warnings, but since nobody develops on such
  compilers anymore that's ok.  This change in policy will allow us to
  more easily employ inline functions.  I chose to remove code
  previously conditional on PG_USE_INLINE as it seemed confusing to
  have code dependent on a define that's always defined.  Blacklisting
  of compilers, like in c53f73879f, now has to be done differently. A
  platform template can define PG_FORCE_DISABLE_INLINE to force inline
  to be defined empty.  Discussion:
  20150701161447.GB30708 <at>

- Fix typo in commit de6fd1c.  Per buildfarm members mandrill and

- Improve includes introduced in the replication origins patch.
  pg_resetxlog.h contained two superfluous includes, origin.h
  superfluously depended on logical.h, and pg_xlogdump's rmgrdesc.h
  only indirectly included origin.h.  Backpatch: 9.5, where
  replication origins were introduced.

- Address points made in post-commit review of replication origins.
  Amit reviewed the replication origins patch and made some good
  points. Address them. This fixes typos in error messages, docs and
  comments and adds a missing error check (although in a
  should-never-happen scenario).  Discussion:
  CAA4eK1JqUBVeWWKwUmBPryFaje4190ug0y-OAUHWQ6tD83V4xg <at>
  Backpatch: 9.5, where replication origins were introduced.

- Don't include low level locking code from frontend code.  Some
  frontend code like e.g. pg_xlogdump or pg_resetxlog, has to use
  backend headers. Unfortunately until now that code includes most of
  the locking code. It's generally not nice to expose such low level
  details, but de6fd1c898 made that a hard problem. We fall back to
  defining 'inline' away if the compiler doesn't support it - that can
  cause linker errors like on buildfarm animal pademelon if a inline
  function references backend only code.  To fix that problem separate
  definitions from lock.h that are required from frontend code into
  lockdefs.h and use it in the relevant places. I've only removed the
  minimal amount of necessary definitions for now - it might turn out
  that we want more for other reasons.  To avoid such details being
  exposed again put some checks against being included from frontend
  code into atomics.h, lock.h, lwlock.h and s_lock.h. It's otherwise
  fairly easy to indirectly include these headers.  Discussion:
  20150806070902.GE12214 <at>

- Attempt to work around a 32bit xlc compiler bug from a different
  place.  In de6fd1c8 I moved the the work around from 53f73879 into
  the aix template. The previous location was removed in the former
  commit, and I thought that it would be nice to emit a warning when
  running configure.  That didn't turn out to work because at the
  point the template is included we don't know whether we're compiling
  a 32/64 bit binary and it's possible to install compilers for both
  on a 64 bit kernel/OS.  So go back to a less ambitious approach and
  define PG_FORCE_DISABLE_INLINE in port/aix.h, without emitting a
  warning. We could try a more fancy approach, but it doesn't seem
  worth it.  This requires moving the check for
  PG_FORCE_DISABLE_INLINE in c.h to after including the system headers
  included from therein which isn't perfect, as it seems slightly more
  robust to include all system headers in a similar environment. Oh
  well.  Discussion: 20150807132000.GC13310 <at>

- Fix bug slowing down pgbench when -P is used.  A removed check in
  ba3deeefb made all threads but the main one busy-loop when -P was
  used. All threads computed the time to the next time the progress
  report should be printed, but only the main thread did so and
  re-scheduled it only for the future.  Reported-By: Jesper Pedersen
  Discussion: 55C4E190.3050104 <at>

Noah Misch pushed:

- Link $(WIN32RES) into single-file modules only when PGFILEDESC is
  set.  Commit 0ffc201a51395ca71fe429ef86c872850a5850ee included this
  object unconditionally.  Being unprepared for that, most external,
  single-file modules failed to build.  This better aligns the GNU
  make build system with the heuristic in the MSVC build's
  Project::AddDirResourceFile().  In-tree, installed modules set
  PGFILEDESC, so they will see no change.  Also, under PGXS, omit the
  nonfunctioning rule to build win32ver.rc.  Back-patch to 9.5, where
  the aforementioned commit first appeared.

- Reconcile nodes/*funcs.c with recent work.  A few of the
  discrepancies had semantic significance, but I did not track down
  the resulting user-visible bugs, if any.  Back-patch to 9.5, where
  all but one discrepancy appeared.  The _equalCreateEventTrigStmt()
  situation dates to 9.3 but does not affect semantics.  catversion
  bump due to readfuncs.c field order changes.

Bruce Momjian pushed:

- docs:  HTML-escape '>' in '=>' using HTML entities

- 9.5 release notes:  add non-LEAKPROOF view pushdown mention.  Report
  by Dean Rasheed.  Backpatch through 9.5

- 9.5 release notes:  adjustments suggested by Andres Freund.  Report
  by Andres Freund.  Backpatch through 9.5

- 9.5 release notes:  mention change to CRC-32C.  Report by Andres
  Freund.  Backpatch through 9.5

- 9.5 release notes:  mention ON CONFLICT DO NOTHING for FDWs.  Report
  by Peter Geoghegan.  Backpatch through 9.5

- 9.5 release notes:  updates from Andres Freund and Jeff Janes.
  Report by Andres Freund and Jeff Janes.  Backpatch through 9.5

- 9.5 release notes:  add increase buffer mapping partitions item.
  Report by Robert Haas, Andres Freund.  Backpatch through 9.5

- Document items that should appear in the major release notes

- docs:  fix typo in rules.sgml.  Report by Dean Rasheed.  Patch by
  Dean Rasheed.  Backpatch through 9.5

- docs:  update major release notes item checklist

Kevin Grittner pushed:

- Fix `make installcheck` for serializable transactions.  Commit
  e5550d5fec66aa74caad1f79b79826ec64898688 added some new tests for
  ALTER TABLE which involved table scans.  When
  default_transaction_isolation = 'serializable' these acquire
  relation-level SIReadLocks.  The test results didn't cope with that.
  Add SIReadLock as the minimum lock level for purposes of these
  tests.  This could also be fixed by excluding this type of lock from
  the my_locks view, but it would be a bug for SIReadLock to show up
  for a relation which was not otherwise locked, so do it this way to
  allow that sort of condition to cause a regression test failure.
  There is some question whether we could avoid taking SIReadLocks
  during these operations, but confirming the safety of that and
  figuring out how to avoid the locks is not trivial, and would be a
  separate patch.  Backpatch to 9.4 where the new tests were added.

Álvaro Herrera pushed:

- Fix BRIN to use SnapshotAny during summarization.  For correctness
  of summarization results, it is critical that the snapshot used
  during the summarization scan is able to see all tuples that are
  live to all transactions -- including tuples inserted or deleted by
  in-progress transactions.  Otherwise, it would be possible for a
  transaction to insert a tuple, then idle for a long time while a
  concurrent transaction executes summarization of the range: this
  would result in the inserted value not being considered in the
  summary.  Previously we were trying to use a MVCC snapshot in
  conjunction with adding a "placeholder" tuple in the index: the
  snapshot would see all committed tuples, and the placeholder tuple
  would catch insertions by any new inserters.  The hole is that prior
  insertions by transactions that are still in progress by the time
  the MVCC snapshot was taken were ignored.  Kevin Grittner reported
  this as a bogus error message during vacuum with default transaction
  isolation mode set to repeatable read (because the error report
  mentioned a function name not being invoked during), but the problem
  is larger than that.  To fix, tweak IndexBuildHeapRangeScan to have
  a new mode that behaves the way we need using SnapshotAny visibility
  rules.  This change simplifies the BRIN code a bit, mainly by
  removing large comments that were mistaken.  Instead, rely on the
  SnapshotAny semantics to provide what it needs.  (The business about
  a placeholder tuple needs to remain: that covers the case that a
  transaction inserts a a tuple in a page that summarization already
  scanned.) Discussion: <at>
  In passing, remove a couple of unused declarations from brin.h and
  reword a comment to be proper English.  This part submitted by Kevin
  Grittner.  Backpatch to 9.5, where BRIN was introduced.

Tatsuo Ishii pushed:

- Fix broken multibyte regression tests.  commit
  9043Fe390f4f0b4586cfe59cbd22314b9c3e2957 broke multibyte regression
  tests because the commit removes the warning message when temporary
  hash indexes is created, which has been added by commit
  07af523870bcfe930134054febd3a6a114942e5b.  Back patched to 9.5
  stable tree.

Magnus Hagander pushed:

- Fix typo in LDAP example.  Reported by William Meitzen

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Michael Paquier sent in a patch to improve test coverage of extensions
with pg_dump.

Anastasia Lubennikova sent in two more revisions of a patch to add
a microvacuum for GiST.

Michael Paquier sent in a patch to fix an infelicity in the TAP tests
for pg_rewind.

Jeff Janes sent in a PoC patch to fix some bloating caused by a buggy
interaction between GIN and the FSM.

David Rowley sent in a patch to make cost_agg() with AGG_HASHED
account for startup costs.

Takashi Horikawa sent in a patch to allow a wal_buffer setting over

Michael Paquier sent in a patch to enforce the lock to
AccessExclusiveLock should two locks of any subcommands differ.

Kevin Grittner sent in a patch to removed some unused declarations in
the BRIN code.

Ildus Kurbangaliev, Robert Haas, and Amit Kapila traded patches to
extend pg_stat_activity and show lwlocks.

Amit Kapila sent in two more revisions of a patch to rename the mapfile if
a backupfile is not present.

David Rowley sent in a patch to allow doing a GROUP BY before a JOIN.

Fabrízio de Royes Mello sent in three more revisions of a patch to
reduce lock level for autovac reloptions in ALTER TABLE SET...

Robert Haas sent in a patch to require that all workers have an entry
in BackendList.

Amit Langote sent in a patch to fix an issue where ON CONFLICT DO
UPDATE using EXCLUDED.column wrongly gives an error about mismatched

Stepan Rutz sent in a patch to add some hints as to which column names
are causing an issue.

Haribabu Kommi sent in a PoC patch to add a cache table.

Andres Freund sent in four more revisions of a patch to ratchet up the
compiler requirements for 9.6.

Robert Abraham sent in a patch to enable creation of GIN indexes on
very large tables by replacing repalloc with repalloc_huge in
ginCombineData in src/backend/access/ginbulk.c.

David Rowley sent in two more revisions of a patch to speed up
timestamptz out.

Robert Haas sent in a patch to bring comments on InitializeMaxBackends
in line with the current reality.

Álvaro Herrera sent in a patch to insure that BRIN pages get

Michael Paquier sent in another revision of a patch to add facilities
for new types of auth, as md5-hashed passwords may not be secure

Satoshi Nagayasu sent in a patch to add 9.5 support for pg_filedump.

Heikki Linnakangas sent in a patch sort buffers in checkpoints.

Satoshi Nagayasu sent in a patch to fix an Assert failure in
pg_stat_statements when queryId is set by other module which is loaded
prior to pg_stat_statements in the shared_preload_libraries parameter.

Tom Lane sent in a patch to SS_finalize_plan processing to the end of
planning, part of the path-ification of the upper planner.

Alexander Korotkov sent in a WIP patch to rework the access method

Daniel Verite sent in a patch to add a \pivot command to psql.

Andrew Dunstan and Michael Paquier traded patches to make TAP test
deal better with working directories.


Sent via pgsql-announce mailing list (pgsql-announce <at>
To make changes to your subscription:

Dave Page | 6 Aug 15:34 2015

PostgreSQL 9.5 Alpha 2 Released

The PostgreSQL Global Development Group announces today that the
second alpha release of PostgreSQL 9.5 is available for download.
This release contains previews of all of the features which will be
available in the final release of version 9.5, although some details
will change before then.  Please download, test, and report what you

Changes Since Alpha1

Many bugs and issues reported by our users and contributors have been
fixed since the release of Alpha1.  These include:

* Make pg_rewind work with symlinks
* Fix several issue with locking
* Numerous changes and improvements to Row Level Security
* Make pg_dump back up RLS policies
* Correct some oversights in BRIN indexes
* Fix behavior of JSON negative array subscripts
* Correct strxfrm() behavior on buggy platforms
* Multiple documentation changes and additions

If you reported an issue while testing PostgreSQL 9.5, please download
Alpha2 and test whether that issue has been fixed.  If you have not
yet tested version 9.5, now is the time for you to help out PostgreSQL

Alpha and Beta Schedule

This is the alpha release of version 9.5, indicating that some changes
to features are still possible before release.  The PostgreSQL Project
will release 9.5 another alpha or beta in September, and then
periodically release alphas or betas as required for testing until the
final release in late 2015.  For more information, and suggestions on
how to test the alpha and betas, see the Beta Testing page.

Full documentation and release notes of the new version is available
online and also installs with PostgreSQL.  Also see the What's New
page for details on some features.


* Downloads Page:

* Alpha Testing Information:

* 9.5 Alpha Release Notes:

* What's New in 9.5:

Dave Page
PostgreSQL Core Team


Sent via pgsql-announce mailing list (pgsql-announce <at>
To make changes to your subscription:

Hans-Jürgen Schönig | 5 Aug 09:57 2015

"PostgreSQL Replication" 2nd edition is out ...

I am glad to announce that the second edition of the Packt PostgreSQL Replication book has been released
last week.
We hope that it is beneficial to as many people as possible.

The book is already available as ebook on Amazon.

	Many thanks,


Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt


Sent via pgsql-announce mailing list (pgsql-announce <at>
To make changes to your subscription:

David Fetter | 3 Aug 01:36 2015

== PostgreSQL Weekly News - August 02 2015 ==

== PostgreSQL Weekly News - August 02 2015 ==

== PostgreSQL Product News ==

cstore_fdw 1.3, a columnar store extension for PostgreSQL, released.

pgbouncer 1.6, a light-weight connection pooler for PostgreSQL, released.

pgCluu 2.4, a Perl program to audit PostgreSQL performance, released.

2UDA, a platform for data analytics, released.

== PostgreSQL Jobs for August ==

== PostgreSQL Local ==

PGDay Campinas 2015 will take place in Campinas on August 7.

PostgresOpen 2015 will being held in Dallas, Texas September 16-18.

PostgreSQL Session #7, will be held September 24th, 2015 in Paris,

PGDay.IT 2015 will take place in Prato on October 23, 2015. The
International Call For Papers is open until August 8.

PostgreSQL Conference Europe 2015 will be held on October 27-30 in the
Vienna Marriott Hotel, in Vienna, Austria.  The CfP is open until
August 7.

PGConf Silicon Valley 2015 is November 17-18 at the South San
Francisco Convention Center.

PGBR2015 (The Brazilian PostgreSQL Conference) will take place in Porto
Alegre, Rio Grande do Sul, on November 18, 19 and 20.

== PostgreSQL in the News ==

Planet PostgreSQL:

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david <at>, German language
to pwn <at>, Italian language to pwn <at>  Spanish language
to pwn <at>

== Applied Patches ==

Heikki Linnakangas pushed:

- Remove false comment about speculative insertion.  There is no full
  discussion of speculative insertions in the executor README. There
  is a high-level explanation in execIndexing.c, but it doesn't seem
  necessary to refer it from here.  Peter Geoghegan

- Reuse all-zero pages in GIN.  In GIN, an all-zeros page would be
  leaked forever, and never reused. Just add them to the FSM in
  vacuum, and they will be reinitialized when grabbed from the FSM. On
  master and 9.5, attempting to access the page's opaque struct also
  caused an assertion failure, although that was otherwise harmless.
  Reported by Jeff Janes. Backpatch to all supported versions.

- Avoid calling PageGetSpecialPointer() on an all-zeros page.  That
  was otherwise harmless, but tripped the new assertion in
  PageGetSpecialPointer().  Reported by Amit Langote. Backpatch to
  9.5, where the assertion was added.

- Fix memory leak in xlogreader facility.  XLogReaderFree failed to
  free the per-block data buffers, when they happened to not be used
  by the latest read WAL record.  Michael Paquier. Backpatch to 9.5,
  where the per-block buffers were added.

- Don't assume that PageIsEmpty() returns true on an all-zeros page.
  It does currently, and I don't see us changing that any time soon,
  but we don't make that assumption anywhere else.  Per Tom Lane's
  suggestion. Backpatch to 9.2, like the previous patch that added
  this assumption.

- Fix handling of all-zero pages in SP-GiST vacuum.  SP-GiST
  initialized an all-zeros page at vacuum, but that was not
  WAL-logged, which is not safe. You might get a torn page write, when
  it gets flushed to disk, and end-up with a half-initialized index
  page. To fix, leave it in the all-zeros state, and add it to the
  FSM. It will be initialized when reused. Also don't set the
  page-deleted flag when recycling an empty page. That was also not
  WAL-logged, and a torn write of that would cause the page to have an
  invalid checksum.  Backpatch to 9.2, where SP-GiST indexes were

- Fix memory leaks in pg_rewind. Several PQclear() calls were missing.
  Originally reported by Vladimir Borodin in the pg_rewind github
  project, patch by Michael Paquier.

- Don't assume that 'char' is signed.  On some platforms, notably ARM
  and PowerPC, 'char' is unsigned by default. This fixes an assertion
  failure at WAL replay on such platforms.  Reported by Noah Misch.
  Backpatch to 9.5, where this was broken.

- Another attempt at fixing memory leak in xlogreader.  max_block_id
  is also reset between reading records.  Michael Paquier

- Fix bug in collecting total_latencies from all threads in pgbench.
  This was broken in 1bc90f7a, which removed the thread-emulation.
  With modest -j and -c settings the result were usually close enough
  that you wouldn't notice it easily, but with a high enough thread
  count it would access uninitialized memory and crash.  Per report
  from Andres Freund offlist.

- Fix typo in comment.  Amit Langote

- Fix TAP tests with "make installcheck".  I neglected that the
  prove_installcheck rule also needs to also define PG_REGRESS, like
  prove_check does.

- Make TAP tests work on Windows.  On Windows, use
  listen_address= to allow TCP connections. We were already
  using "pg_regress --config-auth" to set up HBA appropriately. The
  standard_initdb helper function now sets up the server's
  unix_socket_directories or listen_addresses in the config file, so
  that they don't need to be specified in the pg_ctl command line
  anymore. That way, the pg_ctl invocations in test programs don't
  need to differ between Windows and Unix.  Add another helper
  function to configure the server's pg_hba.conf to allow replication
  connections. The configuration is done similarly to "pg_regress
  --config-auth": trust on domain sockets on Unix, and SSPI
  authentication on Windows.  Replace calls to "cat" and "touch"
  programs with built-in perl code, as those programs don't normally
  exist on Windows.  Add instructions in the docs on how to install
  IPC::Run on Windows. Adjust to not replace PERL5LIB
  completely in, because otherwise cannot install
  IPC::Run in a non-standard location easily.  Michael Paquier,
  reviewed by Noah Misch, some additional tweaking by me.

- Update ax_pthread.m4 to an experimental draft version from upstream.
  The current version is adding a spurious -pthread option on some
  Darwin systems that don't need it, which leads to a bunch of
  "unrecognized option '-pthread'" warnings. There is a proposed fix
  for that in the upstream autoconf archive's bug tracker, see  This commit updates our
  version of ax_pthread.m4 to the "draft2" version proposed there by
  Daniel Richard G. I'm using our buildfarm to help Daniel to test
  this, before he commits this to the upstream repository.

- Fix calculation of latency of pgbench backslash commands.  When we
  loop back to the top of doCustom after processing a backslash
  command, we must reset the "now" timestamp, because that's used to
  calculate the time spent executing the previous command.  Report and
  fix by Fabien Coelho. Backpatch to 9.5, where this was broken.

- Fix race condition that lead to WALInsertLock deadlock with
  commit_delay.  If a call to WaitForXLogInsertionsToFinish() returned
  a value in the middle of a page, and another backend then started to
  insert a record to the same page, and then you called
  WaitXLogInsertionsToFinish() again, the second call might return a
  smaller value than the first call. The problem was in
  GetXLogBuffer(), which always updated the insertingAt value to the
  beginning of the requested page, not the actual requested location.
  Because of that, the second call might return a xlog pointer to the
  beginning of the page, while the first one returned a later position
  on the same page.  XLogFlush() performs two calls to
  WaitXLogInsertionsToFinish() in succession, and holds WALWriteLock
  on the second call, which can deadlock if the second call to
  WaitXLogInsertionsToFinish() blocks.  Reported by Spiros Ioannou.
  Backpatch to 9.4, where the more scalable WALInsertLock mechanism,
  and this bug, was introduced.

- Fix output of ISBN-13 numbers beginning with 979.  An EAN beginning
  with 979 (but not 9790 - those are ISMN's) are accepted as ISBN
  numbers, but they cannot be represented in the old, 10-digit ISBN
  format. They must be output in the new 13-digit ISBN-13 format. We
  printed out an incorrect value for those.  Also add a regression
  test, to test this and some other basic functionality of the module.
  Patch by Fabien Coelho. This fixes bug #13442, reported by B.Z.
  Backpatch to 9.1, where we started to recognize ISBN-13 numbers.

Tom Lane pushed:

- Fix pointer-arithmetic thinko in pg_stat_ssl patch.  Nasty
  memory-stomp bug in commit 9029f4b37406b21a.  It's not apparent how
  this survived even cursory testing :-(.  Per report from Peter

- Further code review for pg_stat_ssl patch.  Fix additional bogosity
  in commit 9029f4b37406b21a.  Include the BackendSslStatusBuffer in
  the BackendStatusShmemSize calculation, avoid ugly and error-prone
  casts to char* and back, put related code stanzas into a consistent
  order (and fix a couple of previous instances of that sin).  All
  cosmetic except for the size oversight.

- Remove an unsafe Assert, and explain join_clause_is_movable_into()
  better.  join_clause_is_movable_into() is approximate, in the sense
  that it might sometimes return "false" when actually it would be
  valid to push the given join clause down to the specified level.
  This is okay ... but there was an Assert in
  get_joinrel_parampathinfo() that's only safe if the answers are
  always exact.  Comment out the Assert, and add a bunch of commentary
  to clarify what's going on.  Per fuzz testing by Andreas
  Seltenreich.  The added regression test is a pretty silly query, but
  it's based on his crasher example.  Back-patch to 9.2 where the
  faulty logic was introduced.

- Reduce chatter from signaling of autovacuum workers.  Don't print a
  WARNING if we get ESRCH from a kill() that's attempting to cancel an
  autovacuum worker.  It's possible (and has been seen in the
  buildfarm) that the worker is already gone by the time we are able
  to execute the kill, in which case the failure is harmless.  About
  the only plausible reason for reporting such cases would be to help
  debug corrupted lock table contents, but this is hardly likely to be
  the most important symptom if that happens.  Moreover issuing a
  WARNING might scare users more than is warranted.  Also, since
  sending a signal to an autovacuum worker is now entirely a routine
  thing, and the worker will log the query cancel on its end anyway,
  reduce the message saying we're doing that from LOG to DEBUG1 level.
  Very minor cosmetic cleanup as well.  Since the main practical
  reason for doing this is to avoid unnecessary buildfarm failures,
  back-patch to all active branches.

- Update our documentation concerning where to create data
  directories.  Although initdb has long discouraged use of a
  filesystem mount-point directory as a PG data directory, this point
  was covered nowhere in the user-facing documentation.  Also, with
  the popularity of pg_upgrade, we really need to recommend that the
  PG user own not only the data directory but its parent directory
  too.  (Without a writable parent directory, operations such as "mv
  data data.old" fail immediately.  pg_upgrade itself doesn't do that,
  but wrapper scripts for it often do.) Hence, adjust the "Creating a
  Database Cluster" section to address these points.  I also took the
  liberty of wordsmithing the discussion of NFS a bit.  These
  considerations aren't by any means new, so back-patch to all
  supported branches.

- Suppress "variable may be used uninitialized" warning.  Also
  re-pgindent, just because I'm a neatnik.

- Prevent platform-dependent output row ordering in a new test query.
  Buildfarm indicates this is necessary.

- Add some test coverage of EvalPlanQual with non-locked tables.  A
  Salesforce colleague of mine griped that the regression tests don't
  exercise EvalPlanQualFetchRowMarks() and allied routines.  Which is
  a fair complaint.  Add test cases that go through the REFERENCE and
  COPY code paths.  Unfortunately we don't have sufficient
  infrastructure right now to exercise the FDW code path in the
  isolation tests, but this is surely better than before.

- Avoid some zero-divide hazards in the planner.  Although I think on
  all modern machines floating division by zero results in Infinity
  not SIGFPE, we still don't want infinities running around in the
  planner's costing estimates; too much risk of that leading to insane
  behavior.  grouping_planner() failed to consider the possibility
  that final_rel might be known dummy and hence have zero rowcount.
  (I wonder if it would be better to set a rows estimate of 1 for
  dummy relations?  But at least in the back branches, changing this
  convention seems like a bad idea, so I'll leave that for another
  day.) Make certain that get_variable_numdistinct() produces a
  nonzero result.  The case that can be shown to be broken is with
  stadistinct < 0.0 and small ntuples; we did not prevent the result
  from rounding to zero.  For good luck I applied clamp_row_est() to
  all the nonconstant return values.  In ExecChooseHashTableSize(),
  Assert that we compute positive nbuckets and nbatch.  I know of no
  reason to think this isn't the case, but it seems like a good safety
  check.  Per reports from Piotr Stefaniak.  Back-patch to all active

- Fix an oversight in checking whether a join with LATERAL refs is
  legal.  In many cases, we can implement a semijoin as a plain
  innerjoin by first passing the righthand-side relation through a
  unique-ification step.  However, one of the cases where this does
  NOT work is where the RHS has a LATERAL reference to the LHS; that
  makes the RHS dependent on the LHS so that unique-ification is
  meaningless.  joinpath.c understood this, and so would not generate
  any join paths of this kind ... but join_is_legal neglected to check
  for the case, so it would think that we could do it.  The upshot
  would be a "could not devise a query plan for the given query"
  failure once we had failed to generate any join paths at all for the
  bogus join pair.  Back-patch to 9.3 where LATERAL was added.

- Teach predtest.c that "foo" implies "foo IS NOT NULL".  Per
  complaint from Peter Holzer.  It's useful to cover this special
  case, since for a boolean variable "foo", earlier parts of the
  planner will have reduced variants like "foo = true" to just "foo",
  and thus we may fail to recognize the applicability of a partial
  index with predicate "foo IS NOT NULL".  Back-patch to 9.5, but not
  further; given the lack of previous complaints this doesn't seem
  like behavior to change in stable branches.

- Fix some planner issues with degenerate outer join clauses.  An
  outer join clause that didn't actually reference the RHS (perhaps
  only after constant-folding) could confuse the join order
  enforcement logic, leading to wrong query results.  Also, nested
  occurrences of such things could trigger an Assertion that on
  reflection seems incorrect.  Per fuzz testing by Andreas
  Seltenreich.  The practical use of such cases seems thin enough that
  it's not too surprising we've not heard field reports about it.
  This has been broken for a long time, so back-patch to all active

- Fix incorrect order of lock file removal and failure to close()
  sockets.  Commit c9b0cbe98bd783e24a8c4d8d8ac472a494b81292
  accidentally broke the order of operations during postmaster
  shutdown: it resulted in removing the per-socket lockfiles after,
  not before,  This creates a race-condition hazard
  for a new postmaster that's started immediately after observing that has disappeared; if it sees the socket lockfile still
  present, it will quite properly refuse to start.  This error appears
  to be the explanation for at least some of the intermittent
  buildfarm failures we've seen in the pg_upgrade test.  Another
  problem, which has been there all along, is that the postmaster has
  never bothered to close() its listen sockets, but has just allowed
  them to close at process death.  This creates a different race
  condition for an incoming postmaster: it might be unable to bind to
  the desired listen address because the old postmaster is still
  incumbent.  This might explain some odd failures we've seen in the
  past, too.  (Note: this is not related to the fact that individual
  backends don't close their client communication sockets.  That
  behavior is intentional and is not changed by this patch.) Fix by
  adding an on_proc_exit function that closes the postmaster's ports
  explicitly, and (in 9.3 and up) reshuffling the responsibility for
  where to unlink the Unix socket files.  Lock file unlinking can stay
  where it is, but teach it to unlink the lock files in reverse order
  of creation.

- Avoid calling memcpy() with a NULL source pointer and count == 0.
  As in commit 0a52d378b03b7d5a, avoid doing something that has
  undefined results according to the C standard, even though in
  practice there does not seem to be any problem with it.  This fixes
  two places in numeric.c that demonstrably could call memcpy() with
  such arguments.  I looked through that file and didn't see any other
  places with similar hazards; this is not to claim that there are not
  such places in other files.  Per report from Piotr Stefaniak.
  Back-patch to 9.5 which is where the previous commit was added.
  We're more or less setting a precedent that we will not worry about
  this type of issue in pre-9.5 branches unless someone demonstrates a
  problem in the field.

Stephen Frost pushed:

- Improve RLS handling in copy.c To avoid a race condition where the
  relation being COPY'd could be changed into a view or otherwise
  modified, keep the original lock on the relation.  Further, fully
  qualify the relation when building the query up.  Also remove the
  poorly thought-out Assert() and check the entire relationOids list
  as, post-RLS, there can certainly be multiple relations involved and
  the planner does not guarantee their ordering.  Per discussion with
  Noah and Andres.  Back-patch to 9.5 where RLS was introduced.

Joe Conway pushed:

- Fix pg_dump output of policies.  pg_dump neglected to wrap
  parenthesis around USING and WITH CHECK expressions -- fixed.
  Reported by Noah Misch.

- Plug RLS related information leak in pg_stats view.  The pg_stats
  view is supposed to be restricted to only show rows about tables the
  user can read. However, it sometimes can leak information which
  could not otherwise be seen when row level security is enabled. Fix
  that by not showing pg_stats rows to users that would be subject to
  RLS on the table the row is related to. This is done by
  creating/using the newly introduced SQL visible function,
  row_security_active().  Along the way, clean up three call sites of
  check_enable_rls(). The second argument of that function should only
  be specified as other than InvalidOid when we are checking as a
  different user than the current one, as in when querying through a
  view. These sites were passing GetUserId() instead of InvalidOid,
  which can cause the function to return incorrect results if the
  current user has the BYPASSRLS privilege and row_security has been
  set to OFF.  Additionally fix a bug causing RI Trigger error
  messages to unintentionally leak information when RLS is enabled,
  and other minor cleanup and improvements. Also add WITH
  (security_barrier) to the definition of pg_stats.  Bumped CATVERSION
  due to new SQL functions and pg_stats view definition.  Back-patch
  to 9.5 where RLS was introduced. Reported by Yaroslav.  Patch by Joe
  Conway and Dean Rasheed with review and input by Michael Paquier and
  Stephen Frost.

- Bump catversion so that HEAD is beyond 9.5.  As pointed out by Tom,
  since HEAD has progressed beyond 9.5 in terms of its catalog, we
  need to be sure catversion of HEAD is advanced beyond that of 9.5.
  Corrects my mistake in the pg_stats view commit cfa928ff.

- Create a pg_shdepend entry for each role in TO clause of policies.
  CreatePolicy() and AlterPolicy() omit to create a pg_shdepend entry
  for each role in the TO clause. Fix this by creating a new shared
  dependency type called SHARED_DEPENDENCY_POLICY and assigning it to
  each role.  Reported by Noah Misch. Patch by me, reviewed by Alvaro
  Herrera.  Back-patch to 9.5 where RLS was introduced.

- Disallow converting a table to a view if row security is present.
  When DefineQueryRewrite() is about to convert a table to a view, it
  checks the table for features unavailable to views.  For example, it
  rejects tables having triggers.  It omits to reject tables having
  relrowsecurity or a pg_policy record. Fix that. To faciliate the
  repair, invent relation_has_policies() which indicates the presence
  of policies on a relation even when row security is disabled for
  that relation.  Reported by Noah Misch. Patch by me, review by
  Stephen Frost. Back-patch to 9.5 where RLS was introduced.

- Add missing post create and alter hooks to policy objects.
  AlterPolicy() and CreatePolicy() lacked their respective hook
  invocations.  Noted by Noah Misch, review by Dean Rasheed.
  Back-patch to 9.5 where RLS was introduced.

- Create new ParseExprKind for use by policy expressions.  Policy
  USING and WITH CHECK expressions were using EXPR_KIND_WHERE for
  parse analysis, which results in inappropriate ERROR messages when
  the expression contains unsupported constructs such as aggregates.
  Create a new ParseExprKind called EXPR_KIND_POLICY and tailor the
  related messages to fit.  Reported by Noah Misch. Reviewed by Dean
  Rasheed, Alvaro Herrera, and Robert Haas. Back-patch to 9.5 where
  RLS was introduced.

- Use appropriate command type when retrieving relation's policies.
  When retrieving policies, if not working on the root target
  relation, we actually want the relation's SELECT policies,
  regardless of the top level query command type. For example in
  UPDATE t1...FROM t2 we need to apply t1's UPDATE policies and t2's
  SELECT policies.  Previously top level query command type was
  applied to all relations, which was wrong. Add some regression
  coverage to ensure we don't violate this principle in the future.
  Report and patch by Dean Rasheed. Cherry picked from larger
  refactoring patch and tweaked by me. Back-patched to 9.5 where RLS
  was introduced.

- Improve CREATE FUNCTION doc WRT to LEAKPROOF RLS interaction.  Patch
  by Dean Rasheed. Back-patched to 9.5 where RLS was introduced.

Robert Haas pushed:

- Centralize decision-making about where to get a backend's PGPROC.
  This code was originally written as part of parallel query effort,
  but it seems to have independent value, because if we make one
  decision about where to get a PGPROC when we allocate and then put
  it back on a different list at backend-exit time, bad things happen.
  This isn't just a theoretical risk; we fixed an actual problem of
  this type in commit e280c630a87e1b8325770c6073097d109d79a00f.

- Document how to build the docs using the website style.  Craig

- Flesh out the background worker documentation.  Make it more clear
  that bgw_main is usually not what you want.  Put the background
  worker flags in a variablelist rather than having them as part of a
  paragraph.  Explain important limits on how bgw_main_arg can be
  used.  Craig Ringer, substantially revised by me.

- Fix incorrect comment.  Amit Langote

Andrew Dunstan pushed:

- Make tap tests store postmaster logs and handle vpaths correctly.
  Given this it is possible that the buildfarm animals running these
  tests will be able to capture adequate logging to allow diagnosis of

- Only adjust negative indexes in json_get up to the length of the
  path.  The previous code resulted in memory access beyond the path
  bounds. The cure is to move it into a code branch that checks the
  value of lex_level is within the correct bounds.  Bug reported and
  diagnosed by Piotr Stefaniak.

- Add IF NOT EXISTS processing to ALTER TABLE ADD COLUMN.  Fabrízio de
  Royes Mello, reviewed by Payal Singh, Alvaro Herrera and Michael

Andres Freund pushed:

- Remove ssl renegotiation support.  While postgres' use of SSL
  renegotiation is a good idea in theory, it turned out to not work
  well in practice. The specification and openssl's implementation of
  it have lead to several security issues. Postgres' use of
  renegotiation also had its share of bugs.  Additionally OpenSSL has
  a bunch of bugs around renegotiation, reported and open for years,
  that regularly lead to connections breaking with obscure error
  messages. We tried increasingly complex workarounds to get around
  these bugs, but we didn't find anything complete.  Since these
  connection breakages often lead to hard to debug problems, e.g.
  spuriously failing base backups and significant latency spikes when
  synchronous replication is used, we have decided to change the
  default setting for ssl renegotiation to 0 (disabled) in the
  released backbranches and remove it entirely in 9.5 and master.
  Author: Andres Freund Discussion:
  20150624144148.GQ4797 <at> Backpatch: 9.5 and master,
  9.0-9.4 get a different patch

- Remove outdated comment in LWLockDequeueSelf's header.  Noticed-By:
  Robert Haas.  Backpatch: 9.5, where the function was added.

- Fix issues around the "variable" support in the lwlock
  infrastructure.  The lwlock scalability work introduced two race
  conditions into the lwlock variable support provided for xlog.c.
  First, and harmlessly on most platforms, it set/read the variable
  without the spinlock in some places. Secondly, due to the removal of
  the spinlock, it was possible that a backend missed changes to the
  variable's state if it changed in the wrong moment because checking
  the lock's state, the variable's state and the queuing are not
  protected by a single spinlock acquisition anymore.  To fix first
  move resetting the variable's from LWLockAcquireWithVar to
  WALInsertLockRelease, via a new function LWLockReleaseClearVar. That
  prevents issues around waiting for a variable's value to change when
  a new locker has acquired the lock, but not yet set the value.
  Secondly re-check that the variable hasn't changed after enqueing,
  that prevents the issue that the lock has been released and already
  re-acquired by the time the woken up backend checks for the lock's
  state.  Reported-By: Jeff Janes.  Analyzed-By: Heikki Linnakangas.
  Reviewed-By: Heikki Linnakangas.  Discussion:
  5592DB35.2060401 <at>  Backpatch: 9.5, where the lwlock
  scalability went in.

- Micro optimize LWLockAttemptLock() a bit.  LWLockAttemptLock
  pointlessly read the lock's state in every loop iteration, even
  though pg_atomic_compare_exchange_u32() returns the old value.
  Instead do that only once before the loop iteration.  Additionally
  there's no need to have the expected_state variable, old_state
  mostly had the same value anyway.  Noticed-By: Heikki Linnakangas.
  Backpatch: 9.5, no reason to let the branches diverge at this point.

Peter Eisentraut pushed:

- pg_basebackup: Add tests for -R option.  Reviewed-by: Michael
  Paquier <michael.paquier <at>>

- pg_basebackup: Add --slot option.  This option specifies a
  replication slot for WAL streaming (-X stream), so that there can be
  continuous replication slot use between WAL streaming during the
  base backup and the start of regular streaming replication.
  Reviewed-by: Michael Paquier <michael.paquier <at>>

- pg_basebackup: Add tests for -X option.  Reviewed-by: Michael
  Paquier <michael.paquier <at>>

Noah Misch pushed:

- MSVC: Future-proof installation file skip logic.  This code relied
  on knowing exactly where in the source tree temporary installations
  might appear.  A reasonable hacker may not think to update this code
  when adding use of a temporary installation, making it fragile.
  Observe that commit 9fa8b0ee90c44c0f97d16bf65e94322988c94864 broke
  it unnoticed, and commit dcae5faccab64776376d354decda0017c648bb53
  fixed it unnoticed.  Back-patch to 9.5 only; use of temporary
  installations is unlikely to change in released versions.

- Blacklist xlc 32-bit inlining.  Per a suggestion from Tom Lane.
  Back-patch to 9.0 (all supported versions).  While only 9.4 and up
  have code known to elicit this compiler bug, we were disabling
  inlining by accident until commit

- MSVC: Revert most 9.5 changes to pre-9.5 tests.  The
  reverted changes did not narrow the semantic gap between the MSVC
  build system and the GNU make build system.  For targets old and new
  that run multiple suites (contribcheck, modulescheck, tapcheck),
  restore to mimicking "make -k" rather than the "make
  -S" default.  Lack of "-k" would be more burdensome than lack of
  "-S".  Keep changes reflecting contemporary changes to the GNU make
  build system, and keep updates to Makefile parsing.  Keep the loss
  of --psqldir in "check" and "ecpgcheck" targets; it had been a no-op
  when used alongside --temp-install.  No log message mentioned any of
  the reverted changes.  Based on a germ by Michael Paquier.
  Back-patch to 9.5.

- Remove redundant "make install" from pg_upgrade test suite.  A
  top-level "make install" includes pg_upgrade since commit
  9fa8b0ee90c44c0f97d16bf65e94322988c94864.  Back-patch to 9.5, where
  that commit first appeared.

- MSVC: Remove duplicate PATH entry in test harness.  Back-patch to
  9.5, where commit 4cb7d671fddc8855c8def2de51fb23df1c8ac0af
  introduced it.

- Clean up Makefile.win32 "-I" flag additions.  The PGXS-case
  directory does not exist in the non-PGXS case, and vice versa.  Add
  one or the other, not both.  This is essentially cosmetic.  It makes
  Makefile.win32 more like the similar code.

- Consolidate makefile code for setting top_srcdir, srcdir and VPATH.
  Responsibility was formerly split between and  As a result of commit
  b58233c71b93a32fcab7219585cafc25a27eb769, in the PGXS case, these
  variables were unset while parsing and callees.
  Inclusion of Makefile.custom did not work from PGXS, and the subtle
  difference seemed like a recipe for future bugs.  Back-patch to 9.4,
  where that commit first appeared.

Álvaro Herrera pushed:

- Fix broken assertion in BRIN code.  The code was assuming that any
  NULL value in scan keys was due to IS NULL or IS NOT NULL, but it
  turns out to be possible to get them with other operators too, if
  they are used in contrived-enough ways.  Easiest way out of the
  problem seems to check explicitely for the IS NOT NULL flag, instead
  of assuming it must be set if the IS NULL flag is not set, when a
  null scan key is found; if neither flag is set, follow the lead of
  other index AMs and assume that all indexable operators must be
  strict, and thus the query is never satisfiable.  Also, add a
  comment to try and lure some future hacker into improving analysis
  of scan keys in brin.  Per report from Andreas Seltenreich;
  diagnosis by Tom Lane.  Backpatch to 9.5.  Discussion: <at>

- Fix volatility marking of commit timestamp functions.  They are
  marked stable, but since they act on instantaneous state and it is
  possible to consult state of transactions as they commit, the
  results could change mid-query.  They need to be marked volatile,
  and this commit does so.  There would normally be a catversion bump
  here, but this is so much a niche feature and I don't believe
  there's real damage from the incorrect marking, that I refrained.
  Backpatch to 9.5, where commit timestamps where introduced.  Per
  note from Fujii Masao.

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Thomas Munro sent in a doc patch to highlight the fact that the
special logic for "<row-valued-expression> IS [ NOT ] NULL" doesn't
apply anywhere else that we handle nulls or talk about [non]-null
values in the manual.

David Rowley and Heikki Linnakangas traded patches to allow multiple
aggregate functions in a query share state.

Ildus Kurbangaliev sent in another revision of a patch to replace
pg_stat_activity.waiting with something more descriptive.

Andres Freund and David Rowley traded patches to make timestamptz less

Fabien COELHO sent in a patch to fix some cases where pgbench could
produce negative latencies.

Qingqing Zhou sent in a patch to add some planner debugging

Fabien COELHO sent in two more revisions of a patch to add pgbench
stats per script, etc.

SAWADA Masahiko sent in another revision of a patch to add a "frozen"
bit to the visibility map.

Pavel Stěhule sent in a patch to allow multiple commands to be
specified separately on the command line to psql.

Michael Paquier sent in another revision of a patch to add TAP tests
for MSVC builds.

Michael Paquier sent in two revisions of a patch to prevent starting
streaming after creating a slot in pg_receivexlog.

Michael Paquier sent in a patch to add a TAP test for pg_dump checking
data dump of extension tables.

Alexander Shulgin sent in a patch to add ddl_deparse core support.

Ashutosh Bapat sent in another revision of a patch to allow
transaction control across multiple nodes with FDWs.

Peter Geoghegan sent in two revisions of a patch to use quicksort and
a merge step to significantly improve on tuplesort's single run
"external sort".

Heikki Linnakangas sent in a patch to allow pgmemcache to create a
temporary installation.

Kaigai Kouhei sent in a patch to fix a bug where
customScan->custom_plans were not copied.

Tom Lane sent in a patch to add type-of-expression notation.

Kyotaro HORIGUCHI sent in another revision of a patch to add
multivariate statistics.

Anastasia Lubennikova sent in a patch to add microvacuum support for
the GiST access method.

Amit Kapila sent in another flock of patches intended to reduce
ProcArrayLock contention.

Michael Paquier sent in another revision of a patch to implement

Josh Berkus sent in a patch to prevent users from setting wal_buffers
over 2GB bytes.

Fabrízio de Royes Mello sent in three more revisions of a patch to
change ALTER TABLE SET to reduce the lock level for autovac

Jesper Pedersen sent in two revisions of a patch to require extensions
to specify the correct number of LWLock's needed.

Álvaro Herrera sent in a patch to fix some infelicities between brin
index vacuum and transaction snapshots.

Tom Lane sent in a patch to clean up the missing ERRCODE assignments.

Peter Geoghegan sent in a patch to fix an incorrect comment about
abbreviated keys.

Nikolay Shaplov sent in a patch to for pageinspect that allows seeing
data stored in the tuple.

Noah Misch sent in a patch to fix some nodes/*funcs.c inconsistencies.


Sent via pgsql-announce mailing list (pgsql-announce <at>
To make changes to your subscription:

Dave Page | 28 Jul 04:12 2015

Reminder: Call for Papers - PostgreSQL Conference Europe 2015

PostgreSQL Conference Europe 2015 will be held on October 27-30 in the
Vienna Marriott Hotel, in Vienna, Austria. It will cover topics for
PostgreSQL users, developers and contributors, as well as decision and
policy makers. For more information about the conference, please see
the website at

We are now accepting proposals for talks, in english or german. If you
are submitting a talk in german, please write a short summary of the
abstract in English in
the "submission notes" field.

Each session will last 45 minutes, and may be on any topic related to
PostgreSQL. Suggested topic areas include:

Developing applications for PostgreSQL
Administering large scale PostgreSQL installations
Case studies and/or success stories of PostgreSQL deployments
PostgreSQL tools and utilities
PostgreSQL hacking
Community & user groups
Tuning the server
Migrating from other systems
Benchmarking & hardware
PostgreSQL related products

Of course, we're happy to receive proposals for talks on other
PostgreSQL related topics as well.

We may also have a limited number of longer, 90-minute, slots
available. Please indicate clearly in your submission if you wish to
make a 90-minute talk.

Finally, there will be a session of five minute lightning talks. A
separate call for proposals will be made for them further on.

The submission deadline is August 7th. Selected speakers will be
notified before August 24th, 2015.

Please submit your proposals by going to and following the instructions

The proposals will be considered by committee who will produce a
schedule to be published nearer the conference date.

All selected speakers will get free entry to the conference (excluding
training sessions). We do not in general cover travel and
accommodations for speakers, but may be able to do that in limited
cases. If you require assistance with funding to be able to attend,
please make a note of this in the submission notes field.

This call for papers is also available on the web at

We look forward to hearing from you, and seeing you in Vienna in October!

Dave Page
PostgreSQL Core Team


Sent via pgsql-announce mailing list (pgsql-announce <at>
To make changes to your subscription:

Petr Jelinek | 1 Aug 16:55 2015

PgBouncer 1.6

Main new features:

* Load user/password hash from Postgres database.
* Pooling mode can be configured both per-database and per-user.
* Per-database and per-user connection limits: max_db_connections and
* Add DISABLE/ENABLE commands to prevent new connections.
* New preferred DNS backend: c-ares.
* Config files have '%include FILENAME' directive to allow
   configuration to be split into several files.

Full details:


PgBouncer is lightweight connection pooler for PostgreSQL.


  Petr Jelinek        
  PostgreSQL Development, 24x7 Support, Training & Services


Sent via pgsql-announce mailing list (pgsql-announce <at>
To make changes to your subscription: