Mark Sauer | 3 Mar 01:25 2009
Picon

Question about changing x.264 bitrate on the fly

I am looking at creating an application of x.264, but I would like to be 
able to specify the bit rate of different sections of the video sequence. 

IE, I'd like to be able to say

frames 1-300 are at 600 kbps
frames 301-450 are at 1200 kbps
frames 451-900 are at 900 kbps

and so on.  I noticed there is an interface to the library 
"x264_encoder_reconfig", but the body doesn't seem to allow for changes 
to the bitrate.

Would it be safe to do something like:

x264_encoder_open( set bitrate to 600 );

for(i=0;i<300;i++) {
   x264_encoder_encode( frame i );
   store the nal blocks in a file, etc
}

x264_encoder_close();
x264_encoder_open( set bitrate to 1200 );

for(i=300;i<450;i++) {
   x264_encoder_encode( frame i );
   store the nal blocks in a file, etc
}

(Continue reading)

David Wolstencroft | 3 Mar 01:39 2009
Picon

Re: Question about changing x.264 bitrate on the fly

x264 --help 


Look at "--zones"

On Mon, Mar 2, 2009 at 4:25 PM, Mark Sauer <mark <at> skitter.tv> wrote:
I am looking at creating an application of x.264, but I would like to be able to specify the bit rate of different sections of the video sequence.
IE, I'd like to be able to say

frames 1-300 are at 600 kbps
frames 301-450 are at 1200 kbps
frames 451-900 are at 900 kbps

and so on.  I noticed there is an interface to the library "x264_encoder_reconfig", but the body doesn't seem to allow for changes to the bitrate.

Would it be safe to do something like:

x264_encoder_open( set bitrate to 600 );

for(i=0;i<300;i++) {
 x264_encoder_encode( frame i );
 store the nal blocks in a file, etc
}

x264_encoder_close();
x264_encoder_open( set bitrate to 1200 );

for(i=300;i<450;i++) {
 x264_encoder_encode( frame i );
 store the nal blocks in a file, etc
}

x264_encoder_close();
x264_encoder_open( set bitrate to 900 );

for(i=451;i<900;i++) {
 x264_encoder_encode( frame i );
 store the nal blocks in a file, etc
}

x264_encoder_close();

Or would there possibly be some issues with this approach?
Thanks
Mark Sauer
_______________________________________________
x264-devel mailing list
x264-devel <at> videolan.org
http://mailman.videolan.org/listinfo/x264-devel

_______________________________________________
x264-devel mailing list
x264-devel <at> videolan.org
http://mailman.videolan.org/listinfo/x264-devel
191919 | 3 Mar 10:32 2009
Picon

libx264 memory leak

I found a memory leak in x264_mb_analyse_load_costs. It keeps some x264_malloc'ed pointer to a global array, but no one remembers to free them in x264_encoder_close.

 184     if( !p_cost_mv[a->i_qp] )
 185     {
 186         x264_emms();
 187         /* could be faster, but isn't called many times */
 188         /* factor of 4 from qpel, 2 from sign, and 2 because mv can be opposite from mvp */
 189         p_cost_mv[a->i_qp] = x264_malloc( (4*4*2048 + 1) * sizeof(int16_t) );
 190         p_cost_mv[a->i_qp] += 2*4*2048;
 191         for( i = 0; i <= 2*4*2048; i++ )

and

 207         for( j=0; j<4; j++ )
 208         {
 209             x264_cost_mv_fpel[a->i_qp][j] = x264_malloc( (4*2048 + 1) * sizeof(int16_t) );
 210             x264_cost_mv_fpel[a->i_qp][j] += 2*2048;
 211             for( i = -2*2048; i < 2*2048; i++ )
 212                 x264_cost_mv_fpel[a->i_qp][j][i] = p_cost_mv[a->i_qp][i*4+j];
 213         }

valgrind reported:

==54848== 917,532 bytes in 14 blocks are possibly lost in loss record 10 of 10
==54848==    at 0xA61D63: malloc (vg_replace_malloc.c:192)
==54848==    by 0x403B24: x264_mb_analyse_load_costs (in /usr/local/bin/ffmpeg)
==54848==    by 0x3FF87D2D: ???

Regards,
jh

_______________________________________________
x264-devel mailing list
x264-devel <at> videolan.org
http://mailman.videolan.org/listinfo/x264-devel
Jason Garrett-Glaser | 3 Mar 11:19 2009
Picon

Re: libx264 memory leak

2009/3/3 191919 <191919 <at> gmail.com>:
> I found a memory leak in x264_mb_analyse_load_costs. It keeps some
> x264_malloc'ed pointer to a global array, but no one remembers to free them
> in x264_encoder_close.
>
>  184     if( !p_cost_mv[a->i_qp] )
>  185     {
>  186         x264_emms();
>  187         /* could be faster, but isn't called many times */
>  188         /* factor of 4 from qpel, 2 from sign, and 2 because mv can be
> opposite from mvp */
>  189         p_cost_mv[a->i_qp] = x264_malloc( (4*4*2048 + 1) *
> sizeof(int16_t) );
>  190         p_cost_mv[a->i_qp] += 2*4*2048;
>  191         for( i = 0; i <= 2*4*2048; i++ )
>
> and
>
>  207         for( j=0; j<4; j++ )
>  208         {
>  209             x264_cost_mv_fpel[a->i_qp][j] = x264_malloc( (4*2048 + 1) *
> sizeof(int16_t) );
>  210             x264_cost_mv_fpel[a->i_qp][j] += 2*2048;
>  211             for( i = -2*2048; i < 2*2048; i++ )
>  212                 x264_cost_mv_fpel[a->i_qp][j][i] =
> p_cost_mv[a->i_qp][i*4+j];
>  213         }
>
> valgrind reported:
>
> ==54848== 917,532 bytes in 14 blocks are possibly lost in loss record 10 of
> 10
> ==54848==    at 0xA61D63: malloc (vg_replace_malloc.c:192)
> ==54848==    by 0x403B24: x264_mb_analyse_load_costs (in
> /usr/local/bin/ffmpeg)
> ==54848==    by 0x3FF87D2D: ???
>
> Regards,
> jh
>
> _______________________________________________
> x264-devel mailing list
> x264-devel <at> videolan.org
> http://mailman.videolan.org/listinfo/x264-devel
>
>

This is intentional.  Because the arrays are static, if libx264 is
called multiple times from the same application, it will not need to
allocate them separately for each process, thus saving memory.

Dark Shikari
_______________________________________________
x264-devel mailing list
x264-devel <at> videolan.org
http://mailman.videolan.org/listinfo/x264-devel

Mojtaba Hosseini | 3 Mar 14:40 2009

Re: Question about changing x.264 bitrate on the fly


> I am looking at creating an application of x.264, but I would like to 
> be able to specify the bit rate of different sections of the video 
> sequence.
>
We've tried the approach you suggest (opening and closing of x264) and 
it works well as long as you apply the memory leak patch. (do a search 
for "memory leaks in encoder.c" in x264 mailing list to get the patch)

mojtaba
_______________________________________________
x264-devel mailing list
x264-devel <at> videolan.org
http://mailman.videolan.org/listinfo/x264-devel

Leon Woestenberg | 3 Mar 16:10 2009
Picon

Re: libx264 memory leak

Hello,

On Tue, Mar 3, 2009 at 11:19 AM, Jason Garrett-Glaser
<darkshikari <at> gmail.com> wrote:
> 2009/3/3 191919 <191919 <at> gmail.com>:
>> I found a memory leak in x264_mb_analyse_load_costs. It keeps some
>> x264_malloc'ed pointer to a global array, but no one remembers to free them
>> in x264_encoder_close.
>>
>
> This is intentional.  Because the arrays are static, if libx264 is
> called multiple times from the same application, it will not need to
> allocate them separately for each process, thus saving memory.
>

Under Unices, using a library constructor/destructor approach is available:

http://www.faqs.org/docs/Linux-HOWTO/Program-Library-HOWTO.html#INIT-AND-CLEANUP

Not sure how Windows handles that though,

Regards,
--

-- 
Leon
_______________________________________________
x264-devel mailing list
x264-devel <at> videolan.org
http://mailman.videolan.org/listinfo/x264-devel

Sean McGovern | 3 Mar 16:24 2009
Picon

Re: libx264 memory leak

Windows DLLs have equivalent functionality within the DllMain entry point.

-----Original Message-----
From: Leon Woestenberg <leon.woestenberg <at> gmail.com>

Date: Tue, 3 Mar 2009 16:10:58 
To: Mailing list for x264 developers<x264-devel <at> videolan.org>
Subject: Re: [x264-devel] libx264 memory leak

Hello,

On Tue, Mar 3, 2009 at 11:19 AM, Jason Garrett-Glaser
<darkshikari <at> gmail.com> wrote:
> 2009/3/3 191919 <191919 <at> gmail.com>:
>> I found a memory leak in x264_mb_analyse_load_costs. It keeps some
>> x264_malloc'ed pointer to a global array, but no one remembers to free them
>> in x264_encoder_close.
>>
>
> This is intentional.  Because the arrays are static, if libx264 is
> called multiple times from the same application, it will not need to
> allocate them separately for each process, thus saving memory.
>

Under Unices, using a library constructor/destructor approach is available:

http://www.faqs.org/docs/Linux-HOWTO/Program-Library-HOWTO.html#INIT-AND-CLEANUP

Not sure how Windows handles that though,

Regards,
--

-- 
Leon
_______________________________________________
x264-devel mailing list
x264-devel <at> videolan.org
http://mailman.videolan.org/listinfo/x264-devel
_______________________________________________
x264-devel mailing list
x264-devel <at> videolan.org
http://mailman.videolan.org/listinfo/x264-devel

git version control | 3 Mar 16:53 2009

commit: Add AltiVec version of hadamard_ac. 2.4x faster than the C version. (Guillaume Poirier )

x264 | branch: master | Guillaume Poirier <gpoirier <at> mplayerhq.hu> | Tue Mar  3 07:44:18 2009 -0800|
[7ddb2c7da0621bb853b6702e6f59619c2d1c6a08] | committer: Guillaume Poirier 

Add AltiVec version of hadamard_ac. 2.4x faster than the C version.
Note this this implementation is pretty naive and should be improved
by implementing what's discussed in this ML thread:
date: Mon, Feb 2, 2009 at 6:58 PM
subject: Re: [x264-devel] [PATCH] AltiVec implementation of hadamard_ac routines

> http://git.videolan.org/gitweb.cgi/x264.git/?a=commit;h=7ddb2c7da0621bb853b6702e6f59619c2d1c6a08
---

 common/ppc/pixel.c |  145 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 145 insertions(+), 0 deletions(-)

diff --git a/common/ppc/pixel.c b/common/ppc/pixel.c
index fc8b417..5120544 100644
--- a/common/ppc/pixel.c
+++ b/common/ppc/pixel.c
 <at>  <at>  -98,6 +98,8  <at>  <at>  PIXEL_SAD_ALTIVEC( pixel_sad_8x8_altivec,   8,  8,  2s, 1 )
 #define VEC_ABS(a)                            \
     a = vec_max( a, vec_sub( zero_s16v, a ) );

+#define VEC_ABSOLUTE(a) (vec_u16_t)vec_max( a, vec_sub( zero_s16v, a ) )
+
 /***********************************************************************
  * VEC_ADD_ABS
  ***********************************************************************
 <at>  <at>  -1850,6 +1852,144  <at>  <at>  static int pixel_sa8d_16x16_altivec( uint8_t *pix1, int i_pix1,
     return i_satd;
 }

+#define HADAMARD4_ALTIVEC(d0,d1,d2,d3,s0,s1,s2,s3) {\
+    vec_s16_t t0 = vec_add(s0, s1);                 \
+    vec_s16_t t1 = vec_sub(s0, s1);                 \
+    vec_s16_t t2 = vec_add(s2, s3);                 \
+    vec_s16_t t3 = vec_sub(s2, s3);                 \
+    d0 = vec_add(t0, t2);                           \
+    d2 = vec_sub(t0, t2);                           \
+    d1 = vec_add(t1, t3);                           \
+    d3 = vec_sub(t1, t3);                           \
+}
+
+#define VEC_LOAD_HIGH( p, num )                                    \
+    vec_u8_t pix8_##num = vec_ld( stride*num, p );                 \
+    vec_s16_t pix16_s##num = vec_perm(pix8_##num, zero_u8v, perm); \
+    vec_s16_t pix16_d##num;
+
+static uint64_t pixel_hadamard_ac_altivec( uint8_t *pix, int stride, const vec_u8_t perm )
+{
+    DECLARE_ALIGNED_16( int32_t sum4_tab[4] );
+    DECLARE_ALIGNED_16( int32_t sum8_tab[4] );
+    LOAD_ZERO;
+
+    VEC_LOAD_HIGH( pix, 0 );
+    VEC_LOAD_HIGH( pix, 1 );
+    VEC_LOAD_HIGH( pix, 2 );
+    VEC_LOAD_HIGH( pix, 3 );
+    HADAMARD4_ALTIVEC(pix16_d0,pix16_d1,pix16_d2,pix16_d3,
+                      pix16_s0,pix16_s1,pix16_s2,pix16_s3);
+
+    VEC_LOAD_HIGH( pix, 4 );
+    VEC_LOAD_HIGH( pix, 5 );
+    VEC_LOAD_HIGH( pix, 6 );
+    VEC_LOAD_HIGH( pix, 7 );
+    HADAMARD4_ALTIVEC(pix16_d4,pix16_d5,pix16_d6,pix16_d7,
+                      pix16_s4,pix16_s5,pix16_s6,pix16_s7);
+
+    VEC_TRANSPOSE_8(pix16_d0, pix16_d1, pix16_d2, pix16_d3,
+                    pix16_d4, pix16_d5, pix16_d6, pix16_d7,
+                    pix16_s0, pix16_s1, pix16_s2, pix16_s3,
+                    pix16_s4, pix16_s5, pix16_s6, pix16_s7);
+
+    HADAMARD4_ALTIVEC(pix16_d0,pix16_d1,pix16_d2,pix16_d3,
+                      pix16_s0,pix16_s1,pix16_s2,pix16_s3);
+
+    HADAMARD4_ALTIVEC(pix16_d4,pix16_d5,pix16_d6,pix16_d7,
+                      pix16_s4,pix16_s5,pix16_s6,pix16_s7);
+
+    vec_u16_t addabs01 = vec_add( VEC_ABSOLUTE(pix16_d0), VEC_ABSOLUTE(pix16_d1) );
+    vec_u16_t addabs23 = vec_add( VEC_ABSOLUTE(pix16_d2), VEC_ABSOLUTE(pix16_d3) );
+    vec_u16_t addabs45 = vec_add( VEC_ABSOLUTE(pix16_d4), VEC_ABSOLUTE(pix16_d5) );
+    vec_u16_t addabs67 = vec_add( VEC_ABSOLUTE(pix16_d6), VEC_ABSOLUTE(pix16_d7) );
+
+    vec_u16_t sum4_v = vec_add(vec_add(addabs01, addabs23), vec_add(addabs45, addabs67));
+    vec_ste(vec_sums(vec_sum4s(sum4_v, zero_s32v), zero_s32v), 12, sum4_tab);
+
+    vec_s16_t tmpi0 = vec_add(pix16_d0, pix16_d4);
+    vec_s16_t tmpi4 = vec_sub(pix16_d0, pix16_d4);
+    vec_s16_t tmpi1 = vec_add(pix16_d1, pix16_d5);
+    vec_s16_t tmpi5 = vec_sub(pix16_d1, pix16_d5);
+    vec_s16_t tmpi2 = vec_add(pix16_d2, pix16_d6);
+    vec_s16_t tmpi6 = vec_sub(pix16_d2, pix16_d6);
+    vec_s16_t tmpi3 = vec_add(pix16_d3, pix16_d7);
+    vec_s16_t tmpi7 = vec_sub(pix16_d3, pix16_d7);
+
+    int sum4 = sum4_tab[3];
+
+    VEC_TRANSPOSE_8(tmpi0, tmpi1, tmpi2, tmpi3,
+                    tmpi4, tmpi5, tmpi6, tmpi7,
+                    pix16_d0, pix16_d1, pix16_d2, pix16_d3,
+                    pix16_d4, pix16_d5, pix16_d6, pix16_d7);
+
+    vec_u16_t addsum04 = vec_add( VEC_ABSOLUTE( vec_add(pix16_d0, pix16_d4) ),
+                                  VEC_ABSOLUTE( vec_sub(pix16_d0, pix16_d4) ) );
+    vec_u16_t addsum15 = vec_add( VEC_ABSOLUTE( vec_add(pix16_d1, pix16_d5) ),
+                                  VEC_ABSOLUTE( vec_sub(pix16_d1, pix16_d5) ) );
+    vec_u16_t addsum26 = vec_add( VEC_ABSOLUTE( vec_add(pix16_d2, pix16_d6) ),
+                                  VEC_ABSOLUTE( vec_sub(pix16_d2, pix16_d6) ) );
+    vec_u16_t addsum37 = vec_add( VEC_ABSOLUTE( vec_add(pix16_d3, pix16_d7) ),
+                                  VEC_ABSOLUTE( vec_sub(pix16_d3, pix16_d7) ) );
+
+    vec_u16_t sum8_v = vec_add( vec_add(addsum04, addsum15), vec_add(addsum26, addsum37) );
+    vec_ste(vec_sums( vec_sum4s(sum8_v, zero_s32v), zero_s32v), 12, sum8_tab);
+
+    int sum8 = sum8_tab[3];
+
+    DECLARE_ALIGNED_16( int16_t tmp0_4_tab[8] );
+    vec_ste(vec_add(pix16_d0, pix16_d4), 0, tmp0_4_tab);
+
+    sum4 -= tmp0_4_tab[0];
+    sum8 -= tmp0_4_tab[0];
+    return ((uint64_t)sum8<<32) + sum4;
+}
+
+
+static const vec_u8_t hadamard_permtab[] = {
+    CV(0x10,0x00,0x11,0x01, 0x12,0x02,0x13,0x03,     /* pix = mod16 */
+       0x14,0x04,0x15,0x05, 0x16,0x06,0x17,0x07 ),
+    CV(0x18,0x08,0x19,0x09, 0x1A,0x0A,0x1B,0x0B,     /* pix = mod8 */
+       0x1C,0x0C,0x1D,0x0D, 0x1E,0x0E,0x1F,0x0F )
+ };
+
+static uint64_t x264_pixel_hadamard_ac_16x16_altivec( uint8_t *pix, int stride )
+{
+    int index =  ((uintptr_t)pix & 8) >> 3;
+    vec_u8_t permh = hadamard_permtab[index];
+    vec_u8_t perml = hadamard_permtab[!index];
+    uint64_t sum = pixel_hadamard_ac_altivec( pix, stride, permh );
+    sum += pixel_hadamard_ac_altivec( pix+8, stride, perml );
+    sum += pixel_hadamard_ac_altivec( pix+8*stride, stride, permh );
+    sum += pixel_hadamard_ac_altivec( pix+8*stride+8, stride, perml );
+    return ((sum>>34)<<32) + ((uint32_t)sum>>1);
+}
+
+static uint64_t x264_pixel_hadamard_ac_16x8_altivec( uint8_t *pix, int stride )
+{
+    int index =  ((uintptr_t)pix & 8) >> 3;
+    vec_u8_t permh = hadamard_permtab[index];
+    vec_u8_t perml = hadamard_permtab[!index];
+    uint64_t sum = pixel_hadamard_ac_altivec( pix, stride, permh );
+    sum += pixel_hadamard_ac_altivec( pix+8, stride, perml );
+    return ((sum>>34)<<32) + ((uint32_t)sum>>1);
+}
+
+static uint64_t x264_pixel_hadamard_ac_8x16_altivec( uint8_t *pix, int stride )
+{
+    vec_u8_t perm = hadamard_permtab[ (((uintptr_t)pix & 8) >> 3) ];
+    uint64_t sum = pixel_hadamard_ac_altivec( pix, stride, perm );
+    sum += pixel_hadamard_ac_altivec( pix+8*stride, stride, perm );
+    return ((sum>>34)<<32) + ((uint32_t)sum>>1);
+}
+
+static uint64_t x264_pixel_hadamard_ac_8x8_altivec( uint8_t *pix, int stride ) {
+    vec_u8_t perm = hadamard_permtab[ (((uintptr_t)pix & 8) >> 3) ];
+    uint64_t sum = pixel_hadamard_ac_altivec( pix, stride, perm );
+    return ((sum>>34)<<32) + ((uint32_t)sum>>1);
+}
+
+
 /****************************************************************************
  * structural similarity metric
  ****************************************************************************/
 <at>  <at>  -1932,5 +2072,10  <at>  <at>  void x264_pixel_altivec_init( x264_pixel_function_t *pixf )
     pixf->var[PIXEL_16x16] = x264_pixel_var_16x16_altivec;
     pixf->var[PIXEL_8x8]   = x264_pixel_var_8x8_altivec;

+    pixf->hadamard_ac[PIXEL_16x16] = x264_pixel_hadamard_ac_16x16_altivec;
+    pixf->hadamard_ac[PIXEL_16x8]  = x264_pixel_hadamard_ac_16x8_altivec;
+    pixf->hadamard_ac[PIXEL_8x16]  = x264_pixel_hadamard_ac_8x16_altivec;
+    pixf->hadamard_ac[PIXEL_8x8]   = x264_pixel_hadamard_ac_8x8_altivec;
+
     pixf->ssim_4x4x2_core = ssim_4x4x2_core_altivec;
 }

_______________________________________________
x264-devel mailing list
x264-devel <at> videolan.org
http://mailman.videolan.org/listinfo/x264-devel

Loren Merritt | 3 Mar 22:28 2009

Re: libx264 memory leak

On Tue, 3 Mar 2009, Leon Woestenberg wrote:
>
> Under Unices, using a library constructor/destructor approach is available:
>
> http://www.faqs.org/docs/Linux-HOWTO/Program-Library-HOWTO.html#INIT-AND-CLEANUP
>
> Not sure how Windows handles that though,

No way am I using a OS-specific feature for this. If shutting up valgrind 
is really important, just put it in a static array.

--Loren Merritt
_______________________________________________
x264-devel mailing list
x264-devel <at> videolan.org
http://mailman.videolan.org/listinfo/x264-devel

Sean McGovern | 3 Mar 22:34 2009
Picon

Re: libx264 memory leak

How...is that OS specific? Are #ifdef's really that hard, Loren?
-----Original Message-----
From: Loren Merritt <lorenm <at> u.washington.edu>

Date: Tue, 3 Mar 2009 21:28:41 
To: Mailing list for x264 developers<x264-devel <at> videolan.org>
Subject: Re: [x264-devel] libx264 memory leak

On Tue, 3 Mar 2009, Leon Woestenberg wrote:
>
> Under Unices, using a library constructor/destructor approach is available:
>
> http://www.faqs.org/docs/Linux-HOWTO/Program-Library-HOWTO.html#INIT-AND-CLEANUP
>
> Not sure how Windows handles that though,

No way am I using a OS-specific feature for this. If shutting up valgrind 
is really important, just put it in a static array.

--Loren Merritt
_______________________________________________
x264-devel mailing list
x264-devel <at> videolan.org
http://mailman.videolan.org/listinfo/x264-devel

_______________________________________________
x264-devel mailing list
x264-devel <at> videolan.org
http://mailman.videolan.org/listinfo/x264-devel


Gmane