headers
Sergei Golubchik | 2 Jun 2009 09:49
Picon
Favicon

Re: About definition of I_S tables

Hi, scut_tang!

First, don't cross-post in two lists, please.
It would be sufficient to send this mail to internals <at>  only, no need to
send it also to soc <at> .

On Jun 02, scut_tang wrote:
> Hi, Sergei.
> I found table definition of P_S storage engine is:
> There is a directory named performance_schema in var directory, which
> has lots of .frm files and each stands for one table of P_S.
> After perform command "mysql_install_db", performance_schema directory
> is created.
> When user issue a query, like "SELECT * from
> performance_schema.PROCESSLIST", rnd_init generates table_processlist
> object and rnd_next gets data. What I want to point out here is
> rnd_next uses TABLE *table. The member table of handler contains lots
> of definitions of opened table. The MySQL server must get all P_S
> tables definitions from .frm files in performance_schema directory.
> How can P_S do that? How .frm files can be pre-created? I want I_S to
> implement like this. I think it works well. It just has some .frm
> files.

No, I'd rather have I_S tables completely virtual, like they are now.
Not "information_schema" subdirectory, no frm files.

P_S has numerous problems because of physical frm files. There are quite
a few bugs there, for example related to users doing CREATE/ALTER/DROP
on P_S tables.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Stefan Hinz | 2 Jun 2009 18:20
Picon

MySQL University session on June 4: Boosting Performance With MySQL 5.1 Partitioning

Boosting Performance With MySQL 5.1 Partitioning
http://forge.mysql.com/wiki/Boosting_Performance_With_MySQL_5.1_Partitioning

This Thursday (June 4th, 14:00 UTC), Giuseppe Maxia will give a MySQL
University session on Boosting Performance With MySQL 5.1 Partitioning.
Giuseppe is leading the Community team at MySQL and has done various
MySQL University sessions before.

For MySQL University sessions, point your browser to this page:

http://webmeeting.dimdim.com/portal/JoinForm.action?confKey=mysqluniversity

You need a browser with a working Flash plugin. You may register for a
Dimdim account, but you don't have to. (Dimdim is the conferencing
system we're using for MySQL University sessions. It provides integrated
voice streaming, chat, whiteboard, session recording, and more.)

MySQL University is a free educational online program for
engineers/developers. MySQL University sessions are open to anyone, not
just Sun employees. Sessions are recorded (slides and audio), so if you
can't attend the live session you can look at the recording anytime
after the session.

Here's the schedule for the upcoming weeks:

# June 4, 2009: Boosting Performance With MySQL 5.1 Partitioning
(Giuseppe Maxia)
# June 11, 2009: To be scheduled (original session canceled)
# June 18, 2009: No session scheduled
# June 15, 2009: MySQL code contributions (Lenz Grimmer)
(Continue reading)

Permalink | Reply | 0 comments |
headers
Sergei Golubchik | 3 Jun 2009 12:25
Picon
Favicon

Re: About definition of I_S tables

Hi, scut_tang!

On Jun 03, scut_tang wrote:
> Have a good vacation, Sergei.

Thanks.

> >P_S has numerous problems because of physical frm files. There are
> >quite a few bugs there, for example related to users doing
> >CREATE/ALTER/DROP on P_S tables.
> >
> >frm tables have to be pre-created in mysql_install_db (that's the
> >script that creates all system tables, like mysql.user, and so on).
> >
> >This design doesn't really work for plugins. Any plugin may install
> >I_S table any time, so we cannot pre-create them all. And a user can
> >unload a plugin any time - the table have to dissapear automatically.
> >And a user can load a new version of the plugin with a different
> >structure of the same I_S table - the table must be automatically
> >created in the new format. Supporting all this with physical frm
> >files would be really painful.
> >
> >So, I'd rather have I_S tables completely virtual, like they are now.
> >No physical frm files.
> >
> I got what you said. I have a new idea.
> The traditional implementation, which has .frm file, is:
> dispatch_command -> mysql_parse -> mysql_execute_command ->
> execute_sqlcom_select -> open_and_lock_tables ->
> open_and_lock_tables_derived -> open_tables ->open_table ->
(Continue reading)

Permalink | Reply | 0 comments |
headers
Cindy Hao | 3 Jun 2009 20:54
Favicon

bulk insert values

I am implementing start_bulk_insert interface. The input parameter is
ha_rows only. Where Can I get the value list for all columns? Thanks,

Cindy

--

-- 
MySQL Internals Mailing List
For list archives: http://lists.mysql.com/internals
To unsubscribe:    http://lists.mysql.com/internals?unsub=gcdmd-internals <at> m.gmane.org
Permalink | Reply | 1 comment |
headers
Sergei Golubchik | 3 Jun 2009 22:13
Picon
Favicon

Re: bulk insert values

Hi, Cindy!

On Jun 03, Cindy Hao wrote:
> I am implementing start_bulk_insert interface. The input parameter is
> ha_rows only. Where Can I get the value list for all columns? Thanks,

You can not.
start_bulk_insert() is a hint that MySQL is going to insert ha_rows rows
(or 0 if unknown). Rows will be inserted later using write_row() as
usual.

Regards / Mit vielen Grüßen,
Sergei

-- 
   __  ___     ___ ____  __
  /  |/  /_ __/ __/ __ \/ /   Sergei Golubchik <serg <at> sun.com>
 / /|_/ / // /\ \/ /_/ / /__  Principal Software Engineer/Server Architect
/_/  /_/\_, /___/\___\_\___/  Sun Microsystems GmbH, HRB München 161028
       <___/                  Sonnenallee 1, 85551 Kirchheim-Heimstetten
Geschäftsführer: Thomas Schroeder, Wolfgang Engels, Wolf Frenkel
Vorsitzender des Aufsichtsrates: Martin Häring

--

-- 
MySQL Internals Mailing List
For list archives: http://lists.mysql.com/internals
To unsubscribe:    http://lists.mysql.com/internals?unsub=gcdmd-internals <at> m.gmane.org
Permalink | Reply | 0 comments |
headers
Daniel Nichter | 4 Jun 2009 01:08
Favicon

Unknown data between TCP and MySQL headers

Hello,

I am hacking on the MySQL protocol using hex dumps from tcpdump like the 
following:

    0x0000:  4508 004e 266f 4000 3e06 650b 0a36 d4ab
    0x0010:  0a37 c80f c1ff 0cea 3a17 3786 3927 f142
    0x0020:  8018 ffff d5a3 0000 0101 080a 22f5 83d3
    0x0030:  40e0 fd21 1300 0000 0000 000f 0000 0003
    0x0040:  7368 6f77 2064 6174 6162 6173 6573 188d
    0x0050:  06f4

I have a good grasp of the MySQL protocol but this particular packet confuses 
me.  The TCP header says that itself is 8 32-bit words long, therefore the 
MySQL header should begin with '1300 0000 000...'.  Yet the MySQL header does 
not seem to actually begin until '0f 0000 0003...' which is a "show databases" 
query.  What, then, is this extra data '1300 0000 0000 00'?

I thought maybe it is a compressed packet, but the CLIENT_COMPRESS flag is off 
in the client's handshake packet and the doc says that a compression header 
only adds an extra 3 bytes.

Thank you in advance for your help,

Daniel

--

-- 
MySQL Internals Mailing List
For list archives: http://lists.mysql.com/internals
To unsubscribe:    http://lists.mysql.com/internals?unsub=gcdmd-internals <at> m.gmane.org
(Continue reading)

Permalink | Reply | 2 comments |
headers
Slava Akhmechet | 4 Jun 2009 04:17
Picon
Gravatar

Profiling a storage engine

Hi,

We're at a point where we need to understand performance bottlenecks
of our storage engine and we're not sure how to proceed. What is the
best way to do profiling? We are running benchmarks that compare our
performance, cpu utilization, and IO throughput to MyISAM. We're also
using procsystime to give us a picture of how much time we spend on
various system calls. However, we're not sure how to best make sense
of the numbers. We'd like to run gprof, but we're not sure how to
compile MySQL to support it, and whether it will give realistic
numbers.

We'd appreciate if anyone could share their experience on how to
effectively profile a storage engine.

Regards,
- Slava

--

-- 
MySQL Internals Mailing List
For list archives: http://lists.mysql.com/internals
To unsubscribe:    http://lists.mysql.com/internals?unsub=gcdmd-internals <at> m.gmane.org
Permalink | Reply | 9 comments |
headers
MARK CALLAGHAN | 4 Jun 2009 05:01
Picon

Re: Profiling a storage engine

On Wed, Jun 3, 2009 at 7:17 PM, Slava Akhmechet <coffeemug <at> gmail.com> wrote:
> Hi,
>
> We're at a point where we need to understand performance bottlenecks
> of our storage engine and we're not sure how to proceed. What is the
> best way to do profiling? We are running benchmarks that compare our
> performance, cpu utilization, and IO throughput to MyISAM. We're also
> using procsystime to give us a picture of how much time we spend on
> various system calls. However, we're not sure how to best make sense
> of the numbers. We'd like to run gprof, but we're not sure how to
> compile MySQL to support it, and whether it will give realistic
> numbers.
>
> We'd appreciate if anyone could share their experience on how to
> effectively profile a storage engine.

I have never had any luck using gprof on multi-threaded servers. If
you use Linux, then oProfile works great for flat profiles and Google
perftools works great for hierarchical profiles. But to use perftools,
you need to make some changes to the server.

This blog post references code that has the changes. The key change is
to call a perftools function when a thread is created --
http://mysqlha.blogspot.com/2009/03/more-on-using-google-perftools-with.html

-- 
Mark Callaghan
mdcallag <at> gmail.com

--

--
(Continue reading)

Permalink | Reply | 5 comments |
headers
Sergei Golubchik | 4 Jun 2009 16:13
Picon
Favicon

Re: About definition of I_S tables

Hi, scut_tang!

On Jun 04, scut_tang wrote:
> >Yes, but I think it would be easier to go just a bit deeper. That is to
> >keep open_table() and open_table_from_share() as is, and implement the
> >function to create a TABLE_SHARE without an frm file. When TABLE_SHARE
> >exists, open_table_from_share() will work normally, no need to modify
> >it.
> >And the simplest way to create a custom TABLE_SHARE, I suppose, is a
> >table discovery - ha_create_table_from_engine() -> ha_discover().
> 
> I browse those function source code today. They drive me crazy.
> What do "Try to discover table from engine", "table exist in handler" and
> "table in engine" mean?  Could you give some example?
> I know MySQL server will reach ha_create_table_from_engine(), when a
> query as "SELECT * FROM Whatever.none". Whatever.none is not exist.
> The function will return a result means table did not exist. In what
> situation, it will return table created ok?
>  
> Could you say something more about ha_discover() in detail?

Sure. "Table discovery" is a mechanism for a server to, well, discover
tables that already exist in the engine.

Originally it was developed for NDB. Indeed, in the cluster environment
there could be many MySQL daemons connected to one cluster. And a table
created from one of these daemons should automatically be visible in all
others. This is implemented via "table discovery" - when a server
doesn't find an frm file for a table it asks the engine "does this table
exists in the engine?", and the engine can answer "yes, here's an frm
(Continue reading)

Permalink | Reply | 0 comments |
headers
Michael Widenius | 4 Jun 2009 21:53
Favicon

re: Unknown data between TCP and MySQL headers


Hi!

>>>>> "Daniel" == Daniel Nichter <daniel <at> percona.com> writes:

Daniel> Hello,
Daniel> I am hacking on the MySQL protocol using hex dumps from tcpdump like the 
Daniel> following:

Daniel>     0x0000:  4508 004e 266f 4000 3e06 650b 0a36 d4ab
Daniel>     0x0010:  0a37 c80f c1ff 0cea 3a17 3786 3927 f142
Daniel>     0x0020:  8018 ffff d5a3 0000 0101 080a 22f5 83d3
Daniel>     0x0030:  40e0 fd21 1300 0000 0000 000f 0000 0003
Daniel>     0x0040:  7368 6f77 2064 6174 6162 6173 6573 188d
Daniel>     0x0050:  06f4

Daniel> I have a good grasp of the MySQL protocol but this particular packet confuses 
Daniel> me.  The TCP header says that itself is 8 32-bit words long, therefore the 
Daniel> MySQL header should begin with '1300 0000 000...'.  Yet the MySQL header does 
Daniel> not seem to actually begin until '0f 0000 0003...' which is a "show databases" 
Daniel> query.  What, then, is this extra data '1300 0000 0000 00'?

Daniel> I thought maybe it is a compressed packet, but the CLIENT_COMPRESS flag is off 
Daniel> in the client's handshake packet and the doc says that a compression header 
Daniel> only adds an extra 3 bytes.

It's probable a compressed packet after all. Please verify your compress flag!

Here is a breakdown of the packet:
(Continue reading)

Permalink | Reply | 1 comment |

Gmane