protobuf | 25 Nov 19:18 2014

Issue 681 in protobuf: ar

Status: New
Owner: liujisi <at> google.com
Labels: Type-Defect Priority-Medium

New issue 681 by seymour.... <at> gmail.com: ar
https://code.google.com/p/protobuf/issues/detail?id=681

What steps will reproduce the problem?
1.
2.
3.

What is the expected output? What do you see instead?

What version of the product are you using? On what operating system?

Please provide any additional information below.

-- 
You received this message because this project is configured to send all  
issue notifications to this address.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings

--

-- 
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscribe <at> googlegroups.com.
To post to this group, send email to protobuf <at> googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
(Continue reading)

protobuf | 24 Nov 22:10 2014

Issue 680 in protobuf: Python pickling depends on the path used during compilaton

Status: New
Owner: liujisi <at> google.com
Labels: Type-Defect Priority-Medium

New issue 680 by dbertoui... <at> gmail.com: Python pickling depends on the path  
used during compilaton
https://code.google.com/p/protobuf/issues/detail?id=680

What steps will reproduce the problem?
1. Extract the test files
2. Run make in the root directory followed by python test.py. It should  
raise an exception.
3. Run make in the libs directory followed by python test.py. It should  
work fine.
4. Run make in the libs/pblib directory followed by python test.py. It  
should raise an exception.

What is the expected output? What do you see instead?

I would expect the test.py script to run without exception each time.

What version of the product are you using? On what operating system?

* protobuf 2.6.0
* py27-protobuf 2.6.0
* python 2.7.8
* FreeBSD 10.0

Please provide any additional information below.

(Continue reading)

Ganesh Sangle | 22 Nov 01:39 2014
Picon

Writetype mismatch message

I am running into a weird situation where the length of the data passed in the elements leads to a Writetype mismatch message.

Here is snippet for the protobuf:

message QueryResponse
{
    message QueryNode {
        required string node_name = 1;
        required string node_value = 2;
    }
    repeated QueryNode nodes = 3;
}

So this message has repeated elements of two strings.

When I create a response object and set the node_name and node_value such that both the strings are small enough, it works flawlessly.

For example, if the length of the node_name string is 87 and that of node_value is 36, adding 5 elements to the 'node' and sending it over tcp and parsing it over the other side works fine.
However, if the node_name string is 87 chars in length and the node_value is 38 (just two more characters), the same test fails with the following stack trace:

  File "/work/g..y", line 292, in run_query
    query_resp.ParseFromString(query_nodes)
  File "/.../local/protobuf-python/google/protobuf/message.py", line 159, in ParseFromString
    self.MergeFromString(serialized)
  File "/.../local/protobuf-python/google/protobuf/reflection.py", line 1238, in MergeFromString
    bytes_read = _DeserializeOneEntity(message_descriptor, self, decoder)
  File "/.../local/protobuf-python/google/protobuf/reflection.py", line 1082, in _DeserializeOneEntity
    raise RuntimeError('TODO(robinson): Wiretype mismatches not handled.')
RuntimeError: TODO(robinson): Wiretype mismatches not handled.

The overall message length is less that 720bytes, Even if i have one element in the entire message but if the length of one of the string grows by 2, it crashes with this exception.
I could not figure why is this happening. 
Any ideas on how to debug this ? Just changing the lengths causes this to fail.
When i dump the message without parsing on screen, i can see the data correctly - not sure why the parser keeps failing.
Reducing the length of the node_value type by 2 chars makes everything work again...

Any help would be greatly appreciated. 

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscribe <at> googlegroups.com.
To post to this group, send email to protobuf <at> googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
Himanshu Agarwal | 18 Nov 09:50 2014
Picon

"list values" and string values


Hello

I was trying to populate and deserialize following message to string:

message SampleSubMessage{
 repeated
string str = 1;
 repeated fixed32 val
= 2;
}


message
SampleMessage {
 repeated
SampleSubMessage sub = 1;
}


I had already set the SetUseShortRepeatedPrimitives to true before deserializing the message to string.
However in the output string only integer values were printed in "list values" format.

sub {
  str
: ""
  str
: "4"
  val
: [34, 45]
}
sub {
  str
: "2"
  str
: "7"
  val
: [67]
}
sub {
  str
: "3"
}


On further investigation i found that follwing code explicitely excludes strings and message fields for converting to string.

google\protobuf\text_format.cc:1449


void TextFormat::Printer::PrintField(const Message& message,
                                     
const Reflection* reflection,
                                     
const FieldDescriptor* field,
                                     
TextGenerator& generator) const {
 
if (use_short_repeated_primitives_ &&
      field
->is_repeated() &&
      field->cpp_type() != FieldDescriptor::CPPTYPE_STRING &&
      field
->cpp_type() != FieldDescriptor::CPPTYPE_MESSAGE) {
   
PrintShortRepeatedField(message, reflection, field, generator);
   
return;
 
}


Could someone please help me understand the reason for this exclusion.

Thank you

Himanshu Agarwal

.


--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscribe <at> googlegroups.com.
To post to this group, send email to protobuf <at> googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
Himanshu Agarwal | 18 Nov 09:22 2014
Picon

oneof and extended messages


Hello

I was trying to compile following scenerio using protoc:

message SampleMessage {
 repeated fixed32 i
= 1;
 extensions
100 to 199;
}
extend
SampleMessage {
  oneof test_oneof
{
 
string name = 126;
  fixed32 sub_message
= 127;  
 
}
}


But getting following errors.

SampleMessage.proto:6:3: Expected "required", "optional", or "repeated".
SampleMessage.proto:6:20: Missing field number.

Any clue if oneof is supported in extended messages?
Also is there a way to extend oneof field?

Thank you

Himanshu Agarwal

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscribe <at> googlegroups.com.
To post to this group, send email to protobuf <at> googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
protobufel | 18 Nov 01:49 2014
Picon

Announcing ProtoBuf 2.6.1 Java Parser and FileDescriptorProto/FileDescriptor Builder + JSR-341 Expression Language 3.0 with ProtoBuf 2.6.1 + CRUD libraries

Dear All,

I've just officially published a collection of well tested Protocol Buffers' 2.6.1 compliant add-ons/extras for your consumption and productive delight.

I've implemented 100% ProtoBuf 2.6.1 compatible:

  1. pure Java .proto parser and "canonical" FileDescriptorProto/FileDescriptor/FileDescriptorSet(-s) builder (passes ALL ProtoBuf .proto(-s) tests)
  2. enhanced DynamicMessage/Builder combining all relevant features of the ProtoBuf's DynamicMessage.Builder and GeneratedMessage.Builder, including hierarchical sub-builders (passes ALL relevant, - and adapted, - original tests)
  3. Java JSR-341 Expression Language 3.0 with Protocol Buffers 2.6.1, including DynamicMessage/Builder and any GeneratedMessage/Builder
  4. CRUD queries with JSR-341 EL 3.0 and Protocol Buffers 2.6.1

In addition, there is a very efficient file scanning and multi-root resource processing, general use, library, and the Java extended regex implementation.

All artifacts are available on Maven Central; New BSD License (except 2 OpenJDK GPLv2 w/ exceptions licensed dependency for file scanning and extended regex); pure Java 7 (can be easily backported); minimal dependencies!

The project is located at GitHub https://github.com/protobufel/protobuf-el

Your comments and critique are welcome!

Best Regards,

David Tesler

 





    

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscribe <at> googlegroups.com.
To post to this group, send email to protobuf <at> googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
Madwyn Wen | 17 Nov 17:44 2014
Picon

RPC_Implementations: a Protobuf RPC lib and a minimal RPC server implementation

Would you please add my new implementations to the wiki.

libpbrpc: https://github.com/madwyn/libpbrpc

pbrpc: https://github.com/madwyn/pbrpc

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscribe <at> googlegroups.com.
To post to this group, send email to protobuf <at> googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
Alexey Chernushev | 16 Nov 11:48 2014
Picon

Can't deserialize repeated double field

Hi!
I have problem with repeated double field in deserializing process. I want to read 300 messages from protobuf file, but ReadRaw got failed in last message (sometimes 10 before last, it depended on number of encoding type).
I've prepared little example where I am reproducing that error

https://gist.github.com/alexeyche/d6af8a43d346edc12868

Could you please say, am I doing something wrong?

$ protoc -I. --cpp_out=. ./time_series.proto
$ g++ main.cpp time_series.pb.cc -std=c++11 -L/usr/local/lib -lprotobuf -I/usr/local/include
$ ./a.out synthetic_control_TRAIN out.pb

...reading and printing all messages and on 299 message...

Can't do ReadRaw of message size 489
terminate called without an active exception

=== 
synthetic_control_TRAIN file you can get here https://yadi.sk/d/gxZy8JSvcjiVD

If ignore bad status of ReadRaw, then repeated field of last message will be with almost zeros (e.g. 15e-17) in the end. Maybe it's memory leak?


my machine:

$ protoc --version
libprotoc 2.6.1

$ g++ -v
Using built-in specs.
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.8/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 4.8.1-10ubuntu9' --with-bugurl=file:///usr/share/doc/gcc-4.8/README.Bugs --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.8 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.8 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-4.8-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.8.1 (Ubuntu/Linaro 4.8.1-10ubuntu9) 

$ uname -a
Linux alexeyche-Lenovo-G780 3.11.0-12-generic #19-Ubuntu SMP Wed Oct 9 16:20:46 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscribe <at> googlegroups.com.
To post to this group, send email to protobuf <at> googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
Jianshi Huang | 14 Nov 11:07 2014
Picon

How can I deserialize a protobuf message to Map<String, Object>?

Hi,

I have a flat object, and I want it to be like a Map<String, Object>

Can I turn a protobuf object to a Map<String, Object>?

Thanks,
Jianshi

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscribe <at> googlegroups.com.
To post to this group, send email to protobuf <at> googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
Andrew Capodieci | 13 Nov 14:57 2014
Picon

Cross Compiling Protobuf For Raspberry Pi

I'm trying to cross compile google Protocol buffers for the Raspberry Pi, so I can cross compile my own software which links to protobuf.

I've installed protobuf-2.6.0 to the directory ~/ExternalLibs/protobuf-2.6.0, where I perform the following steps:

    cd ~/ExternalLibs/protobuf-2.6.0
   
./configure --disable-shared
    make
    make check

 -- all (5/5) tests pass

    sudo make install
    make distclean

This should set up the initial protoc executable that can be run by my build machine (Lenovo w540 laptop running Ubuntu 12.04) during the cross compiling for my host, the raspberry pi.

I then run:

    ./configure --host=arm-linux-gnueabi CC=/opt/cross/x-tools/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc CXX=/opt/cross/x-tools/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-g++ --with-protoc=/usr/bin/protoc --prefix=/home/andrewc1204/raspberrypi_crosscompile/protobuf_xtools/

My cross compiler was built following this tutorial: http://www.bootc.net/archives/2012/05/26/how-to-build-a-cross-compiler-for-your-raspberry-pi/. The only extra thing I did was enable an additional c++ compiler on top of building the c compiler in the ct-ng menus, which is why I'm able to use the g++ compiler on top of the gcc compiler.

This ./configure seems to run, and then I call
   
    make

The output is as follows:

   
make  all-recursive
    make
[1]: Entering directory `/home/andrewc1204/ExternalLibs/protobuf-2.6.0'
    Making all in .
    make[2]: Entering directory `
/home/andrewc1204/ExternalLibs/protobuf-2.6.0'
    make[2]: Leaving directory `/home/andrewc1204/ExternalLibs/protobuf-2.6.0'

   
Making all in src
    make
[2]: Entering directory `/home/andrewc1204/ExternalLibs/protobuf-2.6.0/src'
    /usr/bin/protoc -I. --cpp_out=. google/protobuf/unittest.proto google/protobuf/unittest_empty.proto google/protobuf/unittest_import.proto google/protobuf/unittest_import_public.proto google/protobuf/unittest_mset.proto google/protobuf/unittest_optimize_for.proto google/protobuf/unittest_embed_optimize_for.proto google/protobuf/unittest_custom_options.proto google/protobuf/unittest_lite.proto google/protobuf/unittest_import_lite.proto google/protobuf/unittest_import_public_lite.proto google/protobuf/unittest_lite_imports_nonlite.proto google/protobuf/unittest_no_generic_services.proto google/protobuf/compiler/cpp/cpp_test_bad_identifiers.proto
    google/protobuf/unittest.proto:173:3: Expected "required", "optional", or "repeated".
    google/protobuf/unittest.proto:173:21: Missing field number.
    google/protobuf/unittest.proto:610:3: Expected "required", "optional", or "repeated".
    google/protobuf/unittest.proto:610:13: Missing field number.
    google/protobuf/unittest.proto:632:3: Expected "required", "optional", or "repeated".
    google/protobuf/unittest.proto:632:13: Missing field number.
    google/protobuf/unittest.proto:647:3: Expected "required", "optional", or "repeated".
    google/protobuf/unittest.proto:647:13: Missing field number.
    google/protobuf/unittest.proto:672:3: Expected "required", "optional", or "repeated".
    google/protobuf/unittest.proto:672:13: Missing field number.
    google/protobuf/unittest.proto:853:1: Expected "required", "optional", or "repeated".
    google/protobuf/unittest.proto:853:21: Missing field number.
    google/protobuf/unittest.proto:862:1: Reached end of input in message definition (missing '}').
    make[2]: *** [unittest_proto_middleman] Error 1
    make[2]: Leaving directory `
/home/andrewc1204/ExternalLibs/protobuf-2.6.0/src'
    make[1]: *** [all-recursive] Error 1
    make[1]: Leaving directory `/home/andrewc1204/ExternalLibs/protobuf-2.6.0'

    make
: *** [all] Error 2

I'm stuck at this point, and don't know what else to check to try and debug this problem. Any help is greatly appreciated! Please let me know if any additional information is helpful and I will gladly supply it.

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscribe <at> googlegroups.com.
To post to this group, send email to protobuf <at> googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
protobuf | 12 Nov 11:48 2014

Issue 679 in protobuf: protobuf causes crash beacuse of mutex lock

Status: New
Owner: liujisi <at> google.com
Labels: Type-Defect Priority-Medium

New issue 679 by islm.ya... <at> gmail.com: protobuf causes crash beacuse of  
mutex lock
https://code.google.com/p/protobuf/issues/detail?id=679

What steps will reproduce the problem?
FreeBSD 10, it sometimes crashes on SerializeToOstream

     ostringstream os(ios::out | ios::binary);
     email.SerializeToOstream(&os);

and creates a core of binary here is gdb output:

#0  0x00000000004aa1a1 in google::protobuf::internal::Mutex::Lock ()
#1  0x00000000004ab767 in google::protobuf::internal::MutexLock::MutexLock  
()
#2  0x00000000004aa011 in google::protobuf::internal::LogMessage::Finish ()
#3  0x00000000004aa0e9 in  
google::protobuf::internal::LogFinisher::operator= ()
#4  0x000000000054e623 in  
google::protobuf::internal::WireFormat::VerifyUTF8StringFallback ()
#5  0x000000000044b7a2 in  
google::protobuf::internal::WireFormat::VerifyUTF8String (
     data=0x8071b5798 "testtir asdas asda sda sdasdasd dadsad n  
asdasd\nBilgilerinize.\nSayg�lar�mla,\nSibel Kocaku� �ENG�N\nKalite  
ve �dari ��ler\n<mailto:isim.soyisim <at> vizyonarge.com> e:   
sibel.kocakus <at> vizyonarge.com  |\n<http:/"..., size=836,  
op=google::protobuf::internal::WireFormat::SERIALIZE) at gthr-default.h:199
#6  0x0000000000449007 in emails::Email::SerializeWithCachedSizesToArray  
(this=0x7ffffffed170, target=0x80702b0a2 "word\"  
xmlns:x=\"urn:schemas-mic �$\a\b") at emailInput.pb.cc:43
#7  0x00000000004ac1e7 in  
google::protobuf::MessageLite::SerializePartialToCodedStream ()
#8  0x00000000004ac35d in  
google::protobuf::MessageLite::SerializeToCodedStream ()
#9  0x00000000004ac39b in  
google::protobuf::MessageLite::SerializeToZeroCopyStream ()
#10 0x000000000053b2b0 in google::protobuf::Message::SerializeToOstream ()

What version of the product are you using? On what operating system?
protobuf 2.5.0

Please provide any additional information below.

-- 
You received this message because this project is configured to send all  
issue notifications to this address.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings

--

-- 
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscribe <at> googlegroups.com.
To post to this group, send email to protobuf <at> googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.


Gmane