Take the google's tutorial for examples:
message Person {
required string name = 1;
required int32 id = 2;
optional string email = 3;
enum PhoneType {
MOBILE = 0;
HOME = 1;
WORK = 2;
}
message PhoneNumber {
required string number = 1;
optional PhoneType type = 2 [default = HOME];
}
repeated PhoneNumber phone = 4;
}
message AddressBook {
repeated Person person = 1;
}
It uses an AddressBook wrapper message as storing multiple Person messages. If I serialize or deserialize it I use methods like AddressBook.mergeFrom(FileInputStream)
addressBook.build.().writeTo()
but it will obviously overflow if I have 20 million Person records. And also google says if I want to store multiple messages in one file without using the wrapper message I need to record each message's length, which is not possible for string types.
Is there a good way to serialize a large amount of messages in one file? And I am using Java by the way.
I haven't tried this, but I'd expect it to work:
Writing:
CodedOutputStream output = CodedOutputStream.newInstance(...);
while (...) {
Person person = ...;
output.writeMessageNoTag(person);
}
Reading:
CodedInputStream input = CodedInputStream.newInstance(...);
while (!input.isAtEnd()) {
Person.Builder builder = Person.newBuilder();
input.readMessage(builder, null); // Or specify extension registry
Person person = builder.build();
// Use person
}
See more on this question at Stackoverflow