Protocol Buffers how to use it

As an alternative to XML and databases, the Protocol Buffers format was created by Google to back up structured data in a double form, easy to read and binary, compact and easy to process for programs.
The format is expanding.

Why Protocol Buffers?

This format has been identified by Google as an alternative to XML for faster file processing, especially for its own needs, as millions of servers use data volumes.
The easiest way to process data is that it is in the form of an object in memory and remains constant in the file, so there is a serialization principle, which is to store the entire object in the file.
This can be done in XML, but just as JSON provides a more direct way to do it in JavaScript, PB provides a direct way to do it for other programming languages such as C++ and Java. JSON is based on JavaScript dynamic variables, while PB defines the data type.
JSON is better than XML for JavaScript, PB is better than XML for other languages, because it does not have the disadvantages of XML, large space consumption, and heavy enough processing to convert data. However, it retains its advantages - flexibility and easy reading.

Although PB currently only works with C++ and Java, the authors are working on porting it to other languages, in particular PHP and Silverlight, so it should belong to the web landscape soon.

How PB works

?

The compiler provided by Google allows you to generate a class for the software from the description of the proto-document. This class contains methods that automatically encode data in a binary file or load a binary file and decrypt it.
In the future, you can expand the definition and continue reading already created files.

Thus, we take the following steps:
- You define the structure of the protocol.
- The compiler creates a class that is a subclass of the Message class.
- You write a program using this class.
- You compile it by including the created class and the protobuf bookstore.
- Data can be saved to a proto file using a serialization function inherited from the Message class.
- You can also create a web service and set up a communication channel with the client.
- Your program data is exchanged between the client and the server via your protocol.

How to use Protocol Buffers

Using proto is very easy. Here are the steps to create the first program using Protocol Buffers.

1) Download archive

You will need two archives for the compiler and libraries to include.
On the Google Protocol buffer website, select the archive that corresponds to your system.

Extract the content, resulting in a Protobuf directory and a protoc compiler.

2) Compilation of the protobuf bookstore

The procedure depends on the language, compiler and system, and this is explained in the README.txt file in the root of the Protobuf directory, extracted from the archive.
To illustrate our example, we will use Visual Studio Express under Windows.

Start VSE and load the project file located in the vsprojects directory.
Starting the compilation will probably precede the conversion because the project file must be compatible with older versions of VSE.
MacFile will recreate the number of files, but all we are interested in is:

protobuf.lib
protobuf.dll

Like the files accompanying protobuf.dll with the added extension.

3) Creating a development environment

For practical reasons, it is worth creating a new repertoire for our project. It can be called "proto," for example .
This directory contains:

protoc.exe
protobuf.lib 
protobuf.dll et les fichiers d'accompagnement.
google --- le sous-répertoire de Protobuf

This is a proto compiler (protoco.exe under Windows), libraries and a directory of header files to include.

4) Define proto file

In this minimalist file, we add one initialized data field with the text "Hello World!."

We define a text field called "data":

message hello
{
  required string data;
}

Add the sequence number:

message hello
{
  required string data = 1;
}

Add a default value that initializes the field and is enclosed in brackets:

[default="Hello World!"]

This gives the following complete definition:

message hello
{
  required string data = 1 [default="Hello World!"];
}

The file is saved as hello.proto in our working directory.

5) Compile proto file

A batch file named makehello.bat was created containing the following command:

protoc --cpp_out=. hello.proto

parameter --cpp_out serves the destination directory, the dot indicates the current directory.
hello.proto is a previously created definition file.

The compilation will create the following files:

hello.pb.cc
hello.pb.h

The protoc compiler generated a C++ class matching our definition and a corresponding header file.
We just need to create a C++ program to use this class.

6) Write a hello program

An example of using a class would be simply displaying the message contained in the file. Then we will change the content of the field, assigning it another line.

int main()
{
  hello demo;                 // instance "demo" of the "hello" class defined in proto
  
  const std::string& test = demo.data();    // data is the field where the text was stored
  cout << test << "\n";
  
  demo.set_data("How do you do?");    // changing the data
  cout << demo.data() << "\n";
  return 0;
}

The demo.data () parameter allows you to access the contents of the data field of our proto definition. This interface is generated by the protoc compiler.
You also create a demo.set_data () statement to modify the contents of the field.

The full source code is archived.

7) Compile hello program

The batch file will be appended with two lines. The first compiles the file created by the channel, which produces hello.pb.obj, the second compiles our own script:

protoc --cpp_out=. hello.proto

cl hello.pb.cc /EHsc -I. -c

cl hello.cpp  /EHsc -I. hello.pb.obj libprotobuf.lib

The second compilation command included the hello.pb.obj file and the protobuf.lib runtime interface (which allows access to protobuf.dll content).

Run makedemo.bat, the hello.exe program is created. It should show:

Hello World!
How do you do?

8) Class data backup

Using methods inherited from the Message superclass, content modified in our class can be persistently saved and found when the proto file is further used or by the same or another program.

To do this, you can use the C++ fstream function in combination with the inherited SerializeToOstream method.

Define a new proto-simplification file:

message hello
{
  required int32 data = 1 [default=10];
}

The body of the function then becomes:

fstream input("hellowrite.proto", ios::in | ios::binary);

cout << demo.data() << "\n";
  
demo.set_data(demo.data() + 10);     // adding a value
  
fstream output("hellowrite.proto", ios::out | ios::trunc | ios::binary);
demo.SerializeToOstream(&output);

A chaque fois que nous lançons le programme, le nombre affiché est augmenté de 10 ce qui nous prouve que le contenu de data est bien sauvegardé sur disque. En fait un nouveau fichier proto binaire est créé avec la nouvelle valeur stockée.

Le code source complet est dans l'archive.

9) Utilisation sur un serveur

Le compilateur crée aussi des interfaces entre le serveur et les clients, pour cela il faut définir un service:

service hello 
{    
    rpc data(HelloRequest) returns(HelloResponse);  
}

Le processus de la création d'un service est décrit dans l'API Protocol Buffers. 

data_Stub(RpcChannel* channel)

L'instruction crée un accès au channel auquel on peut envoyer un message.
Ceci est détaillé dans le tutoriel de Google.

Aller plus loin

Ceci n'est qu'une introduction à l'utilisation du langage Protocol Buffer. Pour une explication détaillée du format, allez sur le site de Google:

Consulter également le fichier d'en-tête généré par protoc, il fournit la liste des méthodes disponibles.

Les exemples de ce tutoriel sont réduits au minimum utile à la compréhension, ils ne sont pas utilisables tels quels en production. Dans vos applications de nombreux contrôles devront être ajoutés.

Download les examples