Building API with gRPC using protobuf – Part 2

Table of contents

Reading Time: 3 minutes

In the previous blog, we discussed gRPC. Why is gRPC better/worse than REST? Advantages of gRPC and why we have to use gRPC. I also covered the workflow of gRPC. And last not least what are the different use cases of gRPC. In order to continue our series, we will discuss today the protobuf. How can we generate the Java and Scala code corresponding to our defined protobuf? In the previous blog discuss we discuss what are protocol buffers…

A bit of history

Protocol buffers were initially developed at Google to deal with an index server request/response protocol. Prior to protocol buffers, there was a format for requests and responses that used hand marshalling/un-marshalling of requests and responses, and that supported a number of versions of the protocol. This resulted in some very ugly code, like:

 if (version == 3) {
   ...
 } else if (version > 4) {
   if (version == 5) {
     ...
   }
   ...
 }

Explicitly formatted protocols also complicated the rollout of new protocol versions, because developers had to make sure that all servers between the originator of the request and the actual server handling the request understood the new protocol before they could flip a switch to start using the new protocol.

How do they work?

You specify how you want the information you’re serializing to be structured by defining protocol buffer message types in .proto files. Each protocol buffer message is a small logical record of information, containing a series of name-value pairs. Here’s a very basic example of a .proto file that defines a message containing information about a person:

Lets take an example of protocol buffers

message Person {
  string name = 1;
  int32 id = 2;
  string email = 3;
  enum PhoneType {
    MOBILE = 0;
    HOME = 1;
    WORK = 2;
  }
  message PhoneNumber {
    string number = 1;
    PhoneType type = 2 [default = HOME];
  }
  repeated PhoneNumber phone = 4;
}

The message has one or more uniquely numbered fields also each field has a name and a value type, where value types can be numbers (integer or floating-point), booleans, strings, raw bytes, or even other protocol buffer message types as in the example above. Apart from that allowing you to structure your data hierarchically. You can specify optional fields, required fields, and repeated fields but it is deprecated in the latest protobuf version 3. You can find more information about writing .proto files in the Protocol Buffer Language Guide.

Whats happens when we compile our protobuf file?

Once you’ve defined your messages, you run the protocol buffer compiler for your application’s language on your .proto file to generate data access classes. These provide simple accessors for each field like name()and set_name() as well as methods to serialize/parse the whole structure to/from raw bytes – so, for instance, if your chosen language is C++, running the compiler on the above example will generate a class called Person. You can then use this class in your application to populate, serialize, and retrieve Person protocol buffer messages.

We define the protobuf and compile it. What now?

You might then write some code like this in Java:

Person person;
person.set_name("John Doe");
person.set_id(1234);
person.set_email("jdoe@example.com");
fstream output("myfile", ios::out | ios::binary);
person.SerializeToOstream(&output);

Then, later on, you could read your message back in:

fstream input("myfile", ios::in | ios::binary);
Person person;
person.ParseFromIstream(&input);
cout << "Name: " << person.name() << endl;
cout << "E-mail: " << person.email() << endl;

Important note: You can add new fields to your message formats without breaking backward-compatibility; old binaries simply ignore the new field when parsing. So if you have a communications protocol that uses protocol buffers as its data format, you can extend your protocol without having to worry about breaking existing code.

Protocol buffers were designed to solve many of these problems:

New fields could be easily introduced, and intermediate servers that didn’t need to inspect the data could simply parse it and pass through the data without needing to know about all the fields.
Formats were more self-describing and could be dealt with from a variety of languages (C++, Java, etc.)

How to generate Java or Scala code?

Add the corresponding plugins in your project

For Scala
addSbtPlugin("com.thesamet" % "sbt-protoc" % "0.99.28")

For java
/**
  * Grpc related dependencies
  */
val grpcVersion = "1.20.0"
val grpcNetty = "io.grpc" % "grpc-netty" % grpcVersion

val grpcCore = "io.grpc" % "grpc-core" % grpcVersion

val grpcStub = "io.grpc" % "grpc-stub" % grpcVersion

val grpcAuth = "io.grpc" % "grpc-auth" % grpcVersion

val grpcProtobuf = "io.grpc" % "grpc-protobuf" % grpcVersion

val nettyTcnative = "io.netty" % "netty-tcnative-boringssl-static" % "2.0.25.Final" classifier "linux-x86_64"

Create a directory name proto under the main directory
Put your .proto file in the proto directory.
In SBT, go to your project directory for example project your_directoryName and compile it.
It will generate all the code related to gRPC client and server.

Summary

In this blog, we discussed protobuf. What happens we compile them. How to generate the Scala or Java code. In the next blog, we will create a small application using gRPC and Scala.

References:
* gRPC documentation.