In this blog, we will implement an application using Connect Lib and gRPC. First, we will understand what Connect Library is and its basic example followed by the types of protocols in connect Scala. Finally, we will see the steps needed to implement the application using Connect in Scala and in that section we will see what is gRPC, what is gRPC gateway, and how to implement gPRC using scala. Also, we will see some codes written in Scala that compile the proto file and generate the Scala classes. So, let’s get started:
What is Connect?
Connect is a family of libraries for building browser and gRPC-compatible HTTP APIs. Connect automatically generates code which is designed manually just like object classes, handlers, and request-response classes. We need to define the schema in a .proto file and write the application logic like Server, Client, and Implementation class. Connect will automatically generate the required files for you. Connect is fully compatible with gRPC clients and servers.
What is Connect Scala?
In this section, we will write the service in a .proto file and compile it using the “sbt compile” command. There are multiple ways of implementing gRPC in Scala. We are using Akka-gRPC to compile the proto file which will generate the Handlers along with Server Client classes. Then, we will have to write simple implementations and Client & Server classes for our Application using Scala.
In the above architecture diagram, we can see that there are two different ways of building web applications’ backend using gRPC, on the left side you will see the REST ( gRPC gateway ) based way and on the right-hand side is the gRPC web way of building the backend using gRPC. We will use the REST ( gRPC gateway ) way using the gPRC gateway and in the next blog, we will see the use of the gRPC web client. Now Let’s understand the types of Protocols in Scala Connect.
Protocols in Connect Scala
There are two protocols in gRPC Scala that Connect already support with the Go language. They are:
- Scala gRPC web
- Scala gRPC gateway
In this blog, we will go through the gRPC gateway part. For that, let’s understand what gRPC is, what is gRPC gateway and finally come up with its implementation.
What is gRPC?
gRPC is a modern open-source high-performance Remote Procedure Call (RPC) framework that can run in any environment. It can efficiently connect services in and across data centres with pluggable support for load balancing, tracing, health checking, and authentication. It is also applicable in the last mile of distributed computing to connect devices, mobile applications, and browsers to backend services.
The main usage scenarios include:
- Efficiently multi-language services in microservices style architecture.
- Connecting mobile devices, and browser clients to backend services.
- Generating efficient client libraries.
Core features that make it awesome are:
- Supports many languages (11 including scala, java, python etc).
- Highly efficient on wire and with a simple service definition framework.
- Bi-directional streaming with http/2-based transport.
- Pluggable auth, tracing, load balancing, and health checking.
What is a gRPC gateway?
gRPC-Gateway is a plugin that generates a reverse proxy server for gRPC services which converts gRPC into Restful and vice versa. It just reads the proto file and generates a reverse proxy server that translates a RESTFul HTTP API into gRPC. The gRPC gateway is already implemented in the GO language. Let’s implement this using Scala.
Some of the major use cases of the gRPC gateway are:
- Some older clients may not support gRPC and require a Restful/JSON interface.
- Your browser may not support gRPC, making the gRPC-Gateway the only option for the web client wanting to interact with gRPC services.
How to build a Ticket Booking System using Connect Scala?
The overall implementation of the ticket booking application is divided into 4 parts. These are:
- Ngnix Filters (Handle http1.1 request and create a response)
- gRPC web
- gRPC gateway
- gRPC backend
In this blog, we’ll use gRPC Gateway to build a simple TicketBookingSystem, which will allow clients to book tickets and view their booking history. We’ll start by creating a gRPC service definition to define the methods that our service will provide. We’ll then use the protoc compiler, Akka-gRPC here, to generate the necessary code for our gRPC server and client. Once we have our gRPC server up and running, we’ll use gRPC-Gateway to generate a reverse proxy and HTTP/1.1 JSON API for our gRPC service. This will allow us to use familiar RESTful paradigms when designing our API, and enable clients that don’t support gRPC to interact with our service.
The second part which is gRPC web will be covered in the next part of the blog. For now, let’s start with Ngnix Filters first:
Ngnix Filters (Handle http1.1 request and create a response)
Here in this part of the implementation of the Ticket Booking Application, we will configure Ngnix proxy requests to the gRPC-Gateway. Given below is the Ngnix Configuration file:
http {
upstream grpc_backend {
server localhost:9000;
}
server {
listen 80;
location /TicketBookingImpl.bookTicket {
grpc_pass grpc://grpc_backend;
}
}
}The above configuration file specifies that Ngnix is listening on port 80 and forwarding the incoming requests to the “bookTicket” method of Service TicketBookingImpl service and the grpc_pass specifies that the gPRC gateway is running on port 9,000.
If we are testing with Ngnix, we can start the gRPC server and gRPC-Gateway and send the request to the Ngnix proxy. For example:
POST http://localhost/TicketBookingImpl.bookTicket
Content-Type: application/json
{
'{"id": "1","userName":" Utkarsh", "email":"knoldus@gmail.com",
"mobileNumber":"000000"}'
}
This request will be forwarded to the gRPC-Gateway that will translate it into a gRPC request and send it to the gRPC server. The server will process the request and return with some response which will be converted back into HTTP/JSON response by the gateway. Finally, the Ngnix proxy will send the response back to the client.
gRPC gateway
We are using the gRPC gateway to create gRPC services and clients, making it usable over REST at the same time.
gRPC gateway in Scala requires 3 major steps. These include:
- Modifying the .proto file and generating the code
- Implementing the Service
- Starting the gRPC gateway reverse proxy
- Modifying the proto file:
gRPC gateway uses HTTP annotation. So, we first need to modify our proto file and import google/api/annotations.proto.
import "google/api/annotations.proto";For defining the HTTP method, URL or request body options are used in Protocol Buffer’s definition of an RPC method on a service.
// a gRPC service
service TicketBooking {
// SayHello is a rpc call and a option is defined for it
rpc BookTicket (TicketBookingRequest) returns (TicketBookingResponse) {
// option type is http
option (google.api.http) = {
// this is url, for RESTfull/JSON api and method
// this line means when a HTTP post request comes with
"/knoldus/bookTicket" call this rpc method over this service
post: "/knoldus/bookTicket"
body: "*"
};
}In the above example, post is the HTTP method for request and /knoldus/bookTicket is the response.
- Implementing the Server:
After modifying the service proto file, we need to add and implement the service class class using Scala.
package com.knoldus.grpc.connect
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
class TicketBookingImpl extends TicketBooking {
override def bookTicket(ticketRequest: TicketBookingRequest):
Future[TicketBookingResponse] = {
Future.successful(TicketBookingResponse(ticketRequest.userDetails))
}
}- Starting the gRPC Gateway reverse proxy server:
Finally, we will start the GRPC gateway reverse proxy.
import io.grpc.ManagedChannelBuilder
import io.grpc.netty.NettyChannelBuilder
import io.grpc.stub.StreamObserver
object TicketBookingGateway {
def main(args: Array[String]): Unit = {
val channel = NettyChannelBuilder.forAddress("localhost", 8080).usePlaintext().build()
val service = TicketBookingServiceGrpc.stub(channel)
val server = new TicketBookingServer(service)
server.start(8081)
}
}
class TicketBookingServer(service: TicketBookingServiceStub)
{
val server = Http.server.serve(":8081",
TicketBookingGatewayService(service))
Await.ready(server)
}
object TicketBookingGatewayService {
def apply(service: TicketBookingServiceStub) = {
HttpRoutes.of[Task] {
case request @ Method.POST -> Root / "bookTicket" / user =>
val bookTicketRequest = TicketBookingRequest(user)
val bookTicketResponse = service.bookTicket(bookTicketRequest).unsafeRunSync()
Ok(bookTicketResponse)
}
}
}With the gateway server running, you can test your gRPC service using HTTP requests to the gateway endpoints, for example,
curl -X POST http://localhost:8080/bookTicket \
-H "Content-Type: application/json" \
-d '{"id": "1","userName":" Utkarsh", "email":"knoldus@gmail.com",
"mobileNumber":"000000"}'
gRPC Backend
In the implementation part of the gRPC Backend, we will define our TicketBooking service. Akka gRPC automatically looks for .proto files in src/main/protobuf. When we compile our project, the sbt-akka-grpc compiler will pick up the proto file from src/main/protobuf directory and generate the respective classes inside the target folder in the directory specified in the proto file.
So, our service file will look like this:
- TicketBooking.proto
syntax = "proto3";
import "google/protobuf/timestamp.proto";
option java_multiple_files = true;
option java_package = "com.knoldus.grpc.connect";
option java_outer_classname = "BookingProto";
package connect;
// The TicketBooking service definition.
service TicketBooking {
// book a ticket and get response
rpc BookTicket (TicketBookingRequest) returns (TicketBookingResponse) {}
// GetTicket
rpc GetTicket (TicketRequest) returns (TicketBookingResponse) {}
}
// The request message containing the payment details
message UserDetails{
string Id = 1;
string userName =2;
string email =3;
string mobileNumber =4;
}
// The request message containing the payment details
message PaymentDetails{
string ticketId = 1;
string paymentMethod =2;
string transactionId =3;
string ccCard =4;
string pin =5;
string status=6;
string userid =7;
string paymentMode=8;
string reason=9;
}
// The request message containing the ticket details
message TicketDetails{
string uuid = 1;
string threatre =2;
string movieName =3;
string seatNumber =4;
string movieTime =5;
string address =6;
string version =7;
string status=8;
string reason=9;
}
// The request message containing the ticket details
message TicketBookingRequest{
UserDetails userDetails = 1;
TicketDetails ticketDetails =2;
PaymentDetails paymentDetails =3;
optional google.protobuf.Timestamp timestamp = 4;
}
// The request message containing the ticket details
message TicketBookingResponse {
UserDetails userDetails = 1;
TicketDetails ticketDetails =2;
string status=3;
optional google.protobuf.Timestamp timestamp = 4;
}
// The request message containing the ticket details
message TicketRequest{
string uuid = 1;
}Save the above file with “TicketBooking.proto” and run “sbt compile” and your scala compiler will generate the Handlers and other Classes in the target directory. There is one Scala source file for each message defined by the TicketBooking service.
Now, write the Business logic which will extend the TicketBooking service generated in the target directory and implement the service methods.
- TicketBookingImpl.scala
package com.knoldus.grpc.connect
import akka.NotUsed
import akka.actor.typed.ActorSystem
import akka.stream.scaladsl.{BroadcastHub, Keep, MergeHub, Sink, Source}
import scala.concurrent.Future
class TicketBookingImpl(system: ActorSystem[_]) extends TicketBooking {
private implicit val sys: ActorSystem[_] = system
val (inboundHub: Sink[TicketBookingRequest, NotUsed], outboundHub:
Source[TicketBookingResponse, NotUsed]) =
MergeHub.source[TicketBookingRequest]
.map(request => TicketBookingResponse(request.userDetails))
.toMat(BroadcastHub.sink[TicketBookingResponse])(Keep.both)
.run()
override def bookTicket(ticketRequest: TicketBookingRequest):
Future[TicketBookingResponse] = {
Future.successful(TicketBookingResponse(ticketRequest.userDetails))
}
override def getTicket(in: TicketRequest): Future[TicketBookingResponse] =
{
Future.successful(TicketBookingResponse())
}
}
- TicBookingClient.scala
The gRPC client is written in TicBookingClient.scala in which TicketBookingClient is invoked using GrpcClientSettings and calls the request methods.
package com.knoldus.grpc.connect
import akka.actor.typed.ActorSystem
import akka.actor.typed.scaladsl.Behaviors
import akka.grpc.GrpcClientSettings
import scala.concurrent.{ExecutionContext, Future}
import scala.util.{Failure, Success}
object TicBookingClient {
def main(args: Array[String]): Unit = {
implicit val sys: ActorSystem[_] = ActorSystem(Behaviors.empty, "TicketBookingClient")
implicit val ec: ExecutionContext = sys.executionContext
val client = TicketBookingClient(GrpcClientSettings.fromConfig("connect.TicketBooking"))
// creating object of user
val user1 = Some(UserDetails("123","Utkarsh","utk@gmail.com","8173..."))
singleRequestReply(user1)
def singleRequestReply(user : Option[UserDetails]): Unit = {
println(s"Performing request: $user")
val reply = client.bookTicket(TicketBookingRequest(user))
reply.onComplete {
case Success(msg) =>
println(msg)
case Failure(e) =>
println(s"Error: $e")
}
}
}
}
TicketBookingServer.scala
Next, we implement the gRPC service definition. Given below is the service implementation in TicketBookingService.scala. This will implement all the gRPC service functions defined in the TicketBooking proto file.
package com.knoldus.grpc.connect
import akka.actor.typed.ActorSystem
import akka.actor.typed.scaladsl.Behaviors
import akka.http.scaladsl.{ConnectionContext, Http, HttpsConnectionContext}
import akka.http.scaladsl.model.{HttpRequest, HttpResponse}
import akka.pki.pem.{DERPrivateKeyLoader, PEMDecoder}
import com.typesafe.config.ConfigFactory
import java.security.{KeyStore, SecureRandom}
import java.security.cert.{Certificate, CertificateFactory}
import javax.net.ssl.{KeyManagerFactory, SSLContext}
import scala.concurrent.duration.DurationInt
import scala.concurrent.{ExecutionContext, Future}
import scala.io.Source
import scala.util.{Failure, Success}
object TicketBookingServer {
def main(args: Array[String]): Unit = {
val conf = ConfigFactory.parseString("akka.http.server.preview.enable-http2 = on")
.withFallback(ConfigFactory.defaultApplication())
val system = ActorSystem[Nothing](Behaviors.empty, "TicketBookingServer", conf)
new TicketBookingServer(system).run()
}
}
case class TicketBookingServer(system: ActorSystem[Nothing]) {
def run():Future[Http.ServerBinding]= {
implicit val sys = system
implicit val ec: ExecutionContext = system.executionContext
val service: HttpRequest => Future[HttpResponse] =
TicketBookingHandler(new TicketBookingImpl(system))
val bound: Future[Http.ServerBinding] = Http(system)
.newServerAt(interface = "127.0.0.1", port = 8080)
.enableHttps(serverHttpContext)
.bind(service)
.map(_.addToCoordinatedShutdown(hardTerminationDeadline = 10.seconds))
bound.onComplete {
case Success(binding) =>
val address = binding.localAddress
println("gRPC server bound to {}:{}", address.getHostString, address.getPort)
case Failure(ex) =>
println("Failed to bind gRPC endpoint, terminating system", ex)
system.terminate()
}
bound
}
private def serverHttpContext: HttpsConnectionContext = {
val privateKey =
DERPrivateKeyLoader.load(PEMDecoder.decode(readPrivateKeyPem()))
val fact = CertificateFactory.getInstance("X.509")
val cer = fact.generateCertificate(
classOf[TicketBookingServer].getResourceAsStream("/certs/server1.pem")
)
val ks = KeyStore.getInstance("PKCS12")
ks.load(null)
ks.setKeyEntry(
"private",
privateKey,
new Array[Char](0),
Array[Certificate](cer)
)
val keyManagerFactory = KeyManagerFactory.getInstance("SunX509")
keyManagerFactory.init(ks, null)
val context = SSLContext.getInstance("TLS")
context.init(keyManagerFactory.getKeyManagers, null, new SecureRandom)
ConnectionContext.https(context)
}
private def readPrivateKeyPem(): String =
Source.fromResource("certs/server1.key").mkString
}
Conclusion
Overall, this was just a simple example of a Ticket Booking Application using Nginx and the gRPC gateway with Scala. Using these (gRPC gateway and Connect Lib) you can create applications which can be beneficial for both gRPC and REST APIs. With gRPC, you can build a high-performance and scalable micro-services architecture that can handle large volumes of traffic and complex data structures, while with REST APIs, you can provide a more accessible and user-friendly interface for your clients. Finally, building a Ticket Booking Application using gRPC Gateway in Scala and Nginx can be a great learning experience for anyone interested in building distributed systems using modern technologies and best practices.
References
https://www.beyondthelines.net/computing/grpc-rest-gateway-in-scala/
