Making gRPC-Web less mysterious

If you are reading this, I’m fairly confident you might have heard, over-heard or unintentionally stumbled on the term gRPC.

With gRPC you can define your service once in a .proto file and implement clients and servers in any of gRPC’s supported languages, which in turn can be run in environments ranging from servers inside a large data center to your own tablet - all the complexity of communication between different languages and environments is handled for you by gRPC.

You also get all the advantages of working with protocol buffers, including efficient serialization, a simple IDL, and easy interface updating.

gRPC-Web lets you access gRPC services built in this manner from browsers using an idiomatic API.

Learn more: https://grpc.io/docs/what-is-grpc/introduction/

The topic is vast and there's a lot to cover. This blog post however is targeted at unravelling the mysteries of grpc-Web.

What is grpc-Web? You may wonder.

A JavaScript implementation of gRPC for browser clients. gRPC-web clients connect to gRPC services via a special proxy; by default, gRPC-web uses Envoy.

In other words, grpc-Web enables your client app/browser app communicating over HTTP 1.1 to successfully communicate to the gRPC services with gRPC working on top of HTTP 2 protocol.

But, how is this possible? Where and how does the protocol shift take place?

The answer lies in the proxy used: in this case Envoy.

gRPC-Web is supported by a filter that allows a gRPC-Web client to send requests to Envoy over HTTP/1.1 and get proxied to a gRPC server.

(More support present in Envoy for gRPC: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/other_protocols/grpc )

So with those concepts arranged rightly in our head, let's now deploy a gRPC Web application and see the actual packets! Follow along.

Step 1: Deploy grpc-Web services

Follow the steps here: https://grpc.io/docs/platforms/web/quickstart/

>git clone https://github.com/grpc/grpc-web
>cd grpc-web
>docker-compose pull prereqs node-server envoy commonjs-client
>docker-compose up -d node-server envoy commonjs-client

The last command spins up three docker containers: one for envoy, one for the grpc node server and one for common utilities.

Now visit: http://localhost:8081/echotest.html

You'll see the example test app.

Step 2: Send a message "Hello" and intercept the request/response in Burp Suite

First call is to OPTIONS:

The Access-Control-Request-Headers request header is used by browsers when issuing a preflight request to let the server know which HTTP headers the client might send when the actual request is made (such as with setRequestHeader()). The complementary server-side header of Access-Control-Allow-Headers will answer this browser-side header.

Note the values for these headers and you can spot some grpc specific headers:

Access-Control-Request-Headers: content-type,custom-header-1,x-grpc-web,x-user-agent

Access-Control-Allow-Headers: keep-alive,user-agent,cache-control,content-type,content-transfer-encoding,custom-header-1,x-accept-content-transfer-encoding,x-accept-response-streaming,x-user-agent,x-grpc-web,grpc-timeout

The second call is a POST request to GRPC endpoint

Note that it's HTTP 1.1 , with interesting grpc headers:

Content-Type: application/grpc-web-text

Accept: application/grpc-web-text

X-Grpc-Web: 1

The text "AAAAAAcKBWhlbGxv" is Base64 encoded "Hello"

Now, look at the reponse:

content-type: application/grpc-web-text+proto

grpc-accept-encoding: identity

grpc-encoding: identity

The response value:

AAAAAAcKBWhlbGxvgAAAAmxncnBjLXN0YXR1czowDQpncnBjLW1lc3NhZ2U6T0sNCnNlYy1jaC11YToiIE5vdCBBO0JyYW5kIjt2PSI5OSIsICJDaHJvbWl1bSI7dj0iOTYiDQp4LXVzZXItYWdlbnQ6Z3JwYy13ZWItamF2YXNjcmlwdC8wLjENCnNlYy1jaC11YS1tb2JpbGU6PzANCmN1c3RvbS1oZWFkZXItMTp2YWx1ZTENCnVzZXItYWdlbnQ6TW96aWxsYS81LjAgKFdpbmRvd3MgTlQgMTAuMDsgV2luNjQ7IHg2NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzk2LjAuNDY2NC40NSBTYWZhcmkvNTM3LjM2DQphY2NlcHQ6YXBwbGljYXRpb24vZ3JwYy13ZWItdGV4dA0KeC1ncnBjLXdlYjoxDQpzZWMtY2gtdWEtcGxhdGZvcm06Im1hY09TIg0Kb3JpZ2luOmh0dHA6Ly9sb2NhbGhvc3Q6ODA4MQ0Kc2VjLWZldGNoLXNpdGU6c2FtZS1zaXRlDQpzZWMtZmV0Y2gtbW9kZTpjb3JzDQpzZWMtZmV0Y2gtZGVzdDplbXB0eQ0KcmVmZXJlcjpodHRwOi8vbG9jYWxob3N0OjgwODEvDQphY2NlcHQtbGFuZ3VhZ2U6ZW4tR0IsZW4tVVM7cT0wLjksZW47cT0wLjgNCngtZm9yd2FyZGVkLXByb3RvOmh0dHANCngtcmVxdWVzdC1pZDo1NmE1MWU5OC1hYjlkLTRkNTYtYTE5Mi04NWI1M2ViMDhmMTENCg==

is Base64 decoded as:


hello€lgrpc-status:0
grpc-message:OK
sec-ch-ua:" Not A;Brand";v="99", "Chromium";v="96"
x-user-agent:grpc-web-javascript/0.1
sec-ch-ua-mobile:?0
custom-header-1:value1
user-agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36
accept:application/grpc-web-text
x-grpc-web:1
sec-ch-ua-platform:"macOS"
origin:http://localhost:8081
sec-fetch-site:same-site
sec-fetch-mode:cors
sec-fetch-dest:empty
referer:http://localhost:8081/
accept-language:en-GB,en-US;q=0.9,en;q=0.8
x-forwarded-proto:http
x-request-id:56a51e98-ab9d-4d56-a192-85b53eb08f11

Step 3: What does the traffic look like in Envoy?

Get a shell on the envoy node & run tcpdump.

> tcpdump -vvv -s 0 'tcp port 8080 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'

Note the incoming & outgoing request from the proxy:

Note that these are similar values to what we saw in the Burp Suite.

However, one hop is missing, between the first & second log in the image above, the call is converted to HTTP2 and sent over the grpc node server.

How do I know that? Follow the next step.

Step 4: What does the traffic look like in gRPC node?

This is tricky. It might not be intuitive at first but tcpdump won't show you a lot here. HTTP 2 is all about binary, how do you make sense of things you'd see here, because you'd likely see gibberish like this:

Here's a tip, capture the pcap and extract it from the docker container & view it in Wireshark!

> tcpdump -s 0 port 9090 -i eth0 -w grpctraffic.pcap

Then, extract this file to your system to view it in Wireshark.

> docker cp ead29df0b605:/github/grpc-web/net/grpc/gateway/examples/echo/node-server/grpctraffic.pcap .

Now open it in Wireshark! However, note that you'd still not see a lot of details, you'd need to configure your wireshark to understand grpc and http 2.

Follow this to do that: https://grpc.io/blog/wireshark/

Once done, you'd be able to see the traffic in a much more meaningful sense:

Note, this is GRPC over HTTP2.

There are 24 headers in the incoming request to grpc server:

Header: :authority: localhost:8080

Header: :path: /grpc.gateway.testing.EchoService/Echo

Header: :method: POST

Header: :scheme: http

Header: sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="96", "Google Chrome";v="96"

Header: x-user-agent: grpc-web-javascript/0.1

Header: sec-ch-ua-mobile: ?0

Header: custom-header-1: value1

Header: user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36

Header: content-type: application/grpc

Header: accept: application/grpc-web-text

Header: x-grpc-web: 1

Header: sec-ch-ua-platform: "macOS"

Header: origin: http://localhost:8081

Header: sec-fetch-site: same-site

Header: sec-fetch-mode: cors

Header: sec-fetch-dest: empty

Header: referer: http://localhost:8081/

Header: accept-encoding: gzip, deflate, br

Header: accept-language: en-GB,en-US;q=0.9,en;q=0.8

Header: <unknown>:

Header: <unknown>: 9a23e6ee-45f8-42ee-ab18-91d925d9525a

Header: te: trailers

Header: grpc-accept-encoding: identity

Note some of these are populated with values from the HTTP 1.1 request, however some of them are totally HTTP 2 exclusive.

Here's the first look into protobuf message:

Note that it's built up of parts like field number, wiretype, value length, value, as described in the protobuf spec: https://developers.google.com/protocol-buffers/docs/encoding#structure

So, that was the sneak peek into end to end request flow of grpc-Web app. Hope you found it insightful. Should you have any further queries feel free to reach out to me & I'd be happy to engage.

Cheers ~