Skip to content
/ srpc Public
forked from sogou/srpc

RPC based on C++ Workflow. Supports Baidu bRPC, Tencent tRPC, thrift protocols.

License

Notifications You must be signed in to change notification settings

qchenzi/srpc

This branch is 506 commits behind sogou/srpc:master.

Folders and files

NameName
Last commit message
Last commit date

Latest commit

993e0a2 · Dec 16, 2021
Nov 19, 2021
Jun 20, 2021
Dec 16, 2021
Aug 16, 2021
Nov 4, 2021
Dec 2, 2021
Dec 15, 2021
Aug 9, 2020
Aug 15, 2021
Nov 4, 2021
Mar 27, 2021
Nov 12, 2021
Nov 26, 2021
Sep 6, 2021
Sep 14, 2020
Aug 16, 2021
Aug 9, 2020
Jul 31, 2021
Jul 31, 2021
Dec 15, 2021
Aug 9, 2020
Aug 9, 2020
Feb 9, 2021
Nov 12, 2021

Repository files navigation

中文版入口

SRPC

Language Build Status

Introduction

SRPC is an RPC system developed by Sogou. Its main features include:

  • Base on Sogou C++ Workflow, with the following features:
    • High performance
    • Low development and access cost
    • Compatible with SeriesWork and ParallelWork in Workflow
    • One-click migration for existing projects with protobuf/thrift
  • Support several IDL formats, including:
    • Protobuf
    • Thrift
  • Support several data formats, including:
    • Protobuf serialize
    • Thrift Binary serialize
    • JSON serialize
  • Support several compression formats transparently, including:
    • gzip
    • zlib
    • snappy
    • lz4
  • Support several communication protocols transparently, including:
    • tcp
    • http
    • sctp
    • ssl
    • https
  • With HTTP+JSON, you can use any language:
    • As a server, you can accept POST requests with HTTP server developed in any language and parse the HTTP headers.
    • As a client, you can send POST requests with HTTP client developed in any language and add the required HTTP headers.
  • Built-in client/server which can seamlessly communicate with a server/client in other RPC frameworks, including:
    • BRPC
    • TRPC (the only open-source implementation of TRPC protocol so far)
    • GRPC
    • Thrift Framed Binary
    • Thrift Http Binary
  • How to use it together with Workflow:
    • You can use the interface to create an RPC task
    • You can put the RPC task into SeriesWork or ParallelWork, and you can also get the current SeriesWork in the callback.
    • You can also use other features supported by Workflow, including upstream, calculation scheduling, asynchronous file IO, etc.
  • More features and layers

Installation

  • srpc is a static library, libsrpc.a. You only need to add the libsrpc as a dependency in the development environment, and it is not required in the compiled binary release.
  • srpc depends on Workflow and protobuf3.
    • For protobuf, you must install protobuf v3.0.0 or above by yourself.
    • For Workflow, it`s added as dependency automatically via git submodule.
    • For snappy and lz4, source codes are also included as third_party via git submodule.
git clone --recursive https://github.com/sogou/srpc.git
cd srpc
make
sudo make install

Tutorial

Easy to compile tutorial with these commands:

cd tutorial
make

Quick Start

1. example.proto

syntax = "proto3";// You can use either proto2 or proto3. Both are supported by srpc

message EchoRequest {
    string message = 1;
    string name = 2;
};

message EchoResponse {
    string message = 1;
};

service Example {
    rpc Echo(EchoRequest) returns (EchoResponse);
};

2. generate code

protoc example.proto --cpp_out=./ --proto_path=./
srpc_generator protobuf ./example.proto ./

3. server.cc

#include <stdio.h>
#include <signal.h>
#include "example.srpc.h"

using namespace srpc;

class ExampleServiceImpl : public Example::Service
{
public:
    void Echo(EchoRequest *request, EchoResponse *response, RPCContext *ctx) override
    {
        response->set_message("Hi, " + request->name());

        // gzip/zlib/snappy/lz4/none
        // ctx->set_compress_type(RPCCompressGzip);

        // protobuf/json
        // ctx->set_data_type(RPCDataJson);

        printf("get_req:\n%s\nset_resp:\n%s\n",
                request->DebugString().c_str(), response->DebugString().c_str());
    }
};

void sig_handler(int signo) { }

int main()
{
    signal(SIGINT, sig_handler);
    signal(SIGTERM, sig_handler);

    SRPCServer server_tcp;
    SRPCHttpServer server_http;

    ExampleServiceImpl impl;
    server_tcp.add_service(&impl);
    server_http.add_service(&impl);

    server_tcp.start(1412);
    server_http.start(8811);
    getchar(); // press "Enter" to end.
    server_http.stop();
    server_tcp.stop();

    return 0;
}

4. client.cc

#include <stdio.h>
#include "example.srpc.h"

using namespace srpc;

int main()
{
    Example::SRPCClient client("127.0.0.1", 1412);
    EchoRequest req;
    req.set_message("Hello, srpc!");
    req.set_name("workflow");

    client.Echo(&req, [](EchoResponse *response, RPCContext *ctx) {
        if (ctx->success())
            printf("%s\n", response->DebugString().c_str());
        else
            printf("status[%d] error[%d] errmsg:%s\n",
                    ctx->get_status_code(), ctx->get_error(), ctx->get_errmsg());
    });

    getchar(); // press "Enter" to end.
    return 0;
}

5. make

These compile commands are only for Linux system. On other system, complete cmake in tutorial is recommanded.

g++ -o server server.cc example.pb.cc -std=c++11 -lsrpc
g++ -o client client.cc example.pb.cc -std=c++11 -lsrpc

6. run

Terminal 1

./server

Terminal 2

./client
curl 127.0.0.1:8811/Example/Echo -H 'Content-Type: application/json' -d '{message:"from curl",name:"CURL"}'

Output of Terminal 1

get_req:
message: "Hello, srpc!"
name: "workflow"

set_resp:
message: "Hi, workflow"

get_req:
message: "from curl"
name: "CURL"

set_resp:
message: "Hi, CURL"

Output of Terminal 2

message: "Hi, workflow"
{"message":"Hi, CURL"}

Benchmark

  • CPU 2-chip/8-core/32-processor Intel(R) Xeon(R) CPU E5-2630 v3 @2.40GHz
  • Memory all 128G
  • 10 Gigabit Ethernet
  • BAIDU brpc-client in pooled (connection pool) mode

QPS at cross-machine single client→ single server under different concurrency

Client = 1
ClientThread = 64, 128, 256, 512, 1024
RequestSize = 32
Duration = 20s
Server = 1
ServerIOThread = 16
ServerHandlerThread = 16

IMG

QPS at cross-machine multi-client→ single server under different client processes

Client = 1, 2, 4, 8, 16
ClientThread = 32
RequestSize = 32
Duration = 20s
Server = 1
ServerIOThread = 16
ServerHandlerThread = 16

IMG

QPS at same-machine single client→ single server under different concurrency

Client = 1
ClientThread = 1, 2, 4, 8, 16, 32, 64, 128, 256
RequestSize = 1024
Duration = 20s
Server = 1
ServerIOThread = 16
ServerHandlerThread = 16

IMG

QPS at same-machine single client→ single server under different request sizes

Client = 1
ClientThread = 100
RequestSize = 16, 32, 64, 128, 256, 512, 1024, 2048, 4096, 8192, 16384, 32768
Duration = 20s
Server = 1
ServerIOThread = 16
ServerHandlerThread = 16

IMG

Latency CDF for fixed QPS at same-machine single client→ single server

Client = 1
ClientThread = 50
ClientQPS = 10000
RequestSize = 1024
Duration = 20s
Server = 1
ServerIOThread = 16
ServerHandlerThread = 16
Outiler = 1%

IMG

Latency CDF for fixed QPS at cross-machine multi-client→ single server

Client = 32
ClientThread = 16
ClientQPS = 2500
RequestSize = 512
Duration = 20s
Server = 1
ServerIOThread = 16
ServerHandlerThread = 16
Outiler = 1%

IMG

Contact

  • Email - [email protected] - main author
  • Issue - You are very welcome to post questions to issues list.
  • QQ - Group number: 618773193

About

RPC based on C++ Workflow. Supports Baidu bRPC, Tencent tRPC, thrift protocols.

Resources

License

Code of conduct

Stars

Watchers

Forks

Packages

No packages published

Languages

  • C++ 92.8%
  • CMake 4.1%
  • Starlark 2.0%
  • Other 1.1%