从HTTP性能基准测试谈Vert.x高性能的秘密：从JIT编译到网络优化 英文

2020-03-01 819浏览

  • 1.Real-world HTTP performance benchmarking, lessons learned Julien Viet QCon Shangai
  • 2.
  • 3.Once upon a time
  • 4.Round #8
  • 5.Every one was happy
  • 6.But one day...
  • 7.Round #14
  • 8.Round #14
  • 9.Round #14
  • 10.Round #14
  • 11.Round #14
  • 12.Real-world HTTP performance benchmarking, lessons learned
  • 13.Julien Viet Open source developer for 16+ years @vertx_project lead Principal software engineer at Marseille JUG Leader !https://www.julienviet.com/"http://github.com/vietj# @julienviet https://www.mixcloud.com/cooperdbi/
  • 14.Eclipse Vert.x Open source project started in 2012 Eclipse / Apache licensing A toolkit for building reactive applications for the JVM 8K ⋆ on " Built on top of !https://vertx.io# @vertx_project
  • 15.Techempower Framework Benchmark ✓ Performance of production grade deployments of real-world application frameworks and platforms ✓ 464 frameworks - 26 languages ✓ Community ✓ Physical of contributors on GitHub server or cloud (Azure)
  • 16.6 benchmarks ✓ "/plaintext", "/json" # ✓ "/db", ! "/queries", "/updates", "/fortunes" # ! "
  • 17.Things to remember ✓ Benchmarking is hard ✓ Benchmarking != load testing ✓ Measure ✓ Be critic don't guess
  • 18.The lab
  • 19.
  • 20.
  • 21.
  • 22./plaintext
  • 23.Benchmark ✓ Simple Hello World ✓ 16,384 concurrent connections ✓ HTTP ✓ No pipelining (16) back-end ✓ Heavily CPU bound
  • 24.Keep-alive GET OK PUT OK GET OK
  • 25.Head of line blocking
  • 26.Pipelining GET PUT GET OK OK OK
  • 27.Our weapons ✓ Async-profiler ✓ Jitwatch ✓ Wireshark + Flame graphs
  • 28.Code inlining
  • 29.process error process request process body
  • 30.process error process request process body
  • 31.process error process request process body reduce method size to favor inlining
  • 32.b2073fa091d64a1dfe06699bca1a8befddb5a805 process error process request process body $ 2. inline by hand
  • 33.Batch to amortise costs
  • 34.chctx.fireChannelRead(msg) // class VertxHandler void channelRead(Object msg) { Connection conn = getConnection(); Context ctx = conn.getContext(); context.executeFromIO(conn::startRead());channelRead(conn, msg); } // class VertxHttpHandler extends VertxHandler void startRead() { ... } void channelRead(Connection conn, Object msg) { conn.handleMessage(msg); } void handleMessage(Object msg) { ... }
  • 35.799df9e602eabcd51b56052e20cc7d05134!901 chctx.fireChannelRead(msg) Batch here // class VertxHandler public void channelRead(ChannelHandlerContext chctx, Object msg) { Connection conn = getConnection(); Context ctx = conn.getContext(); context.executeFromIO(() -> { conn.startRead(); conn.handleMessage(msg); }); } void startRead() { } void handleMessage(Object msg) { ... }
  • 36.The fastest code is the code that never runs
  • 37.Netty Vert.x Application req.response() .end("Hello World");
  • 38.Netty Vert.x void end(Bu!er bu!er) { FullHttpResponse msg = ... queueForWrite(msg); } Application req.response() .end("Hello World");
  • 39.Netty Vert.x void end(Bu!er bu!er) { FullHttpResponse msg = ... queueForWrite(msg); } void queueForWrite(Object msg) { needsFlush = true; channel.write(encode(obj)); } Application req.response() .end("Hello World");
  • 40.Netty Vert.x void end(Bu!er bu!er) { FullHttpResponse msg = ... queueForWrite(msg); } ChannelFuture write(Object msg) { return pipeline.write(msg); void queueForWrite(Object msg) { } needsFlush = true; channel.write(encode(obj)); } Application req.response() .end("Hello World");
  • 41.Netty Vert.x void end(Bu!er bu!er) { FullHttpResponse msg = ... queueForWrite(msg); } ChannelFuture write(Object msg) { return pipeline.write(msg); void queueForWrite(Object msg) { } needsFlush = true; channel.write(encode(obj)); } // default implementation (inherited) void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) { ctx.write(msg, promise); } Application req.response() .end("Hello World");
  • 42.Netty Vert.x void end(Bu!er bu!er) { FullHttpResponse msg = ... queueForWrite(msg); } ChannelFuture write(Object msg) { return pipeline.write(msg); void queueForWrite(Object msg) { } needsFlush = true; channel.write(encode(obj)); } void write(Object msg, ChannelPromise promise) { next.invoke(msg, promise) } // default implementation (inherited) void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) { ctx.write(msg, promise); } Application req.response() .end("Hello World");
  • 43.Netty Vert.x void end(Bu!er bu!er) { FullHttpResponse msg = ... queueForWrite(msg); } ChannelFuture write(Object msg) { return pipeline.write(msg); void queueForWrite(Object msg) { } needsFlush = true; channel.write(encode(obj)); } void write(Object msg, ChannelPromise promise) { next.invoke(msg, promise) } // default implementation (inherited) void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) { ctx.write(msg, promise); } Application req.response() .end("Hello World");
  • 44.Netty the fastest code is the code that never runs void write(Object msg, ChannelPromise promise) { next.invoke(msg, promise) } Vert.x void end(Bu!er bu!er) { FullHttpResponse msg = ... queueForWrite(msg); } Application req.response() .end("Hello World"); void queueForWrite(Object msg) { needsFlush = true; chctx.write(encode(obj)); } 217b17c78cd54103ae98557510a7ac431e17c5ea
  • 45.Reduce object allocation
  • 46.Netty Vert.x void end(Bu!er bu!er) { FullHttpResponse msg = ... queueForWrite(msg); } void write(Object msg) { write(msg, newPromise()); } void write(Object msg, ChannelPromise promise) { next.invoke(msg, promise) } void queueForWrite(Object msg) { needsFlush = true; chctx.write(encode(obj)); } Application req.response() .end("Hello World");
  • 47.Netty Vert.x void end(Bu!er bu!er) { FullHttpResponse msg = ... queueForWrite(msg); } void write(Object msg) { write(msg, newPromise()); } void write(Object msg, ChannelPromise promise) { next.invoke(msg, promise) } void queueForWrite(Object msg) { needsFlush = true; chctx.write(encode(obj)); } Application req.response() .end("Hello World");
  • 48.Netty Vert.x void end(Bu!er bu!er) { FullHttpResponse msg = ... queueForWrite(msg); } void queueForWrite(Object msg) { needsFlush = true; chctx.write(obj, channel.voidPromise()); } void write(Object msg, ChannelPromise promise) { next.invoke(msg, promise) } Application req.response() .end("Hello World"); reduce GC allocation by using VoidPromise 6b9788dec6e1147782a3a7017ead067778095cba
  • 49.Cache expensive operations
  • 50.void setConnection(Connection conn) { this.conn = conn; } Vert.x void channelReadComplete(ChannelHandlerContext ctx) { Runnable task =conn::endReadAndFlush();// Need to use executeFromIO to avoid race conditions context.executeFromIO(task); } void endReadAndFlush() { if (needFlush) { needFlush = false; channel.flush(); } }
  • 51.void setConnection(Connection conn) { this.conn = conn; } Vert.x void channelReadComplete(ChannelHandlerContext ctx) { Runnable task =conn::endReadAndFlush();// Need to use executeFromIO to avoid race conditions context.executeFromIO(task); } void endReadAndFlush() { if (needFlush) { needFlush = false; channel.flush(); } } Called for every each flush
  • 52.void setConnection(Connection conn) { this.conn = conn; this.task =conn::endReadAndFlush();'>conn::endReadAndFlush();