Senior Software Engineer, AI Gateway - Shanghai
Are you ready to unlock intelligence?
If you don’t think you meet all of the criteria below but are still interested in the job, please apply. Nobody checks every box - we’re looking for candidates that are particularly strong in a few areas, and have some interest and capabilities in others.
About the Role:
As Kong advances into the Agentic Era, the AI Gateway team is developing the vital infrastructure that links enterprise systems with generative AI technologies. In this position, you will focus on engineering the core system powering live AI traffic. You will take ownership of the runtime and data plane responsible for routing, translating, securing, and analyzing every request, ensuring performance, security, and scalability as AI workloads expand.
What You'll Do:
Responsible for designing, building and maintaining Kong’s AI Gateway runtime and its underlying data plane stack.
Architecture, design, and implement new innovative features including proxying capabilities, semantic/intelligent routing, load balancing, guardrails and compression, and identity, that extend Kong’s AI Gateway offering.
Build and maintain security, performance, and compliance foundations, including token cost management, access control, and low-level optimizations for production AI workloads.
Engage with the open source community and extend Kong core to support the latest industry trends.
Fill the gaps between Kong AI Gateway and third-party AI/ML providers, deployment platforms, and tooling.
And any additional tasks required by the manager.
What You'll Bring:
8+ years of programming in server-side applications and components.
Experience designing, prototyping, building and debugging applications that are highly scalable, distributed, reliable and resilient.
Solid hands-on low-level technical expertise in Linux systems troubleshooting, and profiling.
Experience in distributed systems, Linux networking stack and computer systems programming, Kong works with APIs on both L7 and L4 transports and protocols, so this expertise and knowledge are required.
Comfortable working with a remotely distributed team.
Strong sense of ownership and excellent verbal and written communication skills.
Familiarity with AI LLM concepts and applications development.
Bonus Points:
Hands-on experience with NGINX/OpenResty/Kong/LuaJIT.
Hands-on experience with Envoy or Istio.
Hands-on experience with Rust.
Hands-on experience leveraging AI agents to deliver intricate software systems efficiently and with superior quality.
About Kong:
Kong Inc., the AI Connectivity Company, is building the connectivity layer of AI. Trusted by the Fortune 500® and AI-native startups alike, Kong’s unified API and AI platform enables organizations to secure, manage, accelerate, govern, and monetize the flow of intelligence across APIs and AI traffic — on any model, any cloud. For more information, visit www.konghq.com.