Learn the right way to simply mix Ray Serve Deployment with asynchronous Kafka Consumer

Ray is a contemporary open source framework that means that you can create distributed applications in Python with ease. You possibly can create easy training pipelines, do hyperparameter tuning, data processing and model serving.

Ray means that you can create online inference APIs with Ray Serve. You possibly can easily mix several ML models and custom business logic in a single application. Ray Serve routinely creates an HTTP interface in your deployments, taking good care of fault tolerance and replication.

But there may be one thing that Ray Serve misses for now. Many modern distributed applications communicate through Kafka, but there is no such thing as a out-of-the-box option to connect Ray Serve service to Kafka topic.

But don’t panic. It’ll not take an excessive amount of effort to show Ray Serve to speak with Kafka. So, let’s begin.

To begin with we are going to need to organize our local environment. We are going to use a docker-compose file with Kafka and Kafdrop UI docker containers to start out and explore our local Kafka instance (so we assume that you could have Docker and Docker Compose installed). Also we are going to need to put in some Python requirements to get the work done:

All requirements might be downloaded by this link.

Now we are going to create a ray-consumer.py file with Ray Deployment that shall be served with Ray Serve. I won’t go into details about Ray Serve concepts, as you possibly can examine that within the documentation. Mainly it takes the standard Python class and converts it to a asynchronous service Ray Deployment with @serve.deployment decorator: