batch-inference-benchmarks

batch-inference-benchmarks copied to clipboard

Comparison feedback

3

Hey @amogkam ! Thanks for [blog post](https://www.anyscale.com/blog/offline-batch-inference-comparing-ray-apache-spark-and-sagemaker)! Although I found the Ray vs other services results impressive I decided to conduct my own experiments on AWS Sagemaker. After reviewing the...

morelen17

Hi ! I'm curious if it's possible to convert a tensorflow model to tensor rt and use after that Ray for a batch inference ? Thank you

vincentvic

See https://spark.apache.org/docs/3.4.0/api/python/reference/api/pyspark.ml.functions.predict_batch_udf.html and https://developer.nvidia.com/blog/distributed-deep-learning-made-easy-with-spark-3-4/. The current implementation seems to be subject to issues below. > The predict_batch_udf introduces standardized code for: > > - Translating Spark DataFrames into NumPy arrays,...

chenliu0831

Fix parquet file decoding in Sagemaker batch transform script

2

This PR fixes parquet file decoding in Sagemaker batch transform script. Please, re-run your Sagemaker experiments 🙂

morelen17