Deploying Pre-Trained Sklearn Models on Amazon SageMaker
Written on
Chapter 1: Introduction to Model Deployment
In previous discussions, I covered the processes for training and deploying custom Sklearn and TensorFlow models using Amazon SageMaker. However, there are instances when you may have pre-trained models that were developed in different environments. This article will guide you on how to take these pre-trained models and deploy them on SageMaker. Throughout this example, we will primarily use code in a local setting to minimize complexity within the AWS console.
Note: If you're new to AWS, please create an account at the provided link to follow along. This article assumes a basic to intermediate understanding of AWS and SageMaker, and it will concentrate mainly on the deployment aspect rather than model building theory.
Table of Contents
- Setup
- Model Deployment Script
- Model Invocation
- Additional Resources & Conclusion
Section 1.1: Setting Up Your Environment
To begin, we will locally train a Sklearn model that we can subsequently deploy. You can execute the following Python script to set everything up.
The key element in this script is the use of the joblib module, which is essential for saving the trained model. After execution, you should see this artifact in your local directory.
Using joblib is crucial as SageMaker expects models in this format. Along with your local model script, you should also prepare an inference script in advance. This script provides SageMaker with the necessary information on how your model will handle input and output during deployment. You can implement four default functions:
- model_fn: Deserializes and loads the joblib file.
- input_fn: Specifies the expected data format (e.g., JSON, CSV).
- predict_fn: Contains the model's prediction logic.
- output_fn: Processes the output from predict_fn to align with the endpoint's response structure.
Next, you will need to create an appropriate IAM role for SageMaker within the AWS console. Ensure you grant the necessary permissions: S3 Full Access, SageMaker Full Access, and ECR Full Access. This setup will be your main task within the AWS console for this example. Once the role is established, we can proceed to construct our model deployment script.
Section 1.2: Crafting the Model Deployment Script
To initiate the deployment, include the necessary imports in your model deployment script. Ensure that this script resides in the same directory as your inference handler and model data (joblib).
Most of our interaction with AWS services is facilitated through the Python SDK, Boto3. We will utilize the Boto3 client for SageMaker to streamline our model deployment steps.
We start by creating clients for both SageMaker and S3, which will host our model data for SageMaker's access. A crucial requirement is that SageMaker expects model artifacts in a model.tar.gz format. Thus, we will compress the local model artifact and the inference.py script into a tar file.
Following this, we need to upload the model artifact to an S3 location for SageMaker to access.
Now, we can outline the three essential steps for building an endpoint on SageMaker:
- Model Creation
- Endpoint Configuration Creation
- Endpoint Creation
For the model creation, we will need two components: the model data and the container image. We can retrieve the Sklearn inference image directly from SageMaker using the SageMaker SDK.
Once we have the model data and the image, we can proceed to create our SageMaker model. From there, we will use this model to establish our Endpoint Configuration, allowing us to specify the instance type and count for the endpoint.
After this process, we will create the endpoint, which may take a few minutes to become operational. Upon running the script, you should see the successfully created endpoint reflected in your AWS console.
Section 1.3: Testing the Model Invocation
We can develop a separate invoke script to test our endpoint with a sample input. This should not be integrated into the main file to avoid creating a new endpoint with every execution. Simply obtain your endpoint name and specify it within the following script to witness the execution.
Chapter 2: Conclusion and Additional Resources
For the complete code example, please refer to the link provided earlier. Deploying pre-trained models on SageMaker becomes straightforward once you grasp the required model data format and structure. The primary adjustment involves selecting the appropriate image using the SDK, which should correspond to the framework you are utilizing. If the container you need isn't supported by SageMaker, consider exploring options to Bring Your Own Container.
Additional Resources
- SageMaker Inference Examples
- Multi-Model TensorFlow Endpoints with SageMaker
If you found this article helpful, feel free to connect with me on LinkedIn and subscribe to my Medium Newsletter. If you’re new to Medium, consider signing up using my Membership Referral.
Learn how to build a Sklearn model using AWS Sagemaker and a custom training script in this informative video.
This video covers the training of scikit-learn models on Amazon SageMaker, providing practical insights into the process.