Home Artificial Intelligence Utilizing the OpenAI API for Detection of SMS Spam Messages Creating an Open AI Account Accessing OpenAI with Python Designing the Prompt Reading the SMS Spam Collection Dataset Utilizing the GPT-3.5-turbo Model for Spam Detection in Messages Project Summary

Utilizing the OpenAI API for Detection of SMS Spam Messages Creating an Open AI Account Accessing OpenAI with Python Designing the Prompt Reading the SMS Spam Collection Dataset Utilizing the GPT-3.5-turbo Model for Spam Detection in Messages Project Summary

0
Utilizing the OpenAI API for Detection of SMS Spam Messages
Creating an Open AI Account
Accessing OpenAI with Python
Designing the Prompt
Reading the SMS Spam Collection Dataset
Utilizing the GPT-3.5-turbo Model for Spam Detection in Messages
Project Summary

Unlocking efficient text classification with pre-trained models: a case study using OpenAI’s GPT-3.5-turbo

Towards Data Science
Image by https://unsplash.com/es/@tma

Traditionally, any natural language processing text classification project would start with gathering instances, defining their respective labels, and training a classification model, reminiscent of a logistic regression model, to categorise the instances. Currently, the models available in OpenAI may be directly used for classification tasks that may typically require collecting a considerable amount of labeled data to coach the model. These pre-trained models may be used for multiple text-processing tasks, including classification, summarization, spell-checking, and keyword identification.
We don’t require any labeled data or the necessity to train a model. Easy, right?

ChatGPT provides a graphical interface for the models implemented by OpenAI. Nevertheless, what if we wish to run those models directly in Python? Well, the available alternative is the OpenAI API, which allows us to access their models from a programming environment. In this text, we’ll describe with a transient example how we are able to access the API to detect whether an SMS is spam or not. To perform this, we’ll utilize one among the Open AI models, specifically the GPT-3.5-turbo model.

The initial step to access the OpenAI API involves creating an account on OpenAI to acquire the API key required for accessing the models. Upon creating the account, we’ll have $5 of credit at our disposal, which, as we’ll later observe, will allow us to conduct quite a few tests.

In this instance, we’ll utilize the free version of OpenAI, which comes with limitations on requests per minute and per day. Adhering to those limits is crucial to avoid Rate Limit Errors. The values for these two parameters are set at 3 requests per minute and 200 per day. While this naturally imposes constraints, particularly for large-scale projects, it suffices for the needs of this text’s example.

Once we have now created the account, we are able to access the OpenAI models available for the free version from Python using the OpenAI library. First, we create a function called chat_with_gpt to access the GPT-3.5-turbo model. The input to this function will likely be the prompt, which we’ll design afterward.

The following step is to create the prompt that we are going to provide to the GPT-3.5-turbo model. On this particular case, we’re occupied with two things. Firstly, a worth between 0 and 1 indicating the probability of the SMS being spam or not, and secondly, a proof of the model’s reasoning behind that call. Moreover, we desire the lead to JSON format in order that we are able to later convert it right into a dataframe. Below is the template we’ll use for these predictions.

Now, we want some messages to check how well the OpenAI model predicts whether a message is spam or not. For this purpose, we’ll use the SMS Spam Collection dataset available on the UCI Machine Learning Repository.

We read the info and convert it right into a DataFrame. As observed, the dataset consists of two columns: one containing the messages and the second containing the corresponding labels. ham indicates that the message is just not spam, while spam indicates that it’s.

Spam Collection dataset (Image created by the writer)

Now, we’ll use the model to detect whether the messages are spam or not and evaluate how well the pre-trained OpenAI model can predict this problem. As mentioned at the start of the article, there are significant limitations regarding the variety of requests per minute and per day that we are able to make with the free version. The dataset accommodates 5,574 instances, but to check the model, we’ll only use the primary 50 instances. If you’ve gotten a paid version of OpenAI, you possibly can increase the variety of messages tested because the time limitations are much lower.

Labels of instances (Image created by the writer)

Before making predictions with the model, we have now verified that our dataset of fifty instances accommodates each messages which are spam and messages that aren’t. In total, there are 40 non-spam messages and 10 spam messages.

Finally, we proceed to make the prediction. As shown below, we have now kept track of the available credits and tokens in any respect times. Moreover, we have now implemented a sleep function within the code for 60 seconds to make sure compliance with the restrictions regarding the variety of requests.

The message provided by the model with the prediction has been stored in a variable called prediction. This message is a JSON file with the next structure: { “spam”: “0.1”, “reasoning”: “The message appears to be a typical promotional message a few buffet offer and doesn't contain any typical spam keywords or characteristics. The probability of this message being spam is low.” }.

Testing output (Image created by the writer)

The predictions yield a worth between 0 and 1. A worth of 0 indicates that the message is just not spam, while a worth of 1 indicates that it’s. To assign the labels, we’ll use a threshold of 0.5, meaning that a rating higher than 0.5 by the model will categorize the message as spam.

Predicted Labels (Image created by the writer)

Now, all that’s left is to match the actual labels with the expected labels and assess how well the GPT-3.5-turbo model has performed the predictions.

Confusion Matrix (Image created by the writer)

Above is the confusion matrix of the model.

  • For messages that aren’t spam (ham), the model accurately predicted 37 of them as not spam (true negatives), but misclassified 3 of them as spam (false positives).
  • For messages which are spam, the model accurately predicted 10 of them as spam (true positives), with no observed false negatives on this case.

The model demonstrates good sensitivity in detecting spam messages, with a couple of false positives indicating potential areas for improvement in its precision. As shown, the model achieved a 94% accuracy rate, accurately classifying 47 out of fifty instances.

Since we’ve requested the model to supply us with information regarding its reasoning for classifying a message as spam or not, we are able to examine why it misclassified 3 messages that don’t belong to the spam category. Below are the reasons provided by the model:

  • The message accommodates unusual language and grammar mistakes commonly related to spam messages. It also mentions sensitive topics like AIDS, which is a standard tactic utilized in spam messages to evoke emotions and prompt a response. Subsequently, there may be a high probability of this message being spam.
  • The message accommodates suggestive and inappropriate content, which is a standard characteristic of spam messages. Moreover, the usage of improper grammar and language may indicate that this message is spam. Subsequently, the probability of this message being spam is high.
  • The message accommodates suggestive content related to private attributes, which is a standard characteristic of spam messages. Moreover, the usage of explicit language increases the likelihood of this message being considered spam.

Messages with inappropriate content or quite a few spelling errors are inclined to be classified as spam.

Traditionally, projects requiring text classification began with labeled databases and a model that needed training. The emergence of pre-trained Language Models (LLMs) now offers the potential for classifying a large number of texts without the necessity to train a model beforehand, as these models have already been trained for a lot of use cases. In this text, we have now explained the best way to utilize the OpenAI API to access the GPT-3.5-turbo model and determine whether a set of messages is spam or not. The model was in a position to classify 94% of the instances accurately, which indicates a high level of accuracy. In the long run, it could possibly be worthwhile to guage other OpenAI models and explore different prompts that will yield higher performance. By the best way, the execution of the project only costed $0.007.

LEAVE A REPLY

Please enter your comment!
Please enter your name here