Can We Use Scikit-learn and TensorFlow Together?

Last Updated : 23 Jul, 2025

In the world of machine learning, Scikit-learn and TensorFlow are two of the most popular libraries used for building and deploying models. While Scikit-learn excels in providing a wide range of tools for data preprocessing, model selection, and evaluation, TensorFlow shines in creating deep learning models with high flexibility and scalability. A common question that arises is whether these two powerful libraries can be used together in a single pipeline.

The answer is yes, and this article will demonstrate how to integrate Scikit-learn and TensorFlow into a cohesive workflow.

Why Combine Scikit-learn and TensorFlow?

  • Data Preprocessing: Scikit-learn offers extensive tools for data preprocessing, such as scaling, normalization, and encoding, which are crucial before feeding data into a TensorFlow model.
  • Model Evaluation: Scikit-learn's model evaluation tools, including cross-validation and various metrics, are highly useful for assessing the performance of a TensorFlow model.
  • Pipelines: Scikit-learn’s pipeline functionality can integrate TensorFlow models with preprocessing steps, making it easier to manage the entire workflow.

Combining Scikit-learn and TensorFlow: A Step-by-Step Guide

Step 1: Load and Prepare the Data

To illustrate the integration, we’ll use the classic Iris dataset, which is a simple yet effective dataset for classification tasks. The first step is to load and prepare the data.

import numpy as np
import pandas as pd
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from tensorflow.keras.utils import to_categorical

# Load the Iris dataset
iris = load_iris()
X = iris.data
y = iris.target

# One-hot encode the labels
y = to_categorical(y)

# Split the dataset into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

Step 2: Define a TensorFlow Model

Next, we’ll define a simple neural network model using TensorFlow’s Keras API. This model consists of an input layer, two hidden layers, and an output layer.

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.optimizers import Adam

def create_model():
    model = Sequential()
    model.add(Dense(64, input_dim=4, activation='relu'))  # Input layer
    model.add(Dense(64, activation='relu'))               # Hidden layer
    model.add(Dense(3, activation='softmax'))             # Output layer (3 classes)
    model.compile(optimizer=Adam(), loss='categorical_crossentropy', metrics=['accuracy'])
    return model

Step 3: Custom Keras Classifier

To integrate this TensorFlow model into a Scikit-learn pipeline, we need to create a custom classifier that can fit into Scikit-learn’s API. We can achieve this by subclassing BaseEstimator and ClassifierMixin from Scikit-learn.

from sklearn.base import BaseEstimator, ClassifierMixin

class CustomKerasClassifier(BaseEstimator, ClassifierMixin):
    def __init__(self, build_fn=None, epochs=1, batch_size=32, verbose=0):
        self.build_fn = build_fn
        self.epochs = epochs
        self.batch_size = batch_size
        self.verbose = verbose
        self.model_ = None

    def fit(self, X, y):
        self.model_ = self.build_fn()
        self.model_.fit(X, y, epochs=self.epochs, batch_size=self.batch_size, verbose=self.verbose)
        return self

    def predict(self, X):
        return np.argmax(self.model_.predict(X), axis=-1)

    def score(self, X, y):
        _, accuracy = self.model_.evaluate(X, y, verbose=self.verbose)
        return accuracy

Step 4: Create a Pipeline with Scikit-learn and TensorFlow

Now, we can integrate the custom Keras classifier into a Scikit-learn pipeline. The pipeline will include data standardization using Scikit-learn's StandardScaler and model training using the TensorFlow model.

from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline

# Create a Scikit-learn pipeline
pipeline = Pipeline([
    ('scaler', StandardScaler()),  # Standardize the data
    ('model', CustomKerasClassifier(build_fn=create_model, epochs=50, batch_size=10, verbose=0))  # TensorFlow model
])

Step 5: Train and Evaluate the Model

With the pipeline set up, we can now train the model on the training data and evaluate its performance on the test data.

# Train the Model
pipeline.fit(X_train, y_train)

# Evaluate the Model
accuracy = pipeline.score(X_test, y_test)
print(f"Test accuracy: {accuracy:.4f}")

Complete Code

Python 
import numpy as np
import pandas as pd
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.utils import to_categorical
from tensorflow.keras.optimizers import Adam
from sklearn.base import BaseEstimator, ClassifierMixin

# Step 1: Load and Prepare the Data
# Load the Iris dataset
iris = load_iris()
X = iris.data
y = iris.target

# One-hot encode the labels
y = to_categorical(y)

# Split the dataset into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Step 2: Define a TensorFlow Model
def create_model():
    model = Sequential()
    model.add(Dense(64, input_dim=4, activation='relu'))  # Input layer
    model.add(Dense(64, activation='relu'))               # Hidden layer
    model.add(Dense(3, activation='softmax'))             # Output layer (3 classes)
    model.compile(optimizer=Adam(), loss='categorical_crossentropy', metrics=['accuracy'])
    return model

# Custom KerasClassifier
class CustomKerasClassifier(BaseEstimator, ClassifierMixin):
    def __init__(self, build_fn=None, epochs=1, batch_size=32, verbose=0):
        self.build_fn = build_fn
        self.epochs = epochs
        self.batch_size = batch_size
        self.verbose = verbose
        self.model_ = None

    def fit(self, X, y):
        self.model_ = self.build_fn()
        self.model_.fit(X, y, epochs=self.epochs, batch_size=self.batch_size, verbose=self.verbose)
        return self

    def predict(self, X):
        return np.argmax(self.model_.predict(X), axis=-1)

    def score(self, X, y):
        _, accuracy = self.model_.evaluate(X, y, verbose=self.verbose)
        return accuracy

# Step 3: Create a Pipeline with Scikit-learn and TensorFlow
# Create a Scikit-learn pipeline
pipeline = Pipeline([
    ('scaler', StandardScaler()),  # Standardize the data
    ('model', CustomKerasClassifier(build_fn=create_model, epochs=50, batch_size=10, verbose=0))  # TensorFlow model
])

# Step 4: Train the Model
pipeline.fit(X_train, y_train)

# Step 5: Evaluate the Model
accuracy = pipeline.score(X_test, y_test)
print(f"Test accuracy: {accuracy:.4f}")

Output: 

Test accuracy: 1.0000

Conclusion

In this article, we've demonstrated how to combine the strengths of Scikit-learn and TensorFlow in a single machine-learning pipeline. By creating a custom Keras classifier, we can seamlessly integrate TensorFlow models into Scikit-learn’s powerful workflow. This approach allows us to leverage Scikit-learn’s extensive suite of tools for data preprocessing and model evaluation while utilizing TensorFlow for building sophisticated deep-learning models.

The test accuracy displayed at the end of the pipeline gives us an idea of how well our integrated model performs on unseen data, showcasing the effectiveness of using these two libraries together.

This seamless integration opens up numerous possibilities for creating more complex and efficient machine-learning workflows, combining the best of both worlds.

Comment

Explore