Introduction to ‘TensorLayer’: A Python-based Versatile Deep Learning Library Designed for Machine Learning Researchers

On Feb 21, 2022

Blog Banner — Source: https://github.com/tensorlayer/TensorLayer

It takes a long time and a lot of effort to build a working deep learning system. Building advanced neural networks, coordinating several network models, data processing, designing a concise workflow, and handling a large volume of training-related data are all time-consuming processes. Keras and TFLearn are two tools that can help with this development process since they enable flexibility and abstraction for numerous connected modalities. Another tool that is fast gaining popularity is TensorLayer, a Python-based machine learning tool.

The progress of deep learning is being hampered by the increase in interaction. Developers must devote a significant amount of time to integrating components for experimenting with neural networks, handling intermediate training stages, organizing training-related data, and allowing hyperparameter change in response to diverse events.

An integrative development strategy is utilized to reduce the number of cycles required, in which complex operations on neural networks, states, data, and hyper-parameters are abstracted and presented in complementary modules. As a result, developers can effectively explore concepts using high-level module operations and only make changes to modules when they are required.

Module lock-in is not the goal of this technique. Instead, modules are modeled as basic single-function blocks with a common interaction interface, allowing for easy user-defined module plug-ins.

To achieve this goal, TensorLayer is a collective endeavor. It’s a Python toolkit with simple modules to assist academics and engineers in the development of complex deep learning systems. The TensorLayer implementation is optimized for speed and scalability. The distributed training and inference engine are TensorFlow.

The overhead of delegation into TensorFlow is negligible. MongoDB is also used as a storage backend by TensorLayer. This backend is enhanced with an efficient stream controller for managing unbounded training data. To facilitate automation, this controller can batch results from a dataset query and generate batch training jobs as needed.

To efficiently handle large data items like films, TensorLayer uses GridFS as a blob backend and MongoDB as a sample indexer. Finally, in order to provide an asynchronous training workflow, TensorLayer uses an agent pub-sub architecture. Agents can run on a variety of devices and subscribe to different task queues. These queues are kept in a secure location so that failed tasks can be replayed automatically.

Unlike other TensorFlow-based tools like Keras and TFLearn, TensorLayer enables for straightforward low-level control over layer and neural network execution. It also comes with extra dataset and workflow modules that take care of time-consuming data pre-processing, post-processing, module serving, and data management tasks. Its non-intrusive unified module interaction interface accommodates Keras and TFLearn layers and networks.

A layer module in TensorLayer contains reference implementations of a number of layers, including CNN, RNN, dropout, batch normalization, and many others. Layers, like the widely used Lasagne, are created declaratively to form a neural network. Each layer is given its own key to facilitating developers in parameter sharing. The networks are managed with TensorFlow. TensorLayer is a distributed and hybrid platform based on TensorFlow.

Models are logical depictions of self-contained functional units that may be trained, evaluated, and deployed in the field. The network structure of each model is unique. Throughout the training, different versions or states of the model may exist (i.e., weights). State persistence, caching, and reloading are all options.

You maintain track of your training samples and predictions in the dataset module. They’re saved as documents in MongoDB. Each document contains a unique key, a sample, a label, and user-defined tags. To define datasets, declarative queries with tag field requirements are utilized. Queries produce views of the underlying data without the need for additional storage.

The workflow module simplifies the creation of asynchronous feedback loops in model group operations and learning systems. It’s also useful for complex cognitive systems that have components that need to be trained. For example, before training an RNN decoder to generate descriptions based on the observed context, the inventor of an image captioning system trained a CNN to grasp the context of images. TensorLayer can accommodate a two-stage asynchronous training plan in this case.

Following code is the implementation of image classification using transfer learning through TensorLayer. The model used here is VGG16.

! pip install tensorlayer

# interdependencies required for image pre-processing
! pip install scipy==1.2.1

import numpy as np
import tensorflow as tf
 
import tensorlayer as tl
from tensorlayer.models.imagenet_classes import class_names

# get the whole model
vgg = tl.models.vgg16(pretrained=True)

#image loading, pre-processing
img = tl.vis.read_image('/content/steam-train-rides-1570200690.jpg')
img = tl.prepro.imresize(img, (224, 224)).astype(np.float32) / 255

#process the image to the model and get probas
output = vgg(img, is_train=False)
probs = tf.nn.softmax(output)[0].numpy()

# print result
preds = (np.argsort(probs)[::-1])[0:5]
for q in preds:
    print(class_names[q], probs[q])