Still, we ended up with different results. We used the same bilinear method with Tensorflow. OpenCV resize method, by default, uses bilinear transformation. Even when we opened the images consistently across both frameworks, the resize method gave different results. We again see a big difference in the resize method of the two frameworks. image_tf_resized = image_tf_resized.astype(np.uint8) print("image dtype after conversion uint8: ", image_tf_resized.dtype) with tf.Session() as sess: image_tf_resized = image_tf_resized.eval() print("image shape after resize:", image_tf_resized.shape) print("image dtype: ", image_tf_resized.dtype) #Since it is in float32 format, we need to convert it back to uint8. image_tf_4D= tf.expand_dims(image_tf,0) # doing bilinear resize image_tf_resized_4D = tf.image.resize_bilinear( image_tf_4D, (300,300) ) #squeezing back the image to 3D image_tf_resized = tf.squeeze(image_tf_resized_4D) #Above is still a tensor. Hence we will have to expand the image and then squeeze back to three dimensions before we can use it as an image. This is shown below- # resizing the image with tensorflow 1.x print("image shape before resize:", image_tf.shape) print("image dtype: ", image_tf.dtype) #This function takes in a 4D input. This makes the types inconsistent with OpenCV. Once we have the NumPy image, we convert it to uint8 type as by default, session converts the NumPy to float32 type. Since our image is still a tensor, we will create and run a Tensorflow session to get back our resized image in NumPy format. Then resize the image and squeeze the dimensions back to three. Hence, we will have to first expand the image from three to four. It’s resizing method expects a 4D tensor and returns a 4D tensor output. image_cv = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) print(image_cv.dtype) print(np.max(image_cv)) plt_display(image_cv, 'cv2-RGB') The code below shows how to open an image with OpenCV correctly. We do this by using the OpenCV flag COLOR_BGR2RGB. So how to fix this? After opening the image, we convert the image from BGR to RGB. That’s why we get the above wrong colors in the image. pixels get confused and interpret Blue as Red and vice-versa). This is fine but when we display the image, the pixels misinterpret the channels (i.e. When we open an image using OpenCV, by default OpenCV opens the image in Blue, Green and Red channel (BGR). Now that we have an understanding of how color channels work in an image, let’s look into why we saw a different color image above. The order of these channels changes the color of the pixels, as pixels will always interpret first channel as Red, second channel as Green and third channel as Blue. Generally, images have three colors channels Red, Green, and Blue (RGB) which produces the colors in the pixel. Before diving into this, first let’s learn how colors in images are represented. Something’s strange about this figure above. image_path = 'dog2.JPG' def plt_display(image, title): fig = plt.figure() a = fig.add_subplot(1, 1, 1) imgplot = plt.imshow(image) a.set_title(title) image = cv2.imread(image_path) plt_display(image, 'cv2-BGR') %tensorflow_version 1.x import cv2 import matplotlib.pyplot as plt import numpy as np from skimage import io import tensorflow as tf We’ll import a few essential libraries in python. We’ll start with opening a simple JPEG image of a dog with two frameworks Tensorflow 1.x and OpenCV. Please note that I am an employee of Intel and all information and opinions shown in the blog are my own and don’t represent those of my employer. Also, thanks to Tejas Pandey for the help in achieving consistency between OpenCV and Tensorflow. It will also show a way to make them work consistently. This blog will discuss one such use case in detail where OpenCV and Tensorflow show the differences in reading and resizing a JPEG image. It can take up multiple days to figure out what went wrong and could delay the project extensively. The blog “How Tensorflow’s tf.image.resize stole 60 days of my life” ( ) is a perfect example of this type of situation. Therefore, CV solutions developed in one framework may not work as expected in the other framework. In today’s rapid development of frameworks, every framework has its own way of handling images, each with its own specifications. For this, one needs to use a framework to open those images to do some processing on them. Modern Computer Vision (CV) is currently a hot field of research which involves largely working with images. A dive into the differences in JPEG image read and resizing with OpenCV, Tensorflow and Pillow and also on how to make them consistent.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |