JPEG encoder color operator output image quality

Hello Jesse,

Sorry for the late reply, let me try to answer your questions.

I simulated your design with my own simulated images that have the resolution of 4096x3072 after debayering, so the following numbers will differ on your side depending on the used input image.

If you take a look at the SimProbes JpegGrayR, JpegGrayG and JpegGrayB you will notice that they will have a size for example of 1186120 - and this times three (for each channel) is: 3 558 360

If you now take a look at the probe "JpegColor" you will notice the size of 1 269 096 - which means that you will have a much higher
compression using the JPEG color encoder.

Simply put, the algorithm inside the Jpeg color encoder differs vastly from the implementation of using a splitcomponent and three different JPEG_Encoder operators. Thus, resulting in different results.

The jpeg encoder works in blocks, bot the Color encoder and the grayscale encoder, however the Color encoder works with the YCbCr format and some internal downscaling. If you need more information on this please let me know and I will ask the appropiate engineers at Basler.

Also, your results may differ with images that have more color information, you could try to create a chessboard with different color patterns.

Lastly, the color jpeg encoder is better optimized for ressources, if you use three jpeg encoders you will need roughly

LUT: 25.759
FFs: 27.396

BRAM: 135
Embedded ALU: 141
RAMLUT: 3075

But the JPEG color encoder only uses:


LUT: 13220

FFS: 19681
BRAM: 93
Embedded ALU: 55

RAMLUT: 1630


To summarize it: Both implementations have a very different approach on encoding the color information and are therefore hard to compare.

Best Regards

Kevin

Hi,

The calculations on how large the output image will be is quite difficult. The Jpeg Algorithm works in a way that different cosine frequencies are stored. This means that the size of your output image is depending on your image you want to decode. Images that have a repeating pattern (e.g. a chess board) are easier to decode in comparison to images that have a high variance and a lot of edges. The worst possible image to decode in jpeg is probably an image just consisting of color noise.

To understand JPEG better I highly recommend computerphiles videos on that topic:

Part 1:

Hi Jesse,

I will ask the appropiate engineers at Basler and will come back to you as soon as I have the information available.
Please be patient, due to holidays this may take some time.

What I can tell you at the moment is, that depending on the input parallelism different block sizes are used. Which means that not all the time 8x8 blocks are utilized.

Best Regards,

Kevin

Hi Jesse,

We use 4:2:2 subsampling as in the picture shown below:

This means that we filter out some information in the horizontal direction, but not in the vertical direction, which may explains your observed behaviour.

If you need an other form of subsampling you may contact your appropiate sales manager for a custom request.


Best Regards,

Kevin

Hi,

Sorry for the misunderstanding, I had a double check and it is indeed 4-2-0.

As mentioned, JPEG compression does not preserve the data and will introduce jpeg artifacts.


I will come back as soon as I have new information on this topic.

Best Regards,

Kevin

Hi Jesse,

We are currently investigating this further.

Just to double check:

1. When you used C# for encoding, did you use subsampling as well?

2. What did you use to decode the JPEG image? Did you write your own script?

3. Did you decode the JPEG in the simulation or in hardware?


Best Regards

Kevin

Hi Jesse,

Thank you for your input.

The subsampling you implemented does the right job, however it is pre-jpeg compression. So the effect may take place during the jpeg compression.

Lastly, did you use the output during simulation (from the green box) or did you build your application, put into the framegrabber and saved it from MicroDisplay or a C++ application?


Best Regards,

Kevin

Hi Jesse,

I did some testing with OpenCV JPEG compression.

Firstly, I will attach a design that has a chess board generator with uneven tiles, thus creating a test environment that has no artifacts in it. I adapted your original design, you can use it for testing, but I doubt that it will work in hardware.

Furthermore, here is a short Python-Snippet for doing the testing I did:

from matplotlib import pyplot as plt
import numpy as np
import cv2
import imageio




# load the jpeg file from VA here
test = imageio.imread('uneven.tiff')[0,:].tobytes()
#now we write it as jpeg
f = open('newWay.jpg', 'wb')
f.write(test)
f.close()


jpeg_from_va = cv2.imread("newWay.jpg")


#now load the original file
original = cv2.imread("unEvenChessboard.tiff")


# check the jpeg artifacts
jpeg_artifacts_va = np.float32(original) - np.float32(jpeg_from_va)


plt.figure(0,figsize=[20,10])
plt.imshow(jpeg_artifacts_va)


cv2.imwrite("uneven.jpg", original, [int(cv2.IMWRITE_JPEG_QUALITY), 100])


jpeg_by_opencv = cv2.imread("uneven.jpg")


jpeg_artifacts_opencv = np.float32(original) - np.float32(jpeg_by_opencv)
# now do the jpeg compression with opencv




plt.figure(1,figsize=[20,10])
plt.imshow(jpeg_artifacts_opencv)




plt.figure(3,figsize=[20,20])
plt.subplot(2,1,1)
plt.imshow(jpeg_artifacts_va[:200,:200])
plt.title("Visual Applets Artifacts")
plt.subplot(2,1,2)
plt.imshow(jpeg_artifacts_opencv[:200,:200])
plt.title("OpenCV Artifacts")

With this snippet I created the following plot:

pasted-from-clipboard.png

This is a visualization of the different artifacts introduced by Visual Applets and OpenCV, respectively.
They both have a different implementation, therefore the shown artifacts are different.

However, what is more important: In both cases the JPEG artifacts are diferent in the horizontal in the vertical direction with OpenCV and with Visual Applets.


Therefore, your observation, that the artifacts are more visible on the horizontal edges than on the vertical edges does not show that there is a bug in the implementation but can be explained by the loss of information due to jpeg compression.




For further questions on this matter, please refer to our support.



Best Regards,

Kevin

Thank you really much for your input. We have identified an issue and will provide a fix as soon as it is ready.

In the meantime can you please contact me under: kevin.hoefle@baslerweb.com for further information.

Best Regards,

Kevin