JPEG encoder color operator output image quality

  • Hello,


    I use the JPEG encoder color operator to compress image.

    I found that the image quality after decompression is a bit abnormal in the horizontal direction.


    A is the de-mosaicing color image of "BayerImage" .

    B is compressed from A by JPEG.

    C is composed of JPEG Gray R,JPEG Gray G, and JPEG Gray B.


    You can see the horizontal edge of B that the resolution is reduced. But the vertical edge of B that the resolution hasn't changed much.

    When compressing the three grayscale images into JPEG respectively, then compose the three JPEG grayscale images into a color image. The horizontal edge of C that the resolution hasn't changed much.

    Please refer "Compare.JPG".


    Why is there a difference between the resolution of the horizontal edge and the resolution of the vertical edge after JPEG encoder color operator compression?


    Jesse Lin

  • Hello Jesse,

    Sorry for the late reply, let me try to answer your questions.

    I simulated your design with my own simulated images that have the resolution of 4096x3072 after debayering, so the following numbers will differ on your side depending on the used input image.

    If you take a look at the SimProbes JpegGrayR, JpegGrayG and JpegGrayB you will notice that they will have a size for example of 1186120 - and this times three (for each channel) is: 3 558 360

    If you now take a look at the probe "JpegColor" you will notice the size of 1 269 096 - which means that you will have a much higher
    compression using the JPEG color encoder.

    Simply put, the algorithm inside the Jpeg color encoder differs vastly from the implementation of using a splitcomponent and three different JPEG_Encoder operators. Thus, resulting in different results.

    The jpeg encoder works in blocks, bot the Color encoder and the grayscale encoder, however the Color encoder works with the YCbCr format and some internal downscaling. If you need more information on this please let me know and I will ask the appropiate engineers at Basler.

    Also, your results may differ with images that have more color information, you could try to create a chessboard with different color patterns.

    Lastly, the color jpeg encoder is better optimized for ressources, if you use three jpeg encoders you will need roughly

    LUT: 25.759
    FFs: 27.396

    BRAM: 135
    Embedded ALU: 141
    RAMLUT: 3075

    But the JPEG color encoder only uses:


    LUT: 13220

    FFS: 19681
    BRAM: 93
    Embedded ALU: 55

    RAMLUT: 1630


    To summarize it: Both implementations have a very different approach on encoding the color information and are therefore hard to compare.

    Best Regards

    Kevin



  • Hi Kevin


    Would you tell me how to calculate the size for example of 1186120 with JpegGrayR ?


    I tried to sample the YCBCR image to 420 and then convert back to RGB image. You can refer mE5MtVCL_YCBCR.va.


    The converted image(YCbCrToRGB.png) looks normal with horizontal edges.

    But after JPEG compression, every two lines in the horizontal direction, you will find that the values of the upper and lower lines are quite close.

    Could it be generated during JPEG compression?


    Thanks.


    Jesse Lin

  • Hi,

    The calculations on how large the output image will be is quite difficult. The Jpeg Algorithm works in a way that different cosine frequencies are stored. This means that the size of your output image is depending on your image you want to decode. Images that have a repeating pattern (e.g. a chess board) are easier to decode in comparison to images that have a high variance and a lot of edges. The worst possible image to decode in jpeg is probably an image just consisting of color noise.

    To understand JPEG better I highly recommend computerphiles videos on that topic:

    Part 1:


    Part 2:



    Most importantly, JPEG compression is NOT losless - which means you will never get the exact same image back after encoding / decoding - even with JPEGQuality set to 100. You will always include some so called "Jpeg artifacts" . These artifacts are most likely to occur on edges. You can read bout it here; https://en.wikipedia.org/wiki/Compression_artifact . This also explains the artifacts you experience in your image.

    Lastly, you can try it with different simulated images - what I did was simply use a Coordinate_X operator followed by a castbitwidth and set it to 8 to simulate a grayscale image. I attached my experimental synthetic image, so you may get the same size results as I did.


    Best Regards

    Kevin

  • Hi Kevin


    I use C# to compress image with JPEG encoder.

    Because forum will auto scale image , so I upload the image on google drive.

    This is quality level 100

    https://drive.google.com/file/…o5X5FsQT/view?usp=sharing

    This is quality level 50

    https://drive.google.com/file/…-r-7ZnRi/view?usp=sharing


    You can see 8x8 matrix of distortion at quality level 50 image. Distortion in the horizontal direction does not exist alone.

    This is my perception of JPEG compression distortion.


    And I watch the video. DCT uses an 8x8 matrix to reserve the required frequencies.

    Can you help me to confirm whether JPEG encoder color operator has filtered out the horizontal frequency?


    Thanks.


    Jesse Lin

  • Hi Jesse,

    I will ask the appropiate engineers at Basler and will come back to you as soon as I have the information available.
    Please be patient, due to holidays this may take some time.

    What I can tell you at the moment is, that depending on the input parallelism different block sizes are used. Which means that not all the time 8x8 blocks are utilized.

    Best Regards,

    Kevin

  • Hi Jesse,

    We use 4:2:2 subsampling as in the picture shown below:

    1920px-Common_chroma_subsampling_ratios.svg.png



    This means that we filter out some information in the horizontal direction, but not in the vertical direction, which may explains your observed behaviour.

    If you need an other form of subsampling you may contact your appropiate sales manager for a custom request.


    Best Regards,

    Kevin

  • Hi Kevin


    If the operator use 4:2:2 subsampling. It will like this table.

    Y(0,0)Cb(0,0)Cr(0,0) Y(1,0)Cb(0,0)Cr(0,0) Y(2,0)Cb(2,0)Cr(2,0) Y(3,0)Cb(2,0)Cr(2,0)
    Y(0,1)Cb(0,1)Cr(0,1) Y(1,1)Cb(0,1)Cr(0,1) Y(2,1)Cb(2,1)Cr(2,1) Y(3,1)Cb(2,1)Cr(2,1)

    (column,row)


    So the column 0 and column 1 color will be close , and column 2 and column 3 color will be close.

    But my issue is row 0 and row 1 color will be close, Please refer attachment "JPEG.png".


    And if you sure the operator use 4:2:2 subsampling.

    The JPEG_Encoder_Color_850MPs_VCL operator help may need to be corrected.


    Jesse Lin

  • Hi,

    Sorry for the misunderstanding, I had a double check and it is indeed 4-2-0.

    As mentioned, JPEG compression does not preserve the data and will introduce jpeg artifacts.


    I will come back as soon as I have new information on this topic.

    Best Regards,

    Kevin

  • Hi Jesse,

    We are currently investigating this further.

    Just to double check:

    1. When you used C# for encoding, did you use subsampling as well?

    2. What did you use to decode the JPEG image? Did you write your own script?

    3. Did you decode the JPEG in the simulation or in hardware?


    Best Regards

    Kevin

  • Hi Kevin


    1. When you used C# for encoding, did you use subsampling as well?

    I use the wrapped function for testing. So I can't be sure if the function is using sampling.

    But I have tested whether sampling can cause such distortion.You can refer attachment "mE5MtVCL_YCBCR".


    2. What did you use to decode the JPEG image? Did you write your own script?

    3. Did you decode the JPEG in the simulation or in hardware?

    Because the color JPEG operator will output binary data that have JPEG header.And then use the binary data to write it as a JPEG file.

    And open(decode) it by Windows Photo Viewer or Photoshop.



    Best Regards


    Jesse Lin


  • Hi Jesse,

    Thank you for your input.

    The subsampling you implemented does the right job, however it is pre-jpeg compression. So the effect may take place during the jpeg compression.

    Lastly, did you use the output during simulation (from the green box) or did you build your application, put into the framegrabber and saved it from MicroDisplay or a C++ application?


    Best Regards,

    Kevin

  • Hi Jesse,

    I did some testing with OpenCV JPEG compression.

    Firstly, I will attach a design that has a chess board generator with uneven tiles, thus creating a test environment that has no artifacts in it. I adapted your original design, you can use it for testing, but I doubt that it will work in hardware.

    Furthermore, here is a short Python-Snippet for doing the testing I did:






    With this snippet I created the following plot:





    pasted-from-clipboard.png



    This is a visualization of the different artifacts introduced by Visual Applets and OpenCV, respectively.
    They both have a different implementation, therefore the shown artifacts are different.

    However, what is more important: In both cases the JPEG artifacts are diferent in the horizontal in the vertical direction with OpenCV and with Visual Applets.


    Therefore, your observation, that the artifacts are more visible on the horizontal edges than on the vertical edges does not show that there is a bug in the implementation but can be explained by the loss of information due to jpeg compression.




    For further questions on this matter, please refer to our support.



    Best Regards,

    Kevin

  • Hi Kevin


    You're right that the location you find is distorted.

    I tested your code and found that the maximum difference value of VA comes to 108(jpeg_artifacts_va).

    But the maximum difference value of opencv comes to 18(jpeg_artifacts_opencv).

    The difference in VA is five times that of OPENCV. And the difference between the maximum values is 90.


    And please refer the "Compare.png" of attachment. You can easily find their difference.


    Best Regards,

    Jesse Lin

    Files

    • code.zip

      (34.49 kB, downloaded 1 times, last: )
    • Compare.png

      (547.64 kB, downloaded 4 times, last: )