A couple of test cases
 
About rounding
We recommend that you use rounding without bias (round half away from zero).See for instance about rounding in Wikipedia.
A piece of code, that does quantization and rounding:
temp = temp*rval;
rnd = (temp & 0x10000) != 0 ;
bits = (temp & 0xffff) != 0;
pos = (temp & 0x80000000) == 0;
temp = temp >> 17;
temp += rnd && (pos || bits);
 
Case 1
1) Before transform: a=
1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 |
9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 |
17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 |
25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 |
33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 |
41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 |
49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 |
57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 |
2) After sqrt(8)*DCT1[a-128] (and truncating) on the rows
-988 | -19 | 0 | -2 | 0 | -1 | 0 | -1 |
-924 | -19 | 0 | -2 | 0 | -1 | 0 | -1 |
-860 | -19 | 0 | -2 | 0 | -1 | 0 | -1 |
-796 | -19 | 0 | -2 | 0 | -1 | 0 | -1 |
-732 | -19 | 0 | -2 | 0 | -1 | 0 | -1 |
-668 | -19 | 0 | -2 | 0 | -1 | 0 | -1 |
-604 | -19 | 0 | -2 | 0 | -1 | 0 | -1 |
-540 | -19 | 0 | -2 | 0 | -1 | 0 | -1 |
3) After another sqrt(8)*DCT1 (and truncating),
now on the columns,
we have 8*DCT2[a-128] =
-6112 | -152 | 0 | -16 | 0 | -8 | 0 | -8 |
-1167 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
-122 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
-37 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
-10 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
4) The quantization matrix is given by Q =
16 | 11 | 10 | 16 | 24 | 40 | 51 | 61 |
12 | 12 | 14 | 19 | 26 | 58 | 60 | 55 |
14 | 13 | 16 | 24 | 40 | 57 | 69 | 56 |
14 | 17 | 22 | 29 | 51 | 87 | 80 | 62 |
18 | 22 | 37 | 56 | 68 | 109 | 103 | 77 |
24 | 35 | 55 | 64 | 81 | 104 | 113 | 92 |
49 | 64 | 78 | 87 | 103 | 121 | 120 | 101 |
72 | 92 | 95 | 98 | 112 | 100 | 103 | 99 |
5) After quantization the result should be
Y = round(8*B/(8*Q*1/2))
(1/2 is a quality factor used by jpegfiles)
-96 | -3 | 0 | 0 | 0 | 0 | 0 | 0 |
-24 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
-2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
 
Case 2:
1) chess board in the upper left corner of wunderbart)
0 | 255 | 0 | 255 | 0 | 255 | 0 | 255 |
255 | 0 | 255 | 0 | 255 | 0 | 255 | 0 |
0 | 255 | 0 | 255 | 0 | 255 | 0 | 255 |
255 | 0 | 255 | 0 | 255 | 0 | 255 | 0 |
0 | 255 | 0 | 255 | 0 | 255 | 0 | 255 |
255 | 0 | 255 | 0 | 255 | 0 | 255 | 0 |
0 | 255 | 0 | 255 | 0 | 255 | 0 | 255 |
255 | 0 | 255 | 0 | 255 | 0 | 255 | 0 |
2) After transform&quantization
-1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
0 | -6 | 0 | -4 | 0 | -2 | 0 | -6 |
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
0 | -5 | 0 | -3 | 0 | -2 | 0 | -6 |
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
0 | -3 | 0 | -2 | 0 | -2 | 0 | -6 |
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
0 | -4 | 0 | -4 | 0 | -6 | 0 | -17 |
 
Case 3:
1) (next chess board in the upper left corner of wunderbart)
0 | 0 | 255 | 255 | 0 | 0 | 255 | 255 |
0 | 0 | 255 | 255 | 0 | 0 | 255 | 255 |
255 | 255 | 0 | 0 | 255 | 255 | 0 | 0 |
255 | 255 | 0 | 0 | 255 | 255 | 0 | 0 |
0 | 0 | 255 | 255 | 0 | 0 | 255 | 255 |
0 | 0 | 255 | 255 | 0 | 0 | 255 | 255 |
255 | 255 | 0 | 0 | 255 | 255 | 0 | 0 |
255 | 255 | 0 | 0 | 255 | 255 | 0 | 0 |
2) After transform&quantization
-1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
0 | -24 | 0 | -31 | 0 | 7 | 0 | 1 |
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
0 | -35 | 0 | -42 | 0 | 9 | 0 | 2 |
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
0 | 11 | 0 | 13 | 0 | -5 | 0 | -1 |
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
0 | 1 | 0 | 1 | 0 | -1 | 0 | 0 |
 
Case 4:
1) (and next chess board in the upper left corner of wunderbart)
0 | 0 | 0 | 0 | 255 | 255 | 255 | 255 |
0 | 0 | 0 | 0 | 255 | 255 | 255 | 255 |
0 | 0 | 0 | 0 | 255 | 255 | 255 | 255 |
0 | 0 | 0 | 0 | 255 | 255 | 255 | 255 |
255 | 255 | 255 | 255 | 0 | 0 | 0 | 0 |
255 | 255 | 255 | 255 | 0 | 0 | 0 | 0 |
255 | 255 | 255 | 255 | 0 | 0 | 0 | 0 |
255 | 255 | 255 | 255 | 0 | 0 | 0 | 0 |
2) After transform&quantization
-1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
0 | -140 | 0 | 31 | 0 | -7 | 0 | 6 |
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
0 | 35 | 0 | -7 | 0 | 2 | 0 | -2 |
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
0 | -11 | 0 | 2 | 0 | -1 | 0 | 1 |
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
0 | 4 | 0 | -1 | 0 | 1 | 0 | -1 |
Informationsansvarig: Kent Palmkvist
Senast uppdaterad: 2015-11-10