Answers.Body: <blockquote>
    <p>However, is it possible to explain why JPEG works this way, without venturing into domain change, DCT and so on?</p>
  </blockquote>
  
  <p><em>Disclaimer 1</em>: I'm not sure if this is more of a comment or an answer and</p>
  
  <p><em>Disclaimer 2</em>: this is not <em>really</em> my field, but </p>
  
  <p>I'd be inclined to answer: <strong>no</strong>.</p>
  
  <p>Firstly, I don't think it's <em>necessary</em>. I think the intuition behind the concept of transform (at least as in Fourier or DCT) is within the grasp of most high school kids that know <em>some</em> trigonometry: you can probably get away with drawing a simple function, decomposing it into sine waves and writing down their phase and amplitude while waving your hands a lot.</p>
  
  <p>You'd then proceed to reconstruct the original function by means of summation and persuade the students that the representation in terms of 4/5 phase/amplitude coefficients <em>is "more compact"</em> (finite, even) <em>than its representation in terms of infinite pairs in $R \times R$.</em></p>
  
  <p>I'd argue it's also <em>not possible</em> because the DCT is also very much one of - if not <em>the</em> - working principles behind JPEG, differentiating it from classical lossy compression techniques.</p>
  
  <p>Operation in the frequency domain is ultimately <em>the</em> reason why *PEG performs subjectively better than, say, resampling and requantizing at 240x180@16 colors, which is essentially a much naiver lossy compression method that operates purely in the space/time domain.</p>
  
  <p>Note, finally, how the excellent <a href="http://dspguide.com/ch27/6.htm" rel="nofollow noreferrer">Smith handbook</a> - along with probably several other resources - explicitly calls JPEG <strong>“transform compression”</strong>.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-01-28T15:39:29.803Z
Answers.LastActivityDate: 2018-01-28T19:56:48.463Z
Answers.LastEditDate: 2018-01-28T19:56:48.463Z
Answers.UserDisplayName: Tobia Tesan
Answers.Body: <h1>On teaching JPEG</h1>
  <p>I would start with the lossless algorithms, run-length is a good place to start. Then you can go on to discuss human perception, I like to use the example of hiding in plain site.</p>
  <h2>Hiding in plain sight</h2>
  <p>Ask which is harder to spot someone standing by a wall, or someone standing in the middle of a field. Use this as the basis for discussing lossy compression.</p>
  <h2>Optical illusion</h2>
  <p>Explore images that look the same, but are different (when examined). One that I remember is an image white background, with a series of vertical thick black bars. Then to the right of each bar is a thin line with a linear gradient from dark to light. All black bars are identical, all thin lines are identical, however the lines get closer to the bars as you move to the right. (so multiple copies of the bar and line, but each copy to the right has the bar and line closer). This is perceived as the thin lines getting shorter to the right. An example of the hiding in plain sight illusion discussed above.</p>
  <h2>When not to use lossy compression</h2>
  <p>This relates to e-safety, digital literacy, ethics, etc.</p>
  <p>As well as the ones you said: Cartoon (traditional, such as Simpsons), line drawings. There are certain uses of photograph that should not be lossy encoded. These include medical X-ray (especially of fractures), and of nuclear power rods or aircraft, to check for build quality (fractures). I mention these 3 as I know this has been done, at great cost (£Millions), and safety risk.</p>
  <h1>What is JPEG</h1>
  <p>Let me point out that JPEG is not a file format but a set of algorithms, some lossy some lossless. However just after release someone made a library that used some of the algorithms (Discrete cosine transform, hoffman encoding, etc), to make a lossy compression algorithm, this became a de-facto standard).</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-01-28T19:59:03.067Z
Answers.Text: `I would start with the lossless algorithms` seconded. JPEG is essentially `quantization + sampling + classical lossless techniques + transform`
Answers.UserDisplayName: Tobia Tesan
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-01-28T19:59:58.370Z
Answers.Text: `I mention these 3 as I know this has been done` do you happen to have a reference for this? I bet it'd be an entertaining read.
Answers.UserDisplayName: Tobia Tesan
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-01-29T10:02:22.860Z
Answers.Text: @TobiaTesan Sorry it was a long time ago, and I don't remember.
Answers.UserDisplayName: ctrl-alt-delor
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-02-01T02:15:50.377Z
Answers.Text: Thanks, I can't deal with maths behind JPEG, but run-length encoding and visual perception are surely within the reach of my students.
Answers.UserDisplayName: totera
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-01-28T10:26:22.343Z
Answers.LastActivityDate: 2018-01-28T16:59:13.870Z
Answers.LastEditDate: 2020-06-18T08:36:57.023Z
Answers.UserDisplayName: ctrl-alt-delor
Answers.Body: <h2>The basics</h2>
  
  <p>Let me start with a quick overview of how JPEG (as usually applied) compresses color images.</p>
  
  <ol>
  <li><p><strong>Colorspace and downscaling.</strong> Raw images are usually encoded in the RGB color space. The human eye is more sensitive to the luminosity of images than to their chromatic aspects &mdash; as an extreme  example, we can understand black and white photos quite well. Therefore JPEG images store more information about luminosity and less about colors. This is implemented in two steps. First, the RGB color space is transformed into a different color space (in a linear fashion), YCbCr (similar to the TV color space YPbPr) in which Y is luminosity and Cb/Cr encode how much the pixels are blue rather than purple (Cb) and red rather than green (Cr). The Cb and Cr planes are then <em>downscaled</em> or <em>subsampled</em>, that is, their resolution is reduced, usually by a factor of 2 in the horizontal direction (the eye is more sensitive to the vertical direction). From this point on, the three planes are compressed individually.</p></li>
  <li><p><strong>Division into blocks.</strong> Each of the color planes is now divided into 8&times;8 non-overlapping blocks (in contrast, in MP3 the sections are partially overlapping), which will be compressed more-or-less individually. This division is the reason for the so-called <em>blocking effects</em>, since the superimposed grid cuts through features of the image.</p></li>
  <li><p><strong>Spectral compression by quantization.</strong> The basic insight behind JPEG is that nature is noisy, and this noise is filtered by the human visual system. It follows that we can throw away the noise without substantially affecting the way the image looks. It remains to decompose the image into its "substantial" and "noisy" components, and this is accomplished by a 2D Fourier transform. Recall we are trying to compress 8&times;8 blocks. The most important feature of a block is the average value of its 64 pixels, known as the DC component (or, zeroth Fourier coefficient); the DC components of all blocks are just a result of scaling down the original image by 8 in both dimensions. The importance of the other components, known as AC components, depends on their frequency &mdash; the higher frequencies correspond to noise, whereas the lower frequencies are more informative. However there is no clear cut distinction between features and noise. For this reason, we need to retain all components, albeit with different accuracy. Accordingly, we quantize the different components to various degrees, depending on the frequency: components corresponding to higher frequencies are quanrtized more. The exact quantization matrix depends on the disparity between the horizontal and vertical dimensions, and has been determined experimentally. (It also depends on the quality parameter of the compressor.)</p></li>
  <li><p><strong>Entropy encoding.</strong> It remains to encode the quantized components into bits, which is done using Huffman coding, a standard encoding scheme. Since the DC components of all blocks are a downscaled version of the entire image, we expect them to vary continuously. Therefore instead of encoding their actual values, we encode them <em>differentially</em>, that is, we encode the difference from the preceding DC value. The AC components don't have this behavior, and so are encoded <em>absolutely</em>.</p></li>
  </ol>
  
  <h2>Artifacts</h2>
  
  <p>Why does compressing geometric drawings result in artifacts? Geometric drawings are very different from natural images. They don't vary continuously. Spectral compression results in "smearing" effects, since we are only retaining the lower frequencies. The best illustration of this is the <a href="https://en.wikipedia.org/wiki/Gibbs_phenomenon" rel="noreferrer">Gibbs phenomenon</a>, which is the continuous analog of the same issue. When trying to represent a step function using Fourier series, we need to use all frequencies. If we suppress the high frequencies, we get something really different. Lines in a geometric drawing are the discrete analog of step functions.</p>
  
  <p>Another problem is the blocking effects. Imagine a diagonal line. JPEG partitions the space around these lines into 8&times;8 blocks, which are compressed individually. No attempt is made for the resulting lines to match exactly, and indeed they probably don't. This is very apparent because lines are very clean and exact, something that cannot be said about real-life images.</p>
  
  <h2>Teaching ideas</h2>
  
  <p>Using software like Matlab, you can show the effect of quantization on a real-world image, and then compare it to the effect of quantization on clip art. You can vary the quantization steps to show the effect of downplaying the higher frequencies (and also what happens when trying to suppress the lower frequencies).</p>
  
  <p>You can demonstrate the effect of blocking by varying the angle of a line which is compressed block-by-block, or fixing a line and varying the block size. You can use a low quality setting to emphasize the effect, and include the block boundaries to highlight its origins.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-01-28T12:58:04.670Z
Answers.Text: This is a really thorough answer. Thank you for taking the time to write it!
Answers.UserDisplayName: thesecretmaster
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-01-28T15:43:38.043Z
Answers.Text: Also note how in the final step [the DCT matrix is traversed diagonally](http://dspguide.com/graphics/F_27_14.gif), in order to facilitate RLE.
Answers.UserDisplayName: Tobia Tesan
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-02-01T02:03:27.803Z
Answers.Text: Thanks, excellent explanation of JPEG, but way too complex for my class (first year).
Answers.UserDisplayName: totera
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-02-01T06:33:42.063Z
Answers.Text: This explanation is targeted at the teacher. If you don't understand how it works, you can't teach it even on a basic level.
Answers.UserDisplayName: Yuval Filmus
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-01-28T12:03:56.420Z
Answers.LastActivityDate: 2018-01-28T12:03:56.420Z
Answers.UserDisplayName: Yuval Filmus
Body: <p>I will teach a lesson on image file formats in a computer science high school class.</p>
  
  <p>Regarding JPEG, I will say that it achieves lossy compression and that it is well suited for photographs but it should not be used for geometric drawings where it can generate artifacts, showing some examples.</p>
  
  <p>However, is it possible to explain why JPEG works this way, without venturing into domain change (space to frequency conversion), <a href="https://en.wikipedia.org/wiki/Discrete_cosine_transform" rel="nofollow noreferrer">Discrete Cosine Transformation</a>, and so on?</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-01-28T01:33:16.830Z
Comments.Text: Welcome to CSEducators. Thanks for an interesting question. I hope you've had some time to look around here.
Comments.UserDisplayName: Buffy
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-01-28T10:27:21.440Z
Comments.Text: For domain change, are you referring to space to frequency?
Comments.UserDisplayName: ctrl-alt-delor
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-02-01T01:38:46.253Z
Comments.Text: @Buffy Yes... great project!
Comments.UserDisplayName: totera
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-02-01T01:39:13.090Z
Comments.Text: @ctrl-alt-delor Yes, of course.
Comments.UserDisplayName: totera
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-02-01T09:52:18.440Z
Comments.Text: @Totera I have put this information into the question.
Comments.UserDisplayName: ctrl-alt-delor
Tags: <lesson-ideas><secondary-education>
Title: Simple explanation of JPEG working principles
ThreadText: # Simple explanation of JPEG working principles
  
  (Asked by: username_0 on 2018-01-28)
  
  I will teach a lesson on image file formats in a computer science high school class.
  
  Regarding JPEG, I will say that it achieves lossy compression and that it is well suited for photographs but it should not be used for geometric drawings where it can generate artifacts, showing some examples.
  
  However, is it possible to explain why JPEG works this way, without venturing into domain change (space to frequency conversion), [Discrete Cosine Transformation](https://en.wikipedia.org/wiki/Discrete_cosine_transform), and so on?
  
  **Tags:** lesson-ideas, secondary-education
  
  > **Comments:**
  > **username_1**: Welcome to CSEducators. Thanks for an interesting question. I hope you've had some time to look around here.  
  > **username_2**: For domain change, are you referring to space to frequency?  
  > **username_0**: @username_1 Yes... great project!  
  > **username_0**: @username_2 Yes, of course.  
  > **username_2**: @username_0 I have put this information into the question.  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_2 on 2018-01-28. Score: 4 | Accepted answer
  
  # On teaching JPEG
  
  I would start with the lossless algorithms, run-length is a good place to start. Then you can go on to discuss human perception, I like to use the example of hiding in plain site.
  
  ## Hiding in plain sight
  
  Ask which is harder to spot someone standing by a wall, or someone standing in the middle of a field. Use this as the basis for discussing lossy compression.
  
  ## Optical illusion
  
  Explore images that look the same, but are different (when examined). One that I remember is an image white background, with a series of vertical thick black bars. Then to the right of each bar is a thin line with a linear gradient from dark to light. All black bars are identical, all thin lines are identical, however the lines get closer to the bars as you move to the right. (so multiple copies of the bar and line, but each copy to the right has the bar and line closer). This is perceived as the thin lines getting shorter to the right. An example of the hiding in plain sight illusion discussed above.
  
  ## When not to use lossy compression
  
  This relates to e-safety, digital literacy, ethics, etc.
  
  As well as the ones you said: Cartoon (traditional, such as Simpsons), line drawings. There are certain uses of photograph that should not be lossy encoded. These include medical X-ray (especially of fractures), and of nuclear power rods or aircraft, to check for build quality (fractures). I mention these 3 as I know this has been done, at great cost (£Millions), and safety risk.
  
  # What is JPEG
  
  Let me point out that JPEG is not a file format but a set of algorithms, some lossy some lossless. However just after release someone made a library that used some of the algorithms (Discrete cosine transform, hoffman encoding, etc), to make a lossy compression algorithm, this became a de-facto standard).
  
  > **Comments:**
  > **username_3**: `I would start with the lossless algorithms` seconded. JPEG is essentially `quantization + sampling + classical lossless techniques + transform`  
  > **username_3**: `I mention these 3 as I know this has been done` do you happen to have a reference for this? I bet it'd be an entertaining read.  
  > **username_2**: @username_3 Sorry it was a long time ago, and I don't remember.  
  > **username_0**: Thanks, I can't deal with maths behind JPEG, but run-length encoding and visual perception are surely within the reach of my students.  
  
  ---
  
  ### Answer by username_4 on 2018-01-28. Score: 8
  
  ## The basics
  
  Let me start with a quick overview of how JPEG (as usually applied) compresses color images.
  
  1. **Colorspace and downscaling.** Raw images are usually encoded in the RGB color space. The human eye is more sensitive to the luminosity of images than to their chromatic aspects — as an extreme example, we can understand black and white photos quite well. Therefore JPEG images store more information about luminosity and less about colors. This is implemented in two steps. First, the RGB color space is transformed into a different color space (in a linear fashion), YCbCr (similar to the TV color space YPbPr) in which Y is luminosity and Cb/Cr encode how much the pixels are blue rather than purple (Cb) and red rather than green (Cr). The Cb and Cr planes are then *downscaled* or *subsampled*, that is, their resolution is reduced, usually by a factor of 2 in the horizontal direction (the eye is more sensitive to the vertical direction). From this point on, the three planes are compressed individually.
  2. **Division into blocks.** Each of the color planes is now divided into 8×8 non-overlapping blocks (in contrast, in MP3 the sections are partially overlapping), which will be compressed more-or-less individually. This division is the reason for the so-called *blocking effects*, since the superimposed grid cuts through features of the image.
  3. **Spectral compression by quantization.** The basic insight behind JPEG is that nature is noisy, and this noise is filtered by the human visual system. It follows that we can throw away the noise without substantially affecting the way the image looks. It remains to decompose the image into its "substantial" and "noisy" components, and this is accomplished by a 2D Fourier transform. Recall we are trying to compress 8×8 blocks. The most important feature of a block is the average value of its 64 pixels, known as the DC component (or, zeroth Fourier coefficient); the DC components of all blocks are just a result of scaling down the original image by 8 in both dimensions. The importance of the other components, known as AC components, depends on their frequency — the higher frequencies correspond to noise, whereas the lower frequencies are more informative. However there is no clear cut distinction between features and noise. For this reason, we need to retain all components, albeit with different accuracy. Accordingly, we quantize the different components to various degrees, depending on the frequency: components corresponding to higher frequencies are quanrtized more. The exact quantization matrix depends on the disparity between the horizontal and vertical dimensions, and has been determined experimentally. (It also depends on the quality parameter of the compressor.)
  4. **Entropy encoding.** It remains to encode the quantized components into bits, which is done using Huffman coding, a standard encoding scheme. Since the DC components of all blocks are a downscaled version of the entire image, we expect them to vary continuously. Therefore instead of encoding their actual values, we encode them *differentially*, that is, we encode the difference from the preceding DC value. The AC components don't have this behavior, and so are encoded *absolutely*.
  
  ## Artifacts
  
  Why does compressing geometric drawings result in artifacts? Geometric drawings are very different from natural images. They don't vary continuously. Spectral compression results in "smearing" effects, since we are only retaining the lower frequencies. The best illustration of this is the [Gibbs phenomenon](https://en.wikipedia.org/wiki/Gibbs_phenomenon), which is the continuous analog of the same issue. When trying to represent a step function using Fourier series, we need to use all frequencies. If we suppress the high frequencies, we get something really different. Lines in a geometric drawing are the discrete analog of step functions.
  
  Another problem is the blocking effects. Imagine a diagonal line. JPEG partitions the space around these lines into 8×8 blocks, which are compressed individually. No attempt is made for the resulting lines to match exactly, and indeed they probably don't. This is very apparent because lines are very clean and exact, something that cannot be said about real-life images.
  
  ## Teaching ideas
  
  Using software like Matlab, you can show the effect of quantization on a real-world image, and then compare it to the effect of quantization on clip art. You can vary the quantization steps to show the effect of downplaying the higher frequencies (and also what happens when trying to suppress the lower frequencies).
  
  You can demonstrate the effect of blocking by varying the angle of a line which is compressed block-by-block, or fixing a line and varying the block size. You can use a low quality setting to emphasize the effect, and include the block boundaries to highlight its origins.
  
  > **Comments:**
  > **username_5**: This is a really thorough answer. Thank you for taking the time to write it!  
  > **username_3**: Also note how in the final step [the DCT matrix is traversed diagonally](http://dspguide.com/graphics/F_27_14.gif), in order to facilitate RLE.  
  > **username_0**: Thanks, excellent explanation of JPEG, but way too complex for my class (first year).  
  > **username_4**: This explanation is targeted at the teacher. If you don't understand how it works, you can't teach it even on a basic level.  
  
  ---
  
  ### Answer by username_3 on 2018-01-28. Score: 1
  
  > However, is it possible to explain why JPEG works this way, without venturing into domain change, DCT and so on?
  
  *Disclaimer 1*: I'm not sure if this is more of a comment or an answer and
  
  *Disclaimer 2*: this is not *really* my field, but
  
  I'd be inclined to answer: **no**.
  
  Firstly, I don't think it's *necessary*. I think the intuition behind the concept of transform (at least as in Fourier or DCT) is within the grasp of most high school kids that know *some* trigonometry: you can probably get away with drawing a simple function, decomposing it into sine waves and writing down their phase and amplitude while waving your hands a lot.
  
  You'd then proceed to reconstruct the original function by means of summation and persuade the students that the representation in terms of 4/5 phase/amplitude coefficients *is "more compact"* (finite, even) *than its representation in terms of infinite pairs in $R \times R$.*
  
  I'd argue it's also *not possible* because the DCT is also very much one of - if not *the* - working principles behind JPEG, differentiating it from classical lossy compression techniques.
  
  Operation in the frequency domain is ultimately *the* reason why \*PEG performs subjectively better than, say, resampling and requantizing at 240x180@16 colors, which is essentially a much naiver lossy compression method that operates purely in the space/time domain.
  
  Note, finally, how the excellent [Smith handbook](http://dspguide.com/ch27/6.htm) - along with probably several other resources - explicitly calls JPEG **“transform compression”**.
  
  
  ---
  
  