• TehPers ( TehPers@beehaw.org ) English
      3·
      3 months ago

      To answer the question posed in the title: if you’re processing images given to you by strangers, you should normalize RGB values by 255.

      The author isn’t advocating for you to do so. They are just exploring the two approaches. Even in the sentences following this one where they mention where 256 might work, they conclude with your colleagues breaking it all anyway by using 255.

  • Redkey ( Redkey@programming.dev ) 
    7·
    3 months ago

    There’s a whole lot of math in that article, and to be honest I’m not 100% clear on what they’re trying to say. I think they’re talking about how standard quantization isn’t making full or “accurate” use of the number space, although if you read through to the end they note that there’s really nothing terribly wrong with the standard way. But here’s my take.

    To simplify, I’ll use an integer range 0-4 instead of 0-255, but the principle’s the same.

    So, say that we’re quantizing from a floating-point range of 0.0-1.0 to an integer range of 0-4. Doing things the usual way, that’d be the orange numbers on the bottom of this diagram:

    But you can see that this doesn’t line up cleanly with a nice, regular division of the range into 5 equal blocks. If we just multiply by 4 (our integer max) and then round off to the nearest integer, the original value ranges represented by our integer 0 and 4 are half the size of the others. E.g. 0.0 to 0.49 become 0, but 0.5 to 1.49 become 1.

    Now, if we want to represent true 0.0 and 1.0 simply with our integer range (0-4), we should use 0 to exactly equal 0.0, and 4 to exactly equal 1.0. And converting from integer to float by calculating [value / 4] (or [value / 255] in reality) will get us exactly that. I wouldn’t change that.

    What I would suggest changing–to anyone who cares enough about it–is how the numbers are converted from float to integer. If you look at the diagram again, you can see that there is one orange integer per evenly-divided block, even if they don’t line up with the centers of the blocks. If we take a float (0.0-1.0) representation and multiply it by our integer max plus one, we’ll get the blue numbers at the top of the diagram. Then all we have to do is round down to the nearest integer and cap the result at 4 (or 255, or whatever our integer max is), and we’ll have a nice, even distribution in our conversion.

    I could have misunderstood, but it seems like all of the errors and uncertainty discussed in the article come from how the multiplied floats are truncated into integers. I think that this method eliminates that cleanly.

  • [object Object] ( panda_abyss@lemmy.ca ) 
    6·
    3 months ago

    The correct way to scale a number from one range to another is ((value - min) / (max - min)) * output_range + output_min and in this way the standard colour normalization should be (value - 0)/(255 - 0) * 1 + 0 = value / 255

    Now the question becomes whether an 8bit pixel value is a correct representation of a colour, or should it quantize to a half value. To answer this, I aggressively point to the above equation. It is fine. There are 256 well considered values, and any system with higher resolutions/precision should also map from 0 to 1 per channel.

    It should be obvious to the reader that 0.5/256 is not zero, and therefore not black. The 256 method is also not idempotent, applying it once destructively alters the data.

    And the argument that 256 has higher resolution: so would 354256, but there’s no sensible place that matters.