10D ISO 1600 is pushed one stop from 800

[JPS@no.komm]

I just took blackframe and super-overexposed white wall images at ISOs
800, 1600, and 3200 on my 10D. I converted them all to uncompressed DNG
files, and looked at the RAW data in a hex editor (set to look at the
data as decimal numbers, assuming 16-bit unsigned data). The data
patterns are the same for the 1600 and 3200, and both are a little
strange. You get a long string of even numbers, then a long string of
perfectly alternating odd and even numbers, then a long string of odd
numbers, then a long string of alternating numbers again. I don't know
if it's the camera or the DNG converter that is doing this to the data
(adding or subtracting one to blocks and striped blocks), but it's quite
clear that there are only 11 bits used for both ISO 3200 *AND* ISO 1600.
Here is some sample data from the ISO 1600 blackframe:

[url]http://www.pbase.com/jps_photo/image/38034746[/url]

Note how only the vertical stripes marked have odd numbers.
--

<>>< ><<> ><<> <>>< ><<> <>>< <>>< ><<>
John P Sheehy <JPS@no.komm>

><<> <>>< <>>< ><<> <>>< ><<> ><<> <>><

[Randall Ainsworth]

In article <6uo7t0l9sc57358l3k83qjaesd1vehq6q5@4ax.com>, <JPS@no.komm>
wrote:

> I just took blackframe and super-overexposed white wall images at ISOs
> 800, 1600, and 3200 on my 10D. I converted them all to uncompressed DNG
> files, and looked at the RAW data in a hex editor (set to look at the
> data as decimal numbers, assuming 16-bit unsigned data). The data
> patterns are the same for the 1600 and 3200, and both are a little
> strange. You get a long string of even numbers, then a long string of
> perfectly alternating odd and even numbers, then a long string of odd
> numbers, then a long string of alternating numbers again. I don't know
> if it's the camera or the DNG converter that is doing this to the data
> (adding or subtracting one to blocks and striped blocks), but it's quite
> clear that there are only 11 bits used for both ISO 3200 *AND* ISO 1600.
> Here is some sample data from the ISO 1600 blackframe:
>
> [url]http://www.pbase.com/jps_photo/image/38034746[/url]
>
> Note how only the vertical stripes marked have odd numbers.

So what?

[JPS@no.komm]

In message <301220040605305803%rag@nospam.techline.com>,
Randall Ainsworth <rag@nospam.techline.com> wrote:

>> Here is some sample data from the ISO 1600 blackframe:
>>
>> [url]http://www.pbase.com/jps_photo/image/38034746[/url]
>>
>> Note how only the vertical stripes marked have odd numbers.

>
>So what?

So, you aren't very bright if you replied to an on-topic thread that
doesn't interest you.
--

<>>< ><<> ><<> <>>< ><<> <>>< <>>< ><<>
John P Sheehy <JPS@no.komm>

><<> <>>< <>>< ><<> <>>< ><<> ><<> <>><

[G.T.]

<JPS@no.komm> wrote in message
news:tc5bt0lr694q3fmhltvggprjtkau3vgc9a@4ax.com...

> In message <301220040605305803%rag@nospam.techline.com>,
> Randall Ainsworth <rag@nospam.techline.com> wrote:
>

> >> Here is some sample data from the ISO 1600 blackframe:
> >>
> >> [url]http://www.pbase.com/jps_photo/image/38034746[/url]
> >>
> >> Note how only the vertical stripes marked have odd numbers.

> >
> >So what?

>
> So, you aren't very bright if you replied to an on-topic thread that
> doesn't interest you.

Well, I'm interested but as someone who is ignorant of what I'm looking at I
have no idea what the significance of the odd numbers is even after reading
your post 3 times.

Greg

[Randall Ainsworth]

In article <tc5bt0lr694q3fmhltvggprjtkau3vgc9a@4ax.com>, <JPS@no.komm>
wrote:

> So, you aren't very bright if you replied to an on-topic thread that
> doesn't interest you.

Who gives a shit about the sequence of 1's and 0's? It's the end result
that's important...it's about photography.

[dylan]

"Randall Ainsworth" <rag@nospam.techline.com> wrote in message
news:311220041158443563%rag@nospam.techline.com...

> In article <tc5bt0lr694q3fmhltvggprjtkau3vgc9a@4ax.com>, <JPS@no.komm>
> wrote:
>

>> So, you aren't very bright if you replied to an on-topic thread that
>> doesn't interest you.

>
> Who gives a shit about the sequence of 1's and 0's? It's the end result
> that's important...it's about photography.

This group's about Photography not digits ? ;oO

[David J Taylor]

dylan wrote:

> "Randall Ainsworth" <rag@nospam.techline.com> wrote in message
> news:311220041158443563%rag@nospam.techline.com...

>> In article <tc5bt0lr694q3fmhltvggprjtkau3vgc9a@4ax.com>,
>> <JPS@no.komm> wrote:
>>

>>> So, you aren't very bright if you replied to an on-topic thread that
>>> doesn't interest you.

>>
>> Who gives a shit about the sequence of 1's and 0's? It's the end
>> result that's important...it's about photography.

>
> This group's about Photography not digits ? ;oO

Well, actually, it's about SLR systems. I would have thought that a
better understanding of the internals of something might have enabled you
to make the best use of it? I think John is trying to explain something,
without shouting it too loudly.

Happy New Year,
David

[JPS@no.komm]

In message <10tb712t3adom5f@corp.supernews.com>,
"G.T." <getnews1@dslextreme.com> wrote:

>
><JPS@no.komm> wrote in message
>news:tc5bt0lr694q3fmhltvggprjtkau3vgc9a@4ax.com.. .

>> In message <301220040605305803%rag@nospam.techline.com>,
>> Randall Ainsworth <rag@nospam.techline.com> wrote:
>>

>> >> Here is some sample data from the ISO 1600 blackframe:
>> >>
>> >> [url]http://www.pbase.com/jps_photo/image/38034746[/url]
>> >>
>> >> Note how only the vertical stripes marked have odd numbers.
>> >
>> >So what?

>>
>> So, you aren't very bright if you replied to an on-topic thread that
>> doesn't interest you.

>
>Well, I'm interested but as someone who is ignorant of what I'm looking at I
>have no idea what the significance of the odd numbers is even after reading
>your post 3 times.

Note that I said, "that doesn't interest you"; not, "that isn't clear to
you".

The odd numbers are a peculiarity; I don't understand them myself. They
occur in the raw data (which reads like english text in the image) not
at all at the beginning of the image (where all levels are even
numbers), then further into the image every other pixel has an odd
level, and there are sections that are all odd as well. The DNG
converter is not supposed to alter data at all, with the exception of
filling in defective pixels with interpolated data (which would result
in individual pixels breaking out of the pattern).

One speculation that I have is that Canon is doing this to make the data
look OK in a histogram (an equal number of odd and even values should be
expected

The fact, however, that the RAW data is all even or odd within patterns
suggests that the data is not really 12-bit at its source, but rather,
11-bit. What does this mean for the user? It means that you get the
same quality data by setting the camera to ISO 800 instead of 1600, if
you are shooting RAW, with an EC of -1. It also means that you get an
extra stop of highlights this way, as the camera would clip any value
above 2023 if the camera were set to ISO 1600.

For those of us who shoot in low light, this is actually very beneficial
to know. I have suspected that the camera is cheating ISO 1600 for a
long time, and consequently, I have been setting the camera to ISO 800
instead of ISO 1600 when shooting wildlife at dusk. That way, if there
truly is enough light for ISO 800, I will get the better ISO 800 image,
but if there is not enough light, it will shoot a shot that will "push"
to ISO 1600 with the same quality and more dynamic headroom than if the
camera were actually set to ISO 1600! I even set the EC to +1 at ISO
800 sometimes, if there aren't a lot of bright highlights. That will
make the aperture stop down more, when there is sufficient light
(effectively a better ISO 400 than if the camera were actually set to
ISO 400, because more bits represent the subject's dynamic range), but
will also work at ISO 800, or "1600" (as good or better than the camera
does 1600 with 0 EC) when necessary.

Also, "ISO 3200" or "H" is actually ISO 1600, under-exposed and pushed
by a stop, the same way.
--

<>>< ><<> ><<> <>>< ><<> <>>< <>>< ><<>
John P Sheehy <JPS@no.komm>

><<> <>>< <>>< ><<> <>>< ><<> ><<> <>><

[JPS@no.komm]

In message <311220041158443563%rag@nospam.techline.com>,
Randall Ainsworth <rag@nospam.techline.com> wrote:

>In article <tc5bt0lr694q3fmhltvggprjtkau3vgc9a@4ax.com>, <JPS@no.komm>
>wrote:

>> So, you aren't very bright if you replied to an on-topic thread that
>> doesn't interest you.

>Who gives a shit about the sequence of 1's and 0's? It's the end result
>that's important...it's about photography.

Digital photography completely depends on ones and zeros for the "end
result".
--

<>>< ><<> ><<> <>>< ><<> <>>< <>>< ><<>
John P Sheehy <JPS@no.komm>

><<> <>>< <>>< ><<> <>>< ><<> ><<> <>><

[Frank ess]

[email]JPS@no.komm[/email] wrote:

> In message <311220041158443563%rag@nospam.techline.com>,
> Randall Ainsworth <rag@nospam.techline.com> wrote:
>

>> In article <tc5bt0lr694q3fmhltvggprjtkau3vgc9a@4ax.com>,
>> <JPS@no.komm> wrote:

>

>>> So, you aren't very bright if you replied to an on-topic thread that
>>> doesn't interest you.

>

>> Who gives a shit about the sequence of 1's and 0's? It's the end
>> result that's important...it's about photography.

>
> Digital photography completely depends on ones and zeros for the "end
> result".
>

I think what he was saying is along the lines of, "It's not important to
me how a refrgerator works, as long as it keeps the beer at 38.7º F".

--
Frank ess

[Randall Ainsworth]

In article <38sbt0hrfrqal87a46k8qq5hq0vao9r4ln@4ax.com>, <JPS@no.komm>
wrote:

> Digital photography completely depends on ones and zeros for the "end
> result".

You guys get too bogged down in the electronics/computer end of things.

[JPS@no.komm]

In message <EaOdnXpU9MhCakjcRVn-jA@giganews.com>,
"Frank ess" <frank@fshe2fs.com> wrote:

>JPS@no.komm wrote:

>> In message <311220041158443563%rag@nospam.techline.com>,
>> Randall Ainsworth <rag@nospam.techline.com> wrote:
>>

>>> In article <tc5bt0lr694q3fmhltvggprjtkau3vgc9a@4ax.com>,
>>> <JPS@no.komm> wrote:

>>

>>>> So, you aren't very bright if you replied to an on-topic thread that
>>>> doesn't interest you.

>>

>>> Who gives a shit about the sequence of 1's and 0's? It's the end
>>> result that's important...it's about photography.

>>
>> Digital photography completely depends on ones and zeros for the "end
>> result".
>>

>
>I think what he was saying is along the lines of, "It's not important to
>me how a refrgerator works, as long as it keeps the beer at 38.7º F".

Yes, but if you know that filling the empty shelves with empty boxes
allows less air to exchange when you open the door, and get rid of all
the empty space, your beer will be colder, and your electric bills will
be lower.
--

<>>< ><<> ><<> <>>< ><<> <>>< <>>< ><<>
John P Sheehy <JPS@no.komm>

><<> <>>< <>>< ><<> <>>< ><<> ><<> <>><

[JPS@no.komm]

In message <311220041738269965%rag@nospam.techline.com>,
Randall Ainsworth <rag@nospam.techline.com> wrote:

>In article <38sbt0hrfrqal87a46k8qq5hq0vao9r4ln@4ax.com>, <JPS@no.komm>
>wrote:
>

>> Digital photography completely depends on ones and zeros for the "end
>> result".

>
>You guys get too bogged down in the electronics/computer end of things.

It only sounds "bogged down" to you because you're simple. This is
natural, ABC, 123 stuff to me. I'm smart enough to know that the more
you know how things really work, the simpler it is for you to deal with
them. Less magic, and less black boxes with no controls.

Your less-bogged-down approach will result in images with more noise,
and less shadow detail.
--

<>>< ><<> ><<> <>>< ><<> <>>< <>>< ><<>
John P Sheehy <JPS@no.komm>

><<> <>>< <>>< ><<> <>>< ><<> ><<> <>><

[Randall Ainsworth]

In article <l05ct0d2c14tv4ltakn98h6rqugehokbqc@4ax.com>, <JPS@no.komm>
wrote:

> Your less-bogged-down approach will result in images with more noise,
> and less shadow detail.

I doubt it.

[Alan Browne]

Randall Ainsworth wrote:

> Who gives a shit about the sequence of 1's and 0's? It's the end result
> that's important...it's about photography.

Spoken like a poor carpenter who doesn't understand tools.





--
-- r.p.e.35mm user resource: [url]http://www.aliasimages.com/rpe35mmur.htm[/url]
-- r.p.d.slr-systems: [url]http://www.aliasimages.com/rpdslrsysur.htm[/url]
-- [SI] gallery & rulz: [url]http://www.pbase.com/shootin[/url]
-- e-meil: there's no such thing as a FreeLunch.

[Dustbunny]

On Sat, 01 Jan 2005 00:44:43 -0800, Randall Ainsworth
<rag@nospam.techline.com> wrote:

>In article <l05ct0d2c14tv4ltakn98h6rqugehokbqc@4ax.com>, <JPS@no.komm>
>wrote:
>

>> Your less-bogged-down approach will result in images with more noise,
>> and less shadow detail.

>
>I doubt it.

------------------------
If someone told you your 8MP camera really only had 6MP, you'd be
upset, right? Or suppose your 48-bit scanner was really only using
16-bits. Well think of not using all the bits in those terms, maybe
then you'll appreciate what is being described here. Or look up the
word

[Colin D]

[email]JPS@no.komm[/email] wrote:

>
> I just took blackframe and super-overexposed white wall images at ISOs
> 800, 1600, and 3200 on my 10D. I converted them all to uncompressed DNG
> files, and looked at the RAW data in a hex editor (set to look at the
> data as decimal numbers, assuming 16-bit unsigned data). The data
> patterns are the same for the 1600 and 3200, and both are a little
> strange. You get a long string of even numbers, then a long string of
> perfectly alternating odd and even numbers, then a long string of odd
> numbers, then a long string of alternating numbers again. I don't know
> if it's the camera or the DNG converter that is doing this to the data
> (adding or subtracting one to blocks and striped blocks), but it's quite
> clear that there are only 11 bits used for both ISO 3200 *AND* ISO 1600.
> Here is some sample data from the ISO 1600 blackframe:
>
> [url]http://www.pbase.com/jps_photo/image/38034746[/url]
>
> Note how only the vertical stripes marked have odd numbers.
> --
>
> <>>< ><<> ><<> <>>< ><<> <>>< <>>< ><<>
> John P Sheehy <JPS@no.komm>

> ><<> <>>< <>>< ><<> <>>< ><<> ><<> <>><

Well, I'm quite familiar with bits, bytes, hex and all that, having been
in programming for about 20 years, but I am at a loss to understand what
you are driving at here, John. 12 bits converts to decimal 4095, or
4096 separate steps, but the data in your graphic, since it is for
blackframe, is averaging around 8 or 9 bits (decimal 240 - 268 approx).
The odd numbers imply that bit 0 is a 1, that's all, but I can't see
from this that the highest binary value is represented by 11 bits, or
2048 decimal.

Could you elucidate further, please?

Colin

[JPS@no.komm]

In message <41D7719F.5D7057AD@killspam.127.0.0.1>,
Colin D <ColinD@killspam.127.0.0.1> wrote:

>Well, I'm quite familiar with bits, bytes, hex and all that, having been
>in programming for about 20 years, but I am at a loss to understand what
>you are driving at here, John. 12 bits converts to decimal 4095, or
>4096 separate steps, but the data in your graphic, since it is for
>blackframe, is averaging around 8 or 9 bits (decimal 240 - 268 approx).
>The odd numbers imply that bit 0 is a 1, that's all, but I can't see
>from this that the highest binary value is represented by 11 bits, or
>2048 decimal.

>Could you elucidate further, please?

If the winning lottery numbers were all odd one year, odd every other
week and even on the others the next year, and all even the next year,
what would you think? Do all numbers have an equal chance at any given
time?
--

<>>< ><<> ><<> <>>< ><<> <>>< <>>< ><<>
John P Sheehy <JPS@no.komm>

><<> <>>< <>>< ><<> <>>< ><<> ><<> <>><

[Wolfgang Weisselberg]

[email]JPS@no.komm[/email] <JPS@no.komm> wrote:

> "Frank ess" <frank@fshe2fs.com> wrote:

>>I think what he was saying is along the lines of, "It's not important to
>>me how a refrgerator works, as long as it keeps the beer at 38.7º F".

> Yes, but if you know that filling the empty shelves with empty boxes
> allows less air to exchange when you open the door,

Please look up how little enery it needs to cool down air compared
to water, beer and even the plastic and metal of the fridge itself.

> and get rid of all the empty space, your beer will be colder,
> and your electric bills will be lower.

Buying a smaller fridge in the first place will handsomely remove
all that empty space. This also reduces the surface of the
refrigerator unit, thus reducing the amount of heat that leaks
through from the outside, thus lowering your electric bill

Your beer will not be cooler in any way, if you cool it long
enough and don't have the door open a lot of the time.

F'up set since OT.
-Wolfgang

[Wolfgang Weisselberg]

[email]JPS@no.komm[/email] <JPS@no.komm> wrote:

> The DNG converter is not supposed to alter data at all,

Oh yes, it is! Unless the 10D has a Foveon chip, which it hasn't,
3/4 of the data has to be interpolated. Think bayer patterns
.... and then remember that (customarily) in each row there are
green pixels, but red and blue pixels are usually on seperate rows.

> The fact, however, that the RAW data is all even or odd within patterns
> suggests that the data is not really 12-bit at its source, but rather,
> 11-bit.

This is but one of many possible causes. Programs have been known
to have bugs, the bayer pattern interpolation may cause this,
a non-perfect sensor (some cameras are even differently sensitive
for UV light on their green pixels in alternating rows!) and many
other causes may exist.

> What does this mean for the user? It means that you get the
> same quality data by setting the camera to ISO 800 instead of 1600, if
> you are shooting RAW, with an EC of -1. It also means that you get an
> extra stop of highlights this way, as the camera would clip any value
> above 2023 if the camera were set to ISO 1600.

Have you actually tested that, or is that just a guess? For all
we know, the camera could also compress the highlights, e.g.:

0-1800 => 0-3600 (i.e. *= 2)
1801-4096 => 1801-2023 (compress) => 3602-4096 (*= 2)

> I have suspected that the camera is cheating ISO 1600 for a
> long time,

How do you think the camera handles ISO 800, 400, 200, if not by
'cheating', if not by collecting many bits' depth at ISO 100 and
--- in the easiest case --- simply bit-shifting them?

> and consequently, I have been setting the camera to ISO 800
> instead of ISO 1600 when shooting wildlife at dusk. That way, if there
> truly is enough light for ISO 800, I will get the better ISO 800 image,

Assuming the image is _better_ in a photographic sense.
Something you can _see_ in the finalized picture.

> but if there is not enough light, it will shoot a shot that will "push"
> to ISO 1600 with the same quality and more dynamic headroom than if the
> camera were actually set to ISO 1600!

Unless you overexposure your image (so you actually reach that
"headroom"), said headroom is purely theoretical and does you not
a bit of good. You could as well shoot at 1600 then. And if you
use anything but RAW (and thus a much more laborious workflow)
you loose dynamic range, since JPEG compresses said range ...

> I even set the EC to +1 at ISO
> 800 sometimes, if there aren't a lot of bright highlights. That will
> make the aperture stop down more, when there is sufficient light
> (effectively a better ISO 400 than if the camera were actually set to
> ISO 400, because more bits represent the subject's dynamic range),

So you basically 'expose to the right'?
[url]http://www.luminous-landscape.com/tutorials/expose-right.shtml[/url]

> Also, "ISO 3200" or "H" is actually ISO 1600, under-exposed and pushed
> by a stop, the same way.

And there may be special routines helping the image look better
being run as well.

-Wolfgang