Skip to content

Rewrite BCPD description of the colour data format#662

Open
quinnyo wants to merge 12 commits into
gbdev:masterfrom
quinnyo:bcpd-colour-format
Open

Rewrite BCPD description of the colour data format#662
quinnyo wants to merge 12 commits into
gbdev:masterfrom
quinnyo:bcpd-colour-format

Conversation

@quinnyo
Copy link
Copy Markdown
Contributor

@quinnyo quinnyo commented Apr 19, 2026

This is a rewrite of the BCPD section to hopefully reduce possible confusion and make the colour data format less ambiguous.
Particularly:

  • avoid relying on reader interpretation of 'RGB555'
  • make explicit that "little-endian" is about byte order of the 16 bit colour word
  • reverse the bit table so MSB is first (this table is just about the only one that used the LSB order)
  • explicitly state that you access one byte at a time (need to read/write twice per colour)

Fixes #643 (I think)

@nummacway
Copy link
Copy Markdown

I suggest we add an example:
Farbberechnung

@ISSOtm
Copy link
Copy Markdown
Member

ISSOtm commented Apr 20, 2026

That's actually a pretty good idea! A visual representation of how the encoding is transformed is very helpful.

What I would like, actually, is to add a third layer, which shows how the colour is converted into RGB555 format, and then shows the two bytes being swapped in memory. Is the above image available in SVG form? I'd like to tweak it to propose some further improvements. (Perhaps we can add that in a follow-up PR, but still.)

Copy link
Copy Markdown
Member

@ISSOtm ISSOtm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the wording improvements!

Comment thread src/Palettes.md Outdated
Comment thread src/Palettes.md Outdated
Comment thread src/Palettes.md Outdated
Comment thread src/Palettes.md Outdated
Comment thread src/Palettes.md Outdated
@avivace
Copy link
Copy Markdown
Member

avivace commented Apr 20, 2026

rgb555_encoding(1)(1).drawio
I've started trying to redraw it in draw.io @ISSOtm

@nummacway
Copy link
Copy Markdown

nummacway commented Apr 20, 2026

Are these two distinct proposals? Or are they two distinct pictures that are both to be added?

For my design, I actually like my "ATA/ATAPI cable" style because it is immediately clear which and how many bits are involved.

@avivace
Copy link
Copy Markdown
Member

avivace commented Apr 20, 2026

I like the cable design as well @nummacway but I couldn't find a way to reproduce it in draw.io (which keeps the diagrams in a 'editable' project XML files so it's a preferred format here) :/

How was your diagram drawn?

@nummacway
Copy link
Copy Markdown

Didn't know draw.io was preferred.
I'll change your diagram to snap to grid, to make future updates easier, and will try to have it import my shapes. I found no way to draw this with draw.io's tools.
I drew them in Affinity, because that's what I use. Because the way it exports text, it was never meant as the final result, and I would have recreated it with a text editor.

@nummacway
Copy link
Copy Markdown

This is only the top part of the original file.
Farbberechnung.drawio
I found no way of inserting an editable SVG, so I had to recreate my four custom shapes with drawio's language. For reference, these are their source codes (concatenated in a single file):
ataatapi-shapes.xml
I do find my two separate colors for the green "cables" nice, but I have no idea which colors to use to align with Pan Docs. I leave that to you.

For those who don't want to open drawio, here's a screenshot:
image

@ISSOtm
Copy link
Copy Markdown
Member

ISSOtm commented Apr 20, 2026

I've started transcribing the design as a themable SVG (since no matter how much I try, I can't edit SVGs in graphical editors ^^') I'll host a rendered version online when I've gotten parity, though I also have design changes to suggest. In particular, this uses Pan Docs' font, which has different glyph widths, thus we may consider tweaking the font size? (Though I'm also thinking of swapping the values of the red and green components, simply to balance the amount of characters? xD)

image

(Disregard the misaligned digits, I haven't bothered fixing those yet.) One change I've made is that the “ATA ribbons” are drawn behind the borders, which I'm not fully sure about. One possible change is to squish the ribbons vertically by 1 pixel at each edge, so that the black edges still render there. Feel free to throw feedback!

@ISSOtm
Copy link
Copy Markdown
Member

ISSOtm commented Apr 20, 2026

I do find my two separate colors for the green "cables" nice, but I have no idea which colors to use to align with Pan Docs. I leave that to you.

One layout idea I had was to separate the format conversion and the RAM encoding into two separate layers, so there'd be three total: the RGB8 colour, the RGB555 version (still with the “ribbon cables”, and then two arrows pointing out how the bytes are swapped in memory order, and also dotted lines going left and right to suggest that there are more memory cells around.

@quinnyo
Copy link
Copy Markdown
Contributor Author

quinnyo commented Apr 21, 2026

I like the diagram, but it took me a little while to understand where you're coming from.
I see now that it's about the relationship or converting between 24-bit RGB and the CGB. I did not think about 24-bit colour or really any other format or conversion at all while I was working on this. [This probably says more about me than anything else.]

So I don't think I'm really the target audience for the diagram but I can see the value in it because a lot of people would be coming into this more familiar with this flavour of 24-bit colour than any other. It's a good way to illustrate "what RGB555 means" on a technical level.


I've got one issue with the diagram itself: the order of the ribbon layers. Could the blue ribbon be on a higher layer than the green one? (blue should occlude green) RGB --> R, G_low, G_high, B


@ISSOtm @avivace thanks for the review and patches!
[every time I write 'N(-)bit(s)' I have to check what I'm supposed to do, and this time I didn't! 🐧]

@nummacway
Copy link
Copy Markdown

The layer idea was: Small cables over wide cables.

@ISSOtm
Copy link
Copy Markdown
Member

ISSOtm commented Apr 21, 2026

My current iteration of the diagrams can be viewed online at https://eldred.fr/pandocs+662/Palettes.html#lcd-color-palettes-cgb-only; criticism welcome.

[EDIT]: Updated with the state of the PR as of #662 (comment).

@nummacway

This comment was marked as resolved.

@ISSOtm
Copy link
Copy Markdown
Member

ISSOtm commented May 6, 2026

Since there have been no replies to my last comment in the last two weeks, I'm going to push my changes to this PR, and we'll keep working from there. Anyone opposed to this, please voice your opinion (now or after) and those changes will be reverted.

@ISSOtm ISSOtm force-pushed the bcpd-colour-format branch from c2fcdb5 to 1b7a344 Compare May 6, 2026 18:23
quinnyo and others added 7 commits May 6, 2026 20:47
- less ambiguous description of the colour data format
- clarify meaning of endianness (byte order)
- reverse the order of the bit table, so MSB appears first, matching
  others
Co-authored-by: Eldred Habert <me@eldred.fr>
Co-authored-by: Eldred Habert <me@eldred.fr>
Co-authored-by: "Janni K." <24881711+nummacway@users.noreply.github.com>
Co-authored-by: Rangi42 <sylvie.oukaour+rangi42@gmail.com>
Those have been deprecated in newer `hardware.inc`, and really,
the header names were hard to read with the duplicated names,
especially the OBJ ones.
Anyone being confused by old code should be able to deduce the
alternate naming from their definitions in the accompanying
`hardware.inc`, or they can look it up on older versions of Pan Docs,
or they can ask and be explained.

I nonetheless expect that the impact will be very low at this point.
@ISSOtm ISSOtm force-pushed the bcpd-colour-format branch from 1b7a344 to f924874 Compare May 6, 2026 19:17
@ISSOtm
Copy link
Copy Markdown
Member

ISSOtm commented May 6, 2026

I'm now personally happy with the state of this PR, but I'd like opinions from @quinnyo and @nummacway before merging.

@ISSOtm ISSOtm requested a review from avivace May 6, 2026 19:23
@ISSOtm
Copy link
Copy Markdown
Member

ISSOtm commented May 6, 2026

Also @DelayRGC, since she opened the original issue.

Copy link
Copy Markdown
Member

@avivace avivace left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor comments

Comment thread src/Palettes.md
byte of BGP0 color #1 via BCPD, which contains the color's blue and upper green bits.
[^bit15]:
The 16th bit, bit 15, is **ignored** by the hardware.
In discussion, that bit is generally clear (for example, the canonical pure white is `7FFF` and not `FFFF`), but the hardware treats both identically: it's fine to fill color RAM with $FF bytes to set it to all-white!
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
In discussion, that bit is generally clear (for example, the canonical pure white is `7FFF` and not `FFFF`), but the hardware treats both identically: it's fine to fill color RAM with $FF bytes to set it to all-white!
Conventionally, that bit is generally clear (for example, the canonical pure white is `7FFF` and not `FFFF`), but the hardware treats both identically: it's fine to fill color RAM with $FF bytes to set it to all-white.

'In discussion' sounds a bit confusing here, imo

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also the ! is a bit off tone

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to avoid implying that it would be convention outside of the programs themselves. Perhaps "in writing"?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Proposal (that is definitely not meant as a final idea, but maybe some words can be taken up):

When referring to a certain color, the 16-bit value is usually written with the highest bit set to 0, e.g. white will be referred to as $7FFF, not $FFFF.

(Also note the dollars.)

Copy link
Copy Markdown
Member

@ISSOtm ISSOtm May 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
In discussion, that bit is generally clear (for example, the canonical pure white is `7FFF` and not `FFFF`), but the hardware treats both identically: it's fine to fill color RAM with $FF bytes to set it to all-white!
Colors are conventionally referred to with that bit set to 0 (for example, the canonical pure white is `7FFF` and not `FFFF`), but the hardware treats both identically: thus, it's fine to fill color RAM with $FF bytes to set it to all-white.

I think it's better not to use the dollars here, as they'd imply numbers slightly more than colours. But that's fairly nitpicky.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this better?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm undecided on that one.

I would like to add one thing about the dollar sign: The notation without it could become ambiguous if no A-F digits are present. I think we need some way to mark that the color values (I'm not calling them colors!) are hexadecimal. Since we don't have the power (or need) for a unique notation like CSS's #, $ will do.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"referred to with that bit set to 0" sounds even more quirky, IMO

what about "The canonical form of a color value has that bit set to 0." ?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, no, since that implies that this is also the form that should be stuck to on the hardware itself. I'm trying to keep it clear that this is only the case in literature. (I switched to “that bit set to 0” because “that bit clear” is easy to misread, I've found.)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if the point here is truly about this "convention" or if this note is intended as a way to illustrate that bit 15 is not used by the system.

  • If it's about convention, I'd recommend removing the statement entirely. This is because (I think that) Pan Docs can/should avoid commenting on, relying on, or defining convention whereever possible/reasonable.
  • If it's an illustrative example, remove the reference/s to convention/canon. Strictly speaking, the reader already has the information, so rephrase this as a caution/tip.
    • Possibly focusing more on (not) making assumptions about bit 15, one way or the other, rather than just about $FFFF and $7FFF both being white.

dollar prefix:
I'd prefer to see the dollar sign here. Maybe this isn't as common as I'd expect, but to me these prefixes are just a way to tell the parser (human or otherwise) that the value is rendered in a particular way. So 0x or $ just signifies "hexadecimal nybbles follow", not really anything about the meaning of the data.

A (two's complement) signed integer must have the same issue -- in other words, the signed integer $FFFF is not rendered as a number. You have to expend mental effort to interpret it.


I'm not calling them colors!

I agree. I began using the term 'colour word' in my initial changes. I chose to not use 'colour value', though, because of the existing meaning/s of 'value' in colour theory / visual art / HSV.

Comment thread src/Palettes.md Outdated
Comment thread src/Palettes.md
All background colors are initialized as white by the boot ROM, however it is a
good idea to initialize all colors yourself, e.g. if implementing
a soft-reset mechanic.
All background colors are initialized as white by the boot ROM, however it is a good idea to initialize all colors yourself, e.g. if implementing a soft-reset mechanic.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
All background colors are initialized as white by the boot ROM, however it is a good idea to initialize all colors yourself, e.g. if implementing a soft-reset mechanic.
All background colors are initialized as white by the boot ROM, however it is a good idea to initialize all colors yourself, e.g. by implementing a soft-reset mechanic.

e.g. if implementing ... doesn't sound good

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"by" changes the meaning of the sentence. Maybe "for" would work better?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would also change the meaning of the sentence. The point is that it's a good idea to set all palettes yourself if there's any chance that the init code may run more than once, such as if the game has a soft-reset mechanism.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think of "for a soft-reset mechanic" rather than "for/by/if implementing a soft-reset mechanic"?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like Isso's original proposal.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, changing it to "by" would make it have a different, also undesirable/confusing meaning. But the "if implementing" thing is odd, and the whole point being made is a bit strange, actually:

When wouldn't it be a good idea to initialise all the colours yourself? If they are all white you can't see anything.
Regardless of the behaviour of the boot ROM, it's a "good idea" to initialise the palettes, because you want your palettes to be the ones being used. At least I think so.

[Note: I don't actually know how true the boot ROM statement is, I assume someone checked.]

I know this behaviour is specific to the BG colours, but this "tip" (and the similar one for OB) could be moved up to the general explanation above -- it's really not specific to the register/s.

Comment thread src/Palettes.md Outdated
@quinnyo
Copy link
Copy Markdown
Contributor Author

quinnyo commented May 8, 2026

Hi, sorry I haven't been around, I'll be able to have a look at this in a day or so!

@ISSOtm
Copy link
Copy Markdown
Member

ISSOtm commented May 29, 2026

Ping?

@quinnyo
Copy link
Copy Markdown
Contributor Author

quinnyo commented May 30, 2026

Reviewing now, sorry!

Copy link
Copy Markdown
Contributor Author

@quinnyo quinnyo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's pretty much there.

The structural change -- moving the explanation out from under the register headings -- is a very welcome one. Good move.

Also good to settle on one set of register names (and on the more sensible Index and Data ones).

The biggest issue I see is not really introduced by this change, but is made more apparent by it -- the lack of basic introductory information about the CGB colours/palettes.

[possible further work] The bit 15 discussion seems to suggest there might be value in adding something about reading/writing hexadecimal colour words. Specifically about the nybbles not being aligned with the R,G,B components and the difficulty in interpreting that.

Comment thread src/Palettes.md

The color palettes are stored in two dedicated banks of palette RAM (or <abbr title="Color RAM">CRAM</abbr> for *color RAM*), 64 bytes each[^cram_size]: one for background/window palettes and the other for OBJ palettes.

The two bytes of each color are stored **little-endian**, meaning that the low byte comes first.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

explicit byte order -- avoid confusion about most/least significant bit

Suggested change
The two bytes of each color are stored **little-endian**, meaning that the low byte comes first.
The two bytes of each color are stored in **little-endian** byte order, meaning that the low byte comes first.

Comment thread src/Palettes.md
BGP1 color number 0, and so on. Thus, address $03 allows accessing the second (upper)
byte of BGP0 color #1 via BCPD, which contains the color's blue and upper green bits.
[^bit15]:
The 16th bit, bit 15, is **ignored** by the hardware.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's real though, right? I mean it's stored somewhere. The PPU "ignores" it, I suppose:

Suggested change
The 16th bit, bit 15, is **ignored** by the hardware.
The 16th bit, bit 15, has no effect on the colour and is not used for any other purpose by the system.

Comment thread src/Palettes.md Outdated
Comment thread src/Palettes.md
Comment on lines 30 to 31
## LCD Color Palettes (CGB only)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Introduction? CGB has colours, palettes.

The old bit summing up the CRAM size was essentially doing this before. (Not that that was a good solution)

Comment thread src/Palettes.md

{{#include imgs/src/rgb555.svg}}

The color palettes are stored in two dedicated banks of palette RAM (or <abbr title="Color RAM">CRAM</abbr> for *color RAM*), 64 bytes each[^cram_size]: one for background/window palettes and the other for OBJ palettes.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Palettes haven't really been introduced at this point, but this reads like they have been.

Comment thread src/Palettes.md
byte of BGP0 color #1 via BCPD, which contains the color's blue and upper green bits.
[^bit15]:
The 16th bit, bit 15, is **ignored** by the hardware.
In discussion, that bit is generally clear (for example, the canonical pure white is `7FFF` and not `FFFF`), but the hardware treats both identically: it's fine to fill color RAM with $FF bytes to set it to all-white!
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if the point here is truly about this "convention" or if this note is intended as a way to illustrate that bit 15 is not used by the system.

  • If it's about convention, I'd recommend removing the statement entirely. This is because (I think that) Pan Docs can/should avoid commenting on, relying on, or defining convention whereever possible/reasonable.
  • If it's an illustrative example, remove the reference/s to convention/canon. Strictly speaking, the reader already has the information, so rephrase this as a caution/tip.
    • Possibly focusing more on (not) making assumptions about bit 15, one way or the other, rather than just about $FFFF and $7FFF both being white.

dollar prefix:
I'd prefer to see the dollar sign here. Maybe this isn't as common as I'd expect, but to me these prefixes are just a way to tell the parser (human or otherwise) that the value is rendered in a particular way. So 0x or $ just signifies "hexadecimal nybbles follow", not really anything about the meaning of the data.

A (two's complement) signed integer must have the same issue -- in other words, the signed integer $FFFF is not rendered as a number. You have to expend mental effort to interpret it.


I'm not calling them colors!

I agree. I began using the term 'colour word' in my initial changes. I chose to not use 'colour value', though, because of the existing meaning/s of 'value' in colour theory / visual art / HSV.

Comment thread src/Palettes.md
This register is used to address a byte in the CGB's background palette RAM.
Since there are 8 palettes, 8 palettes × 4 colors/palette × 2 bytes/color = 64 bytes
can be addressed.
Unlike VRAM, OAM, or wave RAM, CRAM cannot be accessed directly, but instead through pairs of registers: one “data” register provides a one-byte view into CRAM, while the corresponding “index” register controls which CRAM address the data register is bound to.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

split long sentence
index before access

Suggested change
Unlike VRAM, OAM, or wave RAM, CRAM cannot be accessed directly, but instead through pairs of registers: one “data” register provides a one-byte view into CRAM, while the corresponding “index” register controls which CRAM address the data register is bound to.
Unlike VRAM, OAM, or wave RAM, CRAM is not exposed in the memory map and cannot be accessed directly.
Instead, each bank of CRAM is accessed through a pair of registers: one register is used to select a CRAM address, and the other provides read/write access to the byte at that address.

Comment thread src/Palettes.md
Since there are 8 palettes, 8 palettes × 4 colors/palette × 2 bytes/color = 64 bytes
can be addressed.
Unlike VRAM, OAM, or wave RAM, CRAM cannot be accessed directly, but instead through pairs of registers: one “data” register provides a one-byte view into CRAM, while the corresponding “index” register controls which CRAM address the data register is bound to.
Much like VRAM, CRAM is inaccessible when the PPU is reading from it, that is, during [Mode 3](<#PPU modes>): writes are ignored, and reads return $FF.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the prior point starting "Unlike VRAM, ..." this one starting "Much like VRAM" makes a slightly off balance contrast.
You could put a "However, ..." at the start, but that reinforces it as an intentional contrast, which doesn't really make sense.

Comment thread src/Palettes.md
All background colors are initialized as white by the boot ROM, however it is a
good idea to initialize all colors yourself, e.g. if implementing
a soft-reset mechanic.
All background colors are initialized as white by the boot ROM, however it is a good idea to initialize all colors yourself, e.g. if implementing a soft-reset mechanic.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, changing it to "by" would make it have a different, also undesirable/confusing meaning. But the "if implementing" thing is odd, and the whole point being made is a bit strange, actually:

When wouldn't it be a good idea to initialise all the colours yourself? If they are all white you can't see anything.
Regardless of the behaviour of the boot ROM, it's a "good idea" to initialise the palettes, because you want your palettes to be the ones being used. At least I think so.

[Note: I don't actually know how true the boot ROM statement is, I assume someone checked.]

I know this behaviour is specific to the BG colours, but this "tip" (and the similar one for OB) could be moved up to the general explanation above -- it's really not specific to the register/s.

Comment thread src/Palettes.md Outdated
Co-authored-by: Quinn <3379314+quinnyo@users.noreply.github.com>
@ISSOtm
Copy link
Copy Markdown
Member

ISSOtm commented May 31, 2026

FWIW, my focus is currently on doing some drawing in order to take a break from programming, so I'll review quinn's contributions to the PR after some days. I'll keep it under two weeks though.

Co-authored-by: Quinn <3379314+quinnyo@users.noreply.github.com>
Comment thread src/Palettes.md Outdated
Fix wildy incorrect arithmetic

Co-authored-by: Eldred Habert <me@eldred.fr>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Rewrite Palettes/BCPD/BGPD to be more intuitive

5 participants