Support for Kubelka-Munk theory "pigment-based" color mixing

[bj-rn]

Similar to mixbox (commercial).

• "Practical Pigment Mixing for Digital Painting" research paper
• mixbox code on github
• MIT implementation of Kubelka-Munk

[waacton]

There are a lot of moving parts to be considered!

Similar to CMYK where ICC profiles are not restricted to 4 colour prints (e.g. CMYK vs CMYKOGV), I'll need to make some decisions around handling N-channel spaces. The paper uses 4 primary pigments, whereas spectral.js appears to use 7.

The paper recommends defaults (Phthalo Blue, Quinacridone Magenta, Hansa Yellow, Titanium White) but ultimately the pigments used are the artist's choice, so I'd want to support customisation. spectral.js uses hardcoded SPDs and I don't know what pigments they refer to. I'm also not familiar enough to know why only SPDs are being used in spectral.js and not absorption and scattering coefficients as described in the paper.

Making things more difficult, it seems like the kind of datasets used in the paper are unavailable, unless I'm missing something?

• Artist Paint Spectral Database - the link to "download Excel spreadsheet" just redirects
• Artist Acrylic Paint Spectral, Colorimetric, and Image Dataset - has a note saying "the spectral reflectance database mentioned in the article is no longer available"

I will probably investigate a first pass at a not-fully-integrated version (as with CMYK), if I can find suitable data.

[tebjan]

Back reference to where the idea started: https://discourse.vvvv.org/t/vl-mixbox/22649

[bj-rn]

Hey, thanks a lot for looking into this! TBH I didn't really read the paper...

Concerning your questions regarding spectral.js maybe this issue provides some answers.

I was able to find the Excel spreadsheet using wayback machine:
https://web.archive.org/web/20190715095836/https://www.rit.edu/cos/colorscience/mellon/Data/Final_artist_database.xlsx

[waacton]

Ah excellent, that makes a bit more sense now, I'll take a look when I get a chance!

There's also more detail about the spectral.js implementation here but I'll need more coffee before I try to follow along 😄

[waacton]

One more thing while it's on my mind: if this does lead to some notion of Pigment in Unicolour, it'd be really convenient if it could be somehow generated from data found on https://infraart.inoe.ro/ (if it's even possible to download the raw data).

[waacton]

I've not forgotten about this, but I wanted to figure out how to support genuine CMYK using ICC profiles first as it's a much more common use case.

There's still work to do there (notably for ICC v4 profiles), but it made me wonder: could "pigment-based" mixing (at least for the suggested defaults Phthalo Blue, Quinacridone Magenta, Hansa Yellow, Titanium White in the first instance) be captured as an ICC lookup transform (4-colour to LAB D50 and reverse)?

I don't know anything about creating an ICC profile but it might be a shortcut to supporting a first pass without customisation of pigments. The ICC have a registry of CMYK profiles that each has its characterisation data reference e.g. FOGRA39. Perhaps with similar data for pigment-based mixing, the conversion could be distilled down to an elaborate series of lookup tables for convenience🤔?

[bj-rn]

I haven't a clue, tbh.

[waacton]

I've spent some time looking into this on the pigments branch and wanted to record my thoughts while they're fresh in my mind.

I'm confident I can support some level of Kubelka-Munk mixing but I'm not yet feeling comfortable about it, and I'll need to ruminate and experiment some more.

I've got a notion of a Pigment which is created using absorption (k) and scattering (s) spectral data

• this is the data format provided in the (no longer available) Artist Paint Spectral Database and used by the Mixbox paper Practical Pigment Mixing for Digital Painting
• I think this is the two-constant Kubelka-Munk theory
• k and s are used to calculate reflectance at each wavelength
• reflectance is corrected using the Saunderson correction, though a variant that assumes spectrophotomeric measurements were made in SPEX mode (specular excluded)
  • I don't know if this is a safe assumption to make (potentially since it mimics how eyes perceive colour), or if I'd need to expose SPEX mode as a parameter
• reflectance is used alongside spectral power distribution to calculate an XYZ colour
• however, k and s are likely to be a different range and interval of wavelengths
  • right now I'm simply linearly interpolating for missing values
  • again, I don't know if that's a reasonable assumption to make
• with all these caveats in place, I can show pigments mixed in a similar way to the images in the Mixbox paper (see below)
  • it's not a perfect match but I don't know the details of their implementation of K-M, or what processing has been done on the images themselves
• however, the colours created from pigments are quite different from the LAB values reported by the paint manufacturer
  • no idea if I should be expecting them to, or whether the above methodology is simply too different to the manufacturer's

In the following images:

1. Left is the K-M demo image from the Mixbox paper
2. Middle is the real paint mixing image from the Mixbox paper
3. Right is my own implementation of K-M, calculated using k and s from Artist Paint Spectral Database (with Saunderson correction constants also from the database, assuming SPEX mode)

Quinacridone Magenta to Titanium White	Phthalo Blue RS to Titanium White	Cobalt Blue to Hansa Yellow Opaque

I could also/instead have a notion of a Pigment which is created using absorption-to-scattering (k/s) ratios

• this is the data format provided in HB 10 mil Drawdowns over White, apparently directly from the manufacturer and found on here
• I think this is the single-constant Kubelka-Munk theory
• k/s is used to calculate reflectance at each wavelength, which can also be corrected if constants are provided, and are again unlikely to be the same wavelength set as the SPD so interpolation may be needed
• The results look very different, but the colours created from these pigments match the manufacturer's LAB values very well
• Sadly there is no Titanium White or Hansa Yellow Opaque data to make a comparison with the reference images from the Mixbox two-constant approach

So the big questions for me to consider:
1. Does a Pigment accept two-constant k and s data, single-constant k/s data, or both?
2. Does Saunderson correction always assume SPEX mode, or is that yet another variable to consider?
3. Is my understanding of all the jargon and terminology even correct?
4. Will any user realistically have any of this data to provide?
5. Is a small dataset of predefined pigments (and Saunderson correction constants) enough to make this feature more approachable?

Finally, Mixbox goes a step further which is out of scope for Unicolour: "unmixing" RGB colours back into some approximation of a pigment for the interactive painting. As far as I understand this is done by precomputing a model based on 4 pigments, and a different choice of pigments would require recomputing the model. I don't see this as feasible at runtime for a library.

[waacton]

Finally, Mixbox goes a step further which is out of scope for Unicolour: "unmixing" RGB colours back into some approximation of a pigment for the interactive painting. As far as I understand this is done by precomputing a model based on 4 pigments, and a different choice of pigments would require recomputing the model. I don't see this as feasible at runtime for a library.

Before I shut the door on this, I'll investigate generating reflectance curves from RGB through this method: http://scottburns.us/reflectance-curves-from-srgb-10/.

(Given there's no "true" answer and this is finding one of infinite possibilities that would generate a target RGB, any implementation would probably end up as some kind of utility outside of the core Unicolour functions. To let people imagine "but what if I had a pigment that actually looked like #FF1493?")

If possible, this would necessitate at least the single-constant implementation (a reflectance value per wavelength, not a k and an s per wavelength). But if the Mixbox-style two-constant implementation seems more realistic and desirable, I might well try to support both, even if the two-constant variant is harder to obtain data for and much less likely to be used outside of a predefined dataset that Unicolour provides.

I'll see where this leads anyway.