Is it really true that larger pixels are better, or is it really only true that sensing more photons is better?
Specifically, what is the reason multiple sensors with smaller pixels could not perform as well or better if their combine total sensor area was larger?
The L16 is an example. It had issues, but they mostly seemed like implementation problems rather than inherent limitations.
After all their challenges seeing that they still apparently raised 100MM in capital, it would seem to suggest something better is possible. https://spot.light.co
Also they have bigger hardware that today’s smartphones would want, but in principal I’ve never understood why smaller pixels are inherently worse as opposed to total photons collected.
I forgot the specific term that describes this, but basically each pixel on a sensor is going to have a threshold for whether it activates (much like a neuron's action potential). Say its 20 photons. If it gets less than that, it is unable to see anything at all. Meanwhile a larger pixel will see something because it will be able to capture more than 20 photons in a given unit of time. On the other hand, the aggregate of several smaller pixels across many sensors will each fail to reach that threshold and thus see nothing.
The L16 has some low light and dynamic range problems, I believe owing to this very situation.
I believe the term you're looking for is reciprocity failure[1] (or schwarzschild effect), though apparently it doesn't happen with digital sensors - only film. Noise becomes an issue in similar situations, but it's not affected by the number of photons hitting the sensor.
Correction: Shot noise[2] is related to the number of photons hitting the sensor.
You might be referring to the dark current, which is the noise floor for any signal coming out of the CCD. There doesn't need to be an activation threshold of a particular number of photons (maybe there is, though?) for the dark current to reduce your signal to noise ratio.
There are fundamental physical limits at play. Smaller sensors, with smaller pixel size are diffraction limited more quickly. This new sensor has a pixel pitch about one eighth of what a modern APS-C DSLR has. That sensor would be diffraction limited at around f1.4. Most camera systems are more limited by lens resolution than diffraction. Fast lenses like those found on smart phones often have issues with optical aberrations. The lens quality issue is both a financial and technological one. It is possible to create a lens that is diffraction limited, but even for a small sensor it is very expensive in practice.
The difficulty is actually more with maxing it flat, so the phone can be thin. If you have length to spare and only a chip to record, you can get awesome f/2 optics (diffraction limited) with only a Schmidt corrector and otherwise two spherical surfaces and 3 flat ones (not counting the flat front outside facing of the Schmidt plate) with a field-flattened Schmidt camera. The only strong downside apart from the limit on FOV at around 4 degrees for this simple design (slight better designs should allow for up to about 30 degrees) is that it's twice as long as the focal length. It should still allow ultra-tele shots by resting the tube on the shoulder, without anything to actively compensate rotation shake.
The issue is that not all noise scales linear with area. Specifically, a larger pixel can hold more electrons, which means you have more dynamic range. Compare e.g. the CMOSIS CMV12000 with the CMV50000. They are pretty much the same, except that the latter has slightly smaller pixels to fit them all inside the chip. The well depth is lower for the latter, without the readout noise compensating fully. The 12 bit of ADC resolution are not sufficient to handle the full well depth of the CMV12000 though, which means that at base ISO of the sensor, i.e. using the maximum photon capacity of the pixels, you have nearly no noise in the image with the CMV12000. Once you activate the iirc 3x analogue gain, you only have a third of the maximum photon capacity left to use, as the rest just reads out full brightness, but you have about 1 LSB per electron (2^12 == 4096, 15k electrons / 3 == 5000 electrons ~ 4096). So, this means that you have a little noise in the last few bits, not much though.
These sensors are awesome btw, the smaller one can actually handle an about equally expensive (if you count a device to handle the datarate) catadioptic, i.e. using both lens(es) and mirror(s), in this case a spherical mirror, a spherical lens, a non-curved circle of glass, and a slightly wavy ground pane of glass to correct the distortions caused by the spherical surfaces, with a 400mm focal length (about 600mm 35mm film equivalent) and f/2 as far as DOF goes, with a little bit light loss and some slight mirror-optics bokeh due to the sensor being in the middle of the 900 mm long tube, 200mm inner diameter. This is a design that does not work well without being able to mount the sensor right in the middle of the tube, and any alternatives are really hard to handle. Oh, and yes, this design is _sharp_. The geometric error due to the not perfectly compensated spherical surfaces is about 4 micrometer circle, and the diffraction circle of confusion is about the same, so combined there is a sharpness about perfect if you were to use a bayer filter to get color images. The thing would weight about 2 to 3kg, not including power supply, but including all that is needed to provide the raw pixel data 12MP@10bit@300fps via QSFP.
Specifically, what is the reason multiple sensors with smaller pixels could not perform as well or better if their combine total sensor area was larger?
The L16 is an example. It had issues, but they mostly seemed like implementation problems rather than inherent limitations.
After all their challenges seeing that they still apparently raised 100MM in capital, it would seem to suggest something better is possible. https://spot.light.co
Also they have bigger hardware that today’s smartphones would want, but in principal I’ve never understood why smaller pixels are inherently worse as opposed to total photons collected.