0

I have a simple image editing web app that lets a user add an image, zoom and pan it inside a canvas and then send a dithered and indexed version to an ESP32 run screen. But the file is only saved at that point, not displayed.

While testing i noticed that for some images the upload stalls at roughly 20%. It happens when the images are scaled to fill the canvas and are consequently flush to the left edge (no horizontal cropping, the canvas/screen is widescreen). The problem does not occur if I zoom in a little further so the left and right sides are cropped a little. So this special case causes problems down the line that eventually causes an upload to stall. However it's actually not important now, so let's forget it right away.

Everything I show you now is inside saveButton.addEventListener("click", () => {..., which when clicked does all the processing and starts the upload. The actual weird behavior starts here:

This is the form we send with xhr.send(formDataIndex):

const formDataIndex = new FormData();
formDataIndex.append("save_path", savePathIndexed);
formDataIndex.append("original_name", indexedFileName);
formDataIndex.append("index", indexBlob, indexedFileName);

and we get indexBlob from

const indexBlob = new Blob([highResIndexBuffer.buffer], { type: "application/octet-stream" });

and highResIndexBuffer from

for (let y = 0; y < scaledHeight; y++) {
    for (let x = 0; x < scaledWidth; x++) {
        let srcIdx = y * scaledWidth + x;
        let dstIdx = (scaledOffsetY + y) * 2560 + (scaledOffsetX + x);
        highResIndexBuffer[dstIdx] = imageIdxBuffer[srcIdx]; // Copy indexed pixel data
    }
}

which only copies values from imageIdxBuffer to highResIndexBuffer. In the tests I did where the upload fails both buffers have the same size and all values are copied. Now here is where we get imageIdxBuffer:

let imageIdxBuffer = new Uint16Array(scaledWidth * scaledHeight).fill(blackIdx);
applyDithering(scaledCtx, imageIdxBuffer);

applyDithering is external (import { applyDithering, invertImage } from "./dither.js";)and I'm not sure if this might contribute to the issue. The only line in applyDithering where we use this array is

imageIdxBuffer[pixIndex] = paletteIndex;

Anyway for those special cases that I mentioned above using this imageIdxBuffer after applyDithering will stall the upload at around 20%. Actually it will still continue VERY slowly up until maybe 40% where it finally stops. The final upload size is roughly 7MB (always the same) and usually completes in a minute or so. However if I just recreate imageIdxBuffer after applyDithering like so:

imageIdxBuffer = new Uint16Array(scaledWidth * scaledHeight).fill(blackIdx);

or even like so:

imageIdxBuffer.fill(blackIdx);

it doesn't stall. And this is where I'm at a loss. It looks like a data issue, but I don't process any data, I send a blob to the ESP32 which only saves the received data to storage. I have also tried filling the array with random data, which works just fine.

So what happens to imageIdxBuffer that could cause those problems? And how can those problems be copied over to highResIndexBuffer? I have done tests with not copying all data, and indeed below a certain amount of bytes copied there is no issue. Random data also causes no problems.

It's important to note again that the data is not processed in any way! The ESP32 only saves the received data to internal storage.

                while True:
                    chunk = await data.read(1024)
                    if not chunk:
                        break
                    bytes_written += f.write(chunk)

ChatGPT was also no help. It suggested "byte sequences that cause the Blob serialization process to hang" which just sounds stupid. It also suggested memory issues, but I don't see why that would affect copies as well. There are no JS errors by the way.

4
  • 3
    It would be much easier to understand the relationships between all the variables if you just posted the code as a simple minimal reproducible example, rather than describing "We get <some variable> from this code", which is all out of order compared to the real code. Commented Oct 27 at 23:52
  • This sounds like a data-sensitive hardware issue in a network component. Many years ago one of my customers was having a problem downloading files, and it turned out that their modem was failing on a certain byte pattern, which happened to appear in the TCP headers at a certain point in the data stream, and replacing the modem fixed it. Commented Oct 27 at 23:56
  • 1
    Can you reproduce the problem when uploading to a different device? Commented Oct 27 at 23:57
  • Could you link to some doc about the await data.read(1024) method? Commented Oct 28 at 0:28

0

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.