Yes, and this is sadly something OctoPrint won't be able to change. The only way to upload something to your printer's SD when it's connected to your printer is via the (low-bandwidth) serial connection, and in a way that requires each and every command sent to your printer to be acknowledged by it first with an ok before the next line can be sent. The file basically needs to be streamed to your printer the same way as if you were printing it (just with an additional command at the start that tells your printer to not print what follows but instead write it to its SD card).
So if you need to upload files to your printer's SD, unplugging the SD and doing that directly on your PC via a card reader is usually the faster option.
There might be additional options with WiFi-enabled SD cards (such as FlashAIR), but those usually need a patched firmware to work in printers, so this should be considered highly experimental and hence there's no supporting plugin for this yet.