Repository navigation
Is possible an inference with just a RTX 3080 of 10GB? #25
Description
Activity
The required VRAM is around 13GB for full precision inference with a batch size of 1
You can also try Colaboratory for inference: #10
Reacted by David Martín Rius@deepanwayx I suppose full inference precision is 32 bit, correct? If so, did you guys made any test to check whether 16 bit would still deliver good acceptable results?
Reacted by David Martín Rius and YANG627Reacted by David Martín RiusYes, the full inference precision is 32-bit. We did not test with 16-bit inference.
Hello,
I know it is very little memory, but it is what I have by now.
By default, the demo code won't inference because of cuda out of memory. I tried to reduce the batch size of the inference to just 1, but is not enough.
Do you know a way to reduce the memory consumption running the inference?
I know that the best solution is to upgrade the GPU to a RTX 3090/4090/A6000, but before that I would like to try another way if possible.
Thank you!
David Martin Rius