agnos.is Forums

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

A 7B model can run on a GPU with just 6GB VRAM, provided the GPU has proper compression storage, which is every gpu named nVidia something.

Technology

1 Posts 1 Posters 0 Views

B This user is from outside of this forum
B This user is from outside of this forum
[email protected]

wrote on last edited by

#1

A 7B model can run on a GPU with just 6GB VRAM, provided the GPU has proper compression storage, which is every gpu named nVidia something.

If the AI assistant runs locally, this is great. If it uses Cloud, welll, that's going to cost money somehow.
1 Reply Last reply

0
System shared this topic on

System shared this topic